Query execution

YDB provides a unified interface for executing queries in the YQL language and a unified distributed execution engine. The same syntax and mechanisms allow accessing data of different nature: from rows in database tables to unbounded streams of events in topics and data in external systems.

From the user's perspective, queries are executed against three types of entities:

• database tables — transactional (OLTP) and analytical (OLAP) queries to data stored in YDB;
• external data sources — federated queries to data located outside YDB;
• topics — queries to unbounded data streams: streaming queries and infinite queries to topics.

The general query processing flow, as well as basic concepts — sessions, transactions, retries, query language, and result sets — are described in the Query execution process section.

Queries to database tables

The main scenario is executing queries against tables stored in YDB. The unified interface can efficiently handle a wide range of workloads — from high-load transactional OLTP queries to complex analytical OLAP queries.

Queries to external data sources (federated queries)

Federated queries allow accessing data located in external systems without moving that data into YDB. Using YQL queries, you can read data from external DBMSs and object storages (S3), and also combine them with data in YDB tables.

For more details, see the Federated queries section.

Queries to topics

Queries to topics allow processing unbounded data streams. Since the data stream is infinite, such a query does not complete after receiving a result, but runs until explicitly cancelled. There are two variants of such queries.

Streaming queries

Streaming queries are the primary way of streaming data processing in production. They are created as persistent schema objects (STREAMING QUERY), read messages from topics as they arrive, write results to output topics or tables, and automatically recover from failures using checkpoints.

For more details, see the Streaming queries section and the description of CREATE STREAMING QUERY.

Queries to topics in table mode

You can also read data from a topic using a regular SELECT by specifying STREAMING = "TRUE" in the WITH clause. Without the LIMIT limit, such a query runs indefinitely, returning results to the client as messages arrive. Unlike streaming queries, it does not create a persistent schema object and does not recover from failures, so it is intended primarily for debugging and checking data in a topic.

For more details, see the Streaming read from a topic section.