Local and external topics in streaming queries

Streaming queries read events from topics and can write results back to topics. Messages can come from or go to a topic in the same database where the query runs, or in another YDB database.

All streaming query scenarios work the same for local and external topics. A single query can read a local topic, write to an external one, and vice versa.

Local topics

Local topics are topics created in the same YDB database as the streaming query.

In query text you refer to them by short name, the same way as to a table in the current database:

SELECT * FROM input_topic WITH (FORMAT = json_each_row, SCHEMA = (...));

INSERT INTO output_topic SELECT ...;

External topics

External topics are topics in another YDB database.

Access from a streaming query goes only through a pre-created external data source with source type YDB. Create it with CREATE EXTERNAL DATA SOURCE; use secrets when authentication is required.

After you create a source, for example named ext_source, a topic input_topic in the external database is referenced as:

SELECT * FROM ext_source.input_topic WITH (FORMAT = json_each_row, SCHEMA = (...));

The name ext_source in the documentation is conventional — your source may have a different name; it must match in CREATE EXTERNAL DATA SOURCE and in the prefix before the topic name.

See also

Previous
Streaming queries
Next
Common patterns