Debug reads from a topic

When developing streaming queries, it is often useful to inspect what arrives in a topic without creating a full streaming query. Run a regular SELECT with STREAMING = TRUE.

Warning

For debugging and inspection only. For production, create streaming queries with CREATE STREAMING QUERY.

Note

In the examples, ydb_source is a pre-created external data source, and topic_name / input_topic are topics available through it.

Raw reads

Simplest option — read messages in raw format without parsing:

SELECT
    Data
FROM
    ydb_source.topic_name
WITH (
    FORMAT = raw,
    SCHEMA = (
        Data String
    ),
    STREAMING = TRUE
)
LIMIT 1

LIMIT is required; without it the query never completes because it waits for new messages indefinitely.

JSON parsing

If the topic stores JSON, parse fields directly:

SELECT
    *
FROM
    ydb_source.topic_name
WITH (
    FORMAT = json_each_row,
    SCHEMA = (
        Time String NOT NULL,
        Level String NOT NULL,
        Host String NOT NULL
    ),
    STREAMING = TRUE
)
LIMIT 5

See also

Previous
Quickstart: read and write topics
Next
Troubleshooting