Debug reading from a topic

When developing streaming queries, it is useful to quickly see what data is coming into a topic without creating a full streaming query. To do this, you can run a regular SELECT with the STREAMING = "TRUE" parameter.

Warning

This method is intended only for debugging and checking data in a topic. For production use, create streaming queries using CREATE STREAMING QUERY.

Note

In the examples:

ext_source — a pre-created external data source.
input_topic — a local or external topic (see local and external topics in streaming queries).

Reading raw data

The simplest way is to read messages in raw format, without parsing the schema:

SELECT
    Data
FROM
    input_topic -- or external topic ext_source.input_topic
WITH (
    FORMAT = raw,
    SCHEMA = (
        Data String
    ),
    STREAMING = "TRUE"
)
LIMIT 1

The LIMIT parameter is required — without it, the query will not complete, as it will wait for new messages indefinitely.

Reading with JSON parsing

If the data in the topic is stored in JSON format, you can immediately parse it by fields:

SELECT
    *
FROM
    input_topic -- or external topic ext_source.input_topic
WITH (
    FORMAT = json_each_row,
    SCHEMA = (
        Time String NOT NULL,
        Level String NOT NULL,
        Host String NOT NULL
    ),
    STREAMING = "TRUE"
)
LIMIT 5

Debug reading from a topic

Reading raw data

Reading with JSON parsing

See also

Was the article helpful?