Writing messages to a topic
The topic write command writes messages to a topic from a file or stdin:
ydb [connection options] topic write <topic-path> \
[--file STR] [--format STR] [--transform STR] \
[additional parameters...]
where [connection options] are database connection options
Parameters
Basic parameters
<topic-path>: Topic path, the only required parameter.
--file VAL (-f VAL): Read a stream of incoming messages and write them to a topic from the specified file. If not set, messages are read from stdin.
--format STR: Format of the incoming message stream.
-
Supported formats
Name Description single-message
(default)The entire input stream is treated as a single message to be written to the topic. newline-delimitedA stream at the input contains multiple messages delimited with the 0x0Anewline character.
--transform VAL: Method for transforming messages.
- Defaults to
none. - Possible values:
base64: Decode each message in the input stream from Base64 and write the output to the topic. If decoding fails, the command is aborted with an error.
none: Write the contents of a message from the input stream to the topic byte by byte without transforming them.
Additional parameters
| Name | Description |
|---|---|
--delimiter STR |
Delimiter byte. The input stream is delimited into messages with the specified byte. Specified only if no --format is set. Specified as an escaped string. |
--message-group-id STR |
Message group string ID. If not set, all messages generated from the input stream are assigned the same ID value as a hexadecimal string representation of a random three-byte integer. |
--codec STR |
Codec used for message compression on the client before sending them to the server. Possible values: RAW (no compression, default), GZIP, and ZSTD. Compression causes higher CPU utilization on the client when reading and writing messages, but usually lets you reduce the volume of data transferred over the network and stored. When consumers read messages, they're automatically decompressed with the codec used when writing them, without specifying any special options. Make sure the specified codec is listed in the topic parameters as supported. |
Examples
Note
The examples use the quickstart profile. To learn more, see Creating a profile to connect to a test database.
All the examples given below use a topic named topic1.
-
Writing a terminal input to a single message Once the command is run, you can type any multi-line text and press
Ctrl+Dto input it.ydb -p quickstart topic write topic1 -
Writing the contents of the
message.binfile to a single message compressed with the GZIP codecydb -p quickstart topic write topic1 -f message.bin --codec GZIP -
Writing the contents of the
example.txtfile delimited into messages line by lineydb -p quickstart topic write topic1 -f example.txt --format newline-delimited -
Writing a resource downloaded via HTTP and delimited into messages with tab characters
curl http://example.com/resource | ydb -p quickstart topic write topic1 --delimiter "\t"