ALTER TOPIC

Using the ALTER TOPIC operator, you can change the settings of a topic, as well as add, modify, or delete a reader.

General command syntax:

ALTER TOPIC topic_path action1, action2, ..., actionN;
  • You can specify several action statements for a consumer. However, the settings applied by them shouldn't
    repeat.

Working with a topic

Set topic parameters

Using the SET (option = value[, ...]) action, you can update your topic settings.

General command syntax:

ALTER TOPIC topic_path SET (option = value[, ...]);
  • option and value — the topic parameter and its value.

Topic parameters

  • metering_mode — resource metering method (RESERVED_CAPACITY — by dedicated resources or REQUEST_UNITS — by actual usage). Relevant for topics in serverless databases. Value type — String.
  • min_active_partitions — the minimum number of active partitions of the topic. Auto-partitioning will not reduce the number of active partitions below this value. Type — integer, default value — 1.
  • max_active_partitions — the maximum number of active partitions of the topic. Auto-partitioning will not increase the number of active partitions above this value. Type — integer, default is min_active_partitions.
  • retention_period: Data retention period in the topic. Value type: Interval, default value: 18h.
  • retention_storage_mb: Limit on the maximum disk space occupied by the topic data. When this value is exceeded, the older data is cleared, like under a retention policy. The consumed space may exceed the set value when autopartitioning is enabled. Value type: integer, default value: 0 (no limit).
  • partition_write_burst_bytes — the size of the write quota reserve for a partition to handle write bursts. When set to 0, the actual write_burst value is taken to be equal to the quota value (which allows write bursts of up to 1 second). Value type — integer, default value: 0.
  • partition_write_speed_bytes_per_second: Maximum allowed write speed per partition. If a write speed for a given partition exceeds this value, the write speed will be capped. Value type: integer, default value: 2097152 (2MB).
  • auto_partitioning_strategy — auto-partitioning mode.
    Allowed values: paused, scale_up, default value — disabled.
  • auto_partitioning_up_utilization_percent — defines the partition load threshold as a percentage of the maximum write speed, upon reaching which an automatic increase in the number of partitions will be initiated. Value type — integer, default value — 80.
  • auto_partitioning_stabilization_window — defines the time interval during which the load level must remain above the set threshold (auto_partitioning_up_utilization_percent) before the number of partitions is automatically increased. Value type — Interval, default value — 5m.

The following command will change the data retention time in the topic and the write speed quota for 1 partition:

ALTER TOPIC `my_topic` SET (
    retention_period = Interval('PT36H'),
    partition_write_speed_bytes_per_second = 3000000
);

Change autopartitioning strategies for the topic

The following command enables auto-partitioning:

ALTER TOPIC `my_topic` SET (
    min_active_partitions = 1,
    max_active_partitions = 5,
    auto_partitioning_strategy = 'scale_up'
);

The following command pauses the topic autopartitioning:

ALTER TOPIC `my_topic` SET (
    auto_partitioning_strategy = 'paused'
);

The following command unpauses the topic autopartitioning:

ALTER TOPIC `my_topic` SET (
    auto_partitioning_strategy = 'scale_up'
);

Working with a reader

Add a reader

ADD CONSUMER — the action adds readers for the topic.

General command syntax:

ALTER TOPIC topic_path ADD CONSUMER consumer_name [WITH (option = value[, ...])];
  • option and value — the reader parameter and its value.

Reader parameters:

  • type — reader type. Possible values: STREAMING and SHARED. Default value: STREAMING.
  • important — important reader flag. Data from the topic will not be deleted until all important readers process it. Value type: boolean, default value: false.
  • availability_period — determines the time messages are available to the reader. The option allows extending the message retention time in the topic beyond retention_period, up to availability_period, if the reader does not confirm their processing. Value type: Interval. Incompatible with the important parameter. No default value.
  • read_from — determines the point in time from which the reader will receive data. Messages written before this point will not be received by the reader. Value type: Datetime, Timestamp, or integer (Unix timestamp as a number). Default value: 0 (reading from the earliest available point in time in the topic).

Reader parameters available only for a shared (common) reader:

  • keep_messages_order — preserves the order of message reading. If the value is true, the order of message processing within a single message group is guaranteed. Default value: false.
  • default_processing_timeout — message processing time. If message processing is not confirmed within this time and the processing time is not extended, the message will return to the queue and be sent for reprocessing. Default value: Interval('PT30S').
  • max_processing_attempts — maximum number of processing attempts for a single message (value type: integer). The option is supported only with dead_letter_policy = move or dead_letter_policy = delete. Default value: 1000.
  • dead_letter_policy — action to take with the message if all processing attempts have failed (value type: String). Possible values: delete, move, none. Default value: none.
  • dead_letter_queue — DLQ topic name (value type: String). Required for dead_letter_policy = move and not supported for dead_letter_policy = none or delete.

The following command will add a reader with default settings to the topic:

ALTER TOPIC `my_topic` ADD CONSUMER my_consumer;

The following command will add an important reader to the topic:

ALTER TOPIC `my_topic` ADD CONSUMER my_consumer2 WITH (important = true);

The following command will add a shared (common) reader to the topic:

ALTER TOPIC `my_topic`
    ADD CONSUMER my_shared_consumer WITH (
        type = 'shared',
        keep_messages_order = false,
        default_processing_timeout = Interval('PT30S'),
        max_processing_attempts = 3,
        dead_letter_policy = 'move',
        dead_letter_queue = 'my_dlq_topic'
    );

Set reader parameters

ALTER CONSUMER consumer_name SET (option = value[, ...]) — the action sets the parameters of the topic reader.

General command syntax:

ALTER TOPIC topic_path ALTER CONSUMER consumer_name SET (option = value[, ...]);
  • option and value — the reader parameter and its value.

The following command will make the reader important:

ALTER TOPIC `my_topic` ALTER CONSUMER my_consumer SET (important = true);

A single command can contain multiple ALTER CONSUMER actions, and their settings must not be duplicated:

ALTER TOPIC `my_topic`
    ALTER CONSUMER my_consumer SET (availability_period = Interval('PT48H'))
    ALTER CONSUMER my_consumer SET (read_from = 0);

Delete a reader

DROP CONSUMER — the action deletes the topic reader.

General command syntax:

ALTER TOPIC topic_path DROP CONSUMER consumer_name;

The following command will delete the reader named old_consumer:

ALTER TOPIC `my_topic` DROP CONSUMER old_consumer;
Previous
ALTER VIEW
Next
ALTER STREAMING QUERY