log_config

The log_config section controls how the YDB server process handles and manages its logs. It allows you to configure logging levels for various components, as well as global log formats and output methods.

Overview

Logging is a critical part of the YDB observability system. The log_config section allows you to configure various aspects of logging, including:

• default logging level;
• logging levels for specific components;
• log output format;
• integration with syslog or external logging services.

Log Output Methods

• To stderr: by default, YDB sends all logs to stderr.
• To file: logs can be written to a file using the backend_file_name parameter.
• To syslog: when the sys_log: true parameter is enabled, logs are redirected to the system syslog and stop being output to stderr. Logs are sent using the /dev/log socket.
• To Unified Agent: when configuring the uaclient_config section, logs are sent to Unified Agent and stop being output to stderr.

When both sys_log and uaclient_config are enabled simultaneously, logs will be sent to both syslog and Unified Agent. If you need to continue outputting logs to stderr while using other methods, activate sys_log_to_stderr: true.

Configuration Options

Entry Objects

The entry field allows you to set logging parameters for individual components. It contains an array of objects with the following structure:

UAClientConfig Object

The uaclient_config field configures integration with Unified Agent:

Log Levels

YDB uses the following log levels, listed from the highest to the lowest severity:

Examples

Basic Configuration

log_config:
  default_level: 5  # NOTICE
  format: "full"

This configuration outputs logs to stderr with logging level NOTICE and above.

Configuration with File Output

log_config:
  default_level: 5  # NOTICE
  format: "full"
  backend_file_name: "/var/log/ydb/ydb.log"

This configuration sends logs to a file while maintaining the default logging level of NOTICE.

Configuration with Syslog Output

log_config:
  default_level: 5  # NOTICE
  sys_log: true
  sys_log_service: "ydb"
  format: "full"

This configuration sends logs to syslog with the service name "ydb".

Setting Log Levels for Individual Components

log_config:
  default_level: 5  # NOTICE
  entry:
    - component: "SCHEMESHARD"
      level: 7  # DEBUG
    - component: "TABLET_MAIN"
      level: 6  # INFO
  backend_file_name: "/var/log/ydb/ydb.log"

Enabling Transaction Lock Invalidation Diagnostics (TLI)

To diagnose transaction locks invalidated errors, enable logging for the TLI component:

log_config:
  entry:
    - component: "TLI"
      level: 6  # INFO

Once enabled, the server writes logs on every lock conflict, including identifiers and query texts of both victims and violators. For details on the log entry format, event correlation, and the find_tli_chain utility, see TLI Logging.

Sampling Configuration

log_config:
  default_level: 5  # NOTICE
  default_sampling_level: 7  # DEBUG
  default_sampling_rate: 10  # Log every 10th message between NOTICE and DEBUG
  entry:
    - component: "BLOBSTORAGE"
      sampling_level: 8  # TRACE
      sampling_rate: 100  # Log every 100th message between NOTICE and TRACE

This configuration sets up log sampling. With default settings, every 10th message with priority between NOTICE and DEBUG will be logged. For the BLOBSTORAGE component, every 100th message with priority between NOTICE and TRACE will be logged.

JSON Format Configuration

log_config:
  default_level: 5  # NOTICE
  format: "json"
  cluster_name: "production-cluster"
  uaclient_config:
    uri: "[fd53::1]:16400"
    grpc_max_message_size: 4194304
    log_name: "ydb_logs"

This configuration outputs logs in JSON format and sends them to Unified Agent.

Notes

• Log levels are specified in the configuration as numeric values, not strings. Use the table above to map between numeric values and their meanings.
• If the backend_file_name parameter is specified, logs are written to this file. If the sys_log parameter is true, logs are sent to the system logger.
• The format parameter determines how log entries are formatted. The "full" format includes all available information, "short" provides a more compact format, and "json" outputs logs in JSON format, which is convenient for parsing by logging services.
• The internal log buffer has the following size limits:
  • Default total size: 10MB (10 * 1024 * 1024 bytes)
  • Default grain size: 64KB (1024 * 64 bytes)
  • Maximum message size: 1KB (1024 bytes)

See Also

• Reference on YDB observability
• Metrics reference
• Tracing in YDB
• Audit log