log_config

The log_config section controls how YDB server processes and manages its logs. It allows you to customize logging levels for different components, as well as global log formats and output methods.

Overview

Logging is a critical part of the YDB observability system. The log_config section lets you configure various aspects of logging, including:

• Default logging level
• Component-specific logging levels
• Log output format
• Integration with system logs or external logging services

Log Output Methods

• To stderr: by default, YDB sends all logs to stderr.
• To file: logs can be written to a file using the backend_file_name parameter.
• To syslog: when the sys_log: true parameter is enabled, logs are redirected to the syslog and stop being output to stderr. Logs are sent using the /dev/log socket.
• To Unified Agent: when configuring the uaclient_config section, logs are sent to Unified Agent and stop being output to stderr.

When both sys_log and uaclient_config are enabled simultaneously, logs will be sent to both syslog and Unified Agent. If you need to continue outputting logs to stderr while using other methods, activate sys_log_to_stderr: true.

Configuration Options

Entry Objects

The entry field contains an array of objects with the following structure:

UAClientConfig Object

The uaclient_config field configures integration with Unified Agent:

Log Levels

YDB uses the following log levels, listed from the highest to the lowest severity:

Examples

Basic Configuration

log_config:
  default_level: 5  # NOTICE
  format: "full"

This configuration outputs logs to stderr with logging level NOTICE and above.

File Output Configuration

log_config:
  default_level: 5  # NOTICE
  format: "full"
  backend_file_name: "/var/log/ydb/ydb.log"

This configuration sends logs to a file while maintaining the default logging level of NOTICE.

Syslog Output Configuration

log_config:
  default_level: 5  # NOTICE
  sys_log: true
  sys_log_service: "ydb"
  format: "full"

This configuration sends logs to syslog with the service name "ydb".

Setting Per-Component Log Levels

log_config:
  default_level: 5  # NOTICE
  entry:
    - component: "SCHEMESHARD"
      level: 7  # DEBUG
    - component: "TABLET_MAIN"
      level: 6  # INFO
  backend_file_name: "/var/log/ydb/ydb.log"

Sampling Configuration

log_config:
  default_level: 5  # NOTICE
  default_sampling_level: 7  # DEBUG
  default_sampling_rate: 10  # Log every 10th message between NOTICE and DEBUG
  entry:
    - component: "BLOBSTORAGE"
      sampling_level: 8  # TRACE
      sampling_rate: 100  # Log every 100th message between NOTICE and TRACE

This configuration sets up sampling for logs. With default settings, every 10th message with priority between NOTICE and DEBUG will be logged. For the BLOBSTORAGE component, every 100th message with priority between NOTICE and TRACE will be logged.

JSON Format Configuration

log_config:
  default_level: 5  # NOTICE
  format: "json"
  cluster_name: "production-cluster"
  uaclient_config:
    uri: "[fd53::1]:16400"
    grpc_max_message_size: 4194304
    log_name: "ydb_logs"

This configuration outputs logs in JSON format and sends them to Unified Agent.

Notes

• Log levels are specified in the configuration as numeric values, not strings. Use the table above to map between numeric values and their meanings.
• If the backend_file_name parameter is specified, logs are written to this file. If the sys_log parameter is true, logs are sent to the system logger.
• The format parameter determines how log entries are formatted. The "full" format includes all available information, "short" provides a more compact format, and "json" outputs logs in JSON format, which is convenient for parsing by logging services.
• The internal log buffer has the following size limits:
  • Default total size: 10MB (10 * 1024 * 1024 bytes)
  • Default grain size: 64KB (1024 * 64 bytes)
  • Maximum message size: 1KB (1024 bytes)

See Also

• Reference on YDB observability
• Metrics reference
• Tracing in YDB
• Audit log