Setting Up YDB Cluster Monitoring

This page explains how to set up monitoring for a YDB cluster.

YDB provides numerous system state metrics. Instant metric values can be viewed in the web interface:

http://<ydb-server-address>:<ydb-port>/counters/

where:

• <ydb-server-address> – YDB server address.

  For a local single-node YDB cluster started using the Quick start instructions, use the address localhost.

• <ydb-port> – YDB port. Default value: 8765.

Related metrics are grouped into subgroups (for example counters auth). To view metric values for a specific subgroup only, navigate to a URL of the following form:

http://<ydb-server-address>:<ydb-port>/counters/counters=<servicename>/

• <servicename> — metric subgroup name.

For example, server hardware resource utilization data is available at the following URL:

http://<ydb-server-address>:<ydb-port>/counters/counters=utils

To collect metric values, you can use the popular open-source tool Prometheus or any other system that supports this format. YDB metric values in Prometheus format are available at URLs of the following form:

http://<ydb-server-address>:<ydb-port>/counters/counters=<servicename>/prometheus

• <servicename> — metric subgroup name.

Data can be visualized using any system that supports the Prometheus format, such as Grafana, Zabbix or Amazon CloudWatch:

Setting Up Monitoring with Prometheus and Grafana

To set up YDB cluster monitoring using Prometheus and Grafana:

1. Install Prometheus.

2. Edit the Prometheus configuration files:

   1. In the targets section of ydbd-storage.yml, specify the addresses of all YDB cluster servers and the ports of storage nodes running on the servers.

      - labels:
          container: ydb-static
        targets:
        - "ydb-s1.example.com:8765"
        - "ydb-s2.example.com:8765"
        - "ydb-s3.example.com:8765"
      

      For a local single-node YDB cluster, specify one address in the targets section:

      - labels:
          container: ydb-static
        targets:
        - "localhost:8765"
      

   2. In the targets section of ydbd-database.yml, specify the addresses of all YDB cluster servers and the ports of all database nodes running on the servers.

      - labels:
          container: ydb-dynamic
        targets:
        - "ydb-s1.example.com:31002"
        - "ydb-s1.example.com:31012"
        - "ydb-s1.example.com:31022"
        - "ydb-s2.example.com:31002"
        - "ydb-s2.example.com:31012"
        - "ydb-s2.example.com:31022"
        - "ydb-s3.example.com:31002"
        - "ydb-s3.example.com:31012"
        - "ydb-s3.example.com:31022"
      

      For a local single-node YDB cluster, specify one address in the targets section:

      - labels:
          container: ydb-dynamic
        targets:
        - "localhost:8765"
      

   3. If necessary, in the tls_config section of prometheus_ydb.yml, specify the Certificate Authority (CA) certificate that signed the other TLS certificates of the YDB cluster:

      scheme: https
      tls_config:
          ca_file: '<ydb-ca-file>'
      

3. Start Prometheus using prometheus_ydb.yml as a configuration file.

4. Install and start Grafana.

5. Create a data source with type prometheus in Grafana and connect it to the running Prometheus instance.

6. Upload YDB dashboards to Grafana.

   You can upload dashboards using the Grafana UI Import tool or run the script. Note that the script uses basic authentication in Grafana. For other cases, modify the script.

   See the Grafana dashboards reference.