Deploying a YDB cluster on virtual or bare-metal servers

In protected mode, traffic between cluster nodes and between the client and cluster is encrypted using the TLS protocol.

1. Create a key and a certificate for the Certification Authority (CA):

   1. Create a directory named secure to store the CA key and one named certs for certificates and node keys:

      mkdir secure
      mkdir certs
      

   2. Create a configuration file named ca.cnf with the following contents:

      [ ca ]
      default_ca = CA_default
      
      [ CA_default ]
      default_days = 365
      database = index.txt
      serial = serial.txt
      default_md = sha256
      copy_extensions = copy
      unique_subject = no
      
      [ req ]
      prompt=no
      distinguished_name = distinguished_name
      x509_extensions = extensions
      
      [ distinguished_name ]
      organizationName = YDB
      commonName = YDB CA
      
      [ extensions ]
      keyUsage = critical,digitalSignature,nonRepudiation,keyEncipherment,keyCertSign
      basicConstraints = critical,CA:true,pathlen:1
      
      [ signing_policy ]
      organizationName = supplied
      commonName = optional
      
      [ signing_node_req ]
      keyUsage = critical,digitalSignature,keyEncipherment
      extendedKeyUsage = serverAuth,clientAuth
      
      # Used to sign client certificates.
      [ signing_client_req ]
      keyUsage = critical,digitalSignature,keyEncipherment
      extendedKeyUsage = clientAuth
      

   3. Create a CA key:

      openssl genrsa -out secure/ca.key 2048
      

      Save this key separately, you'll need it for issuing certificates. If it's lost, you'll have to reissue all certificates.

   4. Create a private Certificate Authority (CA) certificate:

      openssl req -new -x509 -config ca.cnf -key secure/ca.key -out certs/ca.crt -days 1830 -batch
      

   5. Create a text database and an OpenSSL certificate index file:

      touch index.txt
      echo 01 >serial.txt
      

2. Create keys and certificates for the cluster nodes:

   1. Create a node.cnf configuration file with the following contents:

      # OpenSSL node configuration file
      [ req ]
      prompt = no
      distinguished_name = distinguished_name
      req_extensions = extensions
      
      [ distinguished_name ]
      organizationName = YDB
      
      [ extensions ]
      subjectAltName = DNS:<node>.<domain>
      

   2. Create a certificate key:

      openssl genrsa -out certs/node.key 2048
      

   3. Create a Certificate Signing Request (CSR):

      openssl req -new -sha256 -config node.cnf -key certs/node.key -out node.csr -batch
      

   4. Create a node certificate:

      openssl ca -config ca.cnf -keyfile secure/ca.key -cert certs/ca.crt -policy signing_policy \
      -extensions signing_node_req -out certs/node.crt -outdir certs/ -in node.csr -batch
      

      Create similar certificate-key pairs for each node.

   5. Create certificate directories on each node:

      sudo mkdir /opt/ydb/certs
      sudo chown -R ydb:ydb /opt/ydb/certs
      sudo chmod 0750 /opt/ydb/certs
      

   6. Copy the certificates and node keys to the installation folder:

      sudo -u ydb cp certs/ca.crt certs/node.crt certs/node.key /opt/ydb/certs/
      

3. Prepare a configuration file for YDB:

   1. Download a sample config for the appropriate failure model of your cluster:

      • block-4-2: For a single-data center cluster.
      • mirror-3dc: For a cross-data center cluster consisting of 9 nodes.
      • mirror-3dc-3nodes: For a cross-data center cluster consisting of 3 nodes.

   2. In the host_configs section, specify all disks and their types on each cluster node. Possible disk types:

      • ROT: Rotational, HDD.
      • SSD: SSD or NVMe.

      host_configs:
      - drive:
        - path: /dev/disk/by-partlabel/ydb_disk_ssd_01
          type: SSD
        host_config_id: 1
      

   3. In the hosts section, specify the FQDN of each node, their configuration and location in a data_center or rack:

      hosts:
      - host: node1.ydb.tech
        host_config_id: 1
        walle_location:
          body: 1
          data_center: 'zone-a'
          rack: '1'
      - host: node2.ydb.tech
        host_config_id: 1
        walle_location:
          body: 2
          data_center: 'zone-b'
          rack: '1'
      - host: node3.ydb.tech
        host_config_id: 1
        walle_location:
          body: 3
          data_center: 'zone-c'
          rack: '1'
      

   4. Enable user authentication (optional).

      If you plan to use authentication and user access differentiation features in the YDB cluster, add the following parameters to the domains_config section:

      domains_config:
        security_config:
          enforce_user_token_requirement: true
          monitoring_allowed_sids:
          - "root"
          - "ADMINS"
          - "DATABASE-ADMINS"
          administration_allowed_sids:
          - "root"
          - "ADMINS"
          - "DATABASE-ADMINS"
          viewer_allowed_sids:
          - "root"
          - "ADMINS"
          - "DATABASE-ADMINS"
      

4. Enable the traffic encryption mode in the YDB configuration file.

   In the interconnect_config and grpc_config sections, specify the path to the certificate, key, and CA certificate:

   interconnect_config:
       start_tcp: true
       encryption_mode: OPTIONAL
       path_to_certificate_file: "/opt/ydb/certs/node.crt"
       path_to_private_key_file: "/opt/ydb/certs/node.key"
       path_to_ca_file: "/opt/ydb/certs/ca.crt"
   
   grpc_config:
       cert: "/opt/ydb/certs/node.crt"
       key: "/opt/ydb/certs/node.key"
       ca: "/opt/ydb/certs/ca.crt"
   

On every node, create a /etc/systemd/system/ydbd-storage.service configuration file with the following contents:

[Unit]
Description=YDB storage node
After=network-online.target rc-local.service
Wants=network-online.target
StartLimitInterval=10
StartLimitBurst=15

[Service]
Restart=always
RestartSec=1
User=ydb
PermissionsStartOnly=true
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=ydbd
SyslogFacility=daemon
SyslogLevel=err
Environment=LD_LIBRARY_PATH=/opt/ydb/lib
ExecStart=/opt/ydb/bin/ydbd server --log-level 3 --syslog --tcp --yaml-config  /opt/ydb/cfg/config.yaml --grpc-port 2135 --ic-port 19001 --mon-port 8765 --node static
LimitNOFILE=65536
LimitCORE=0
LimitMEMLOCK=3221225472

[Install]
WantedBy=multi-user.target

Run YDB storage on each node:

sudo systemctl start ydbd-storage

To execute administrative commands (including cluster initialization, database creation, disk management, and others) in a cluster with user authentication mode enabled, you must first get an authentication token using the YDB CLI client version 2.0.0 or higher. You must install the YDB CLI client on any computer with network access to the cluster nodes (for example, on one of the cluster nodes) by following the installation instructions.

When the cluster is first installed, it has a single root account with a blank password, so the command to get the token is the following:

ydb -e grpc://<node1.ydb.tech>:2135 -d /Root \
    --user root --no-password auth get-token --force >token-file

Any cluster server can be specified as a connection server (the -e or --endpoint parameter).

If TLS traffic protection was enabled, use the protected grpcs protocol instead of the grpc protocol in the command above and additionally specify the path to the CA certificate in the --ca-file parameter. For example:

ydb -e grpcs://<node1.ydb.tech>:2135 -d /Root --ca-file /opt/ydb/certs/ca.crt \
     --user root --no-password auth get-token --force >token-file

If the command above is executed successfully, the authentication token will be written to token-file. You need to copy this file to the cluster node on which you intend to run the cluster initialization and database creation commands later. Next, run the commands on this cluster node:

export LD_LIBRARY_PATH=/opt/ydb/lib
/opt/ydb/bin/ydbd -f token-file admin blobstorage config init --yaml-file  /opt/ydb/cfg/config.yaml
echo $?

The command execution code should be null.

1. Create a configuration file named /etc/systemd/system/ydbd-testdb.service with the following content:

[Unit]
Description=YDB testdb dynamic node
After=network-online.target rc-local.service
Wants=network-online.target
StartLimitInterval=10
StartLimitBurst=15

[Service]
Restart=always
RestartSec=1
User=ydb
PermissionsStartOnly=true
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=ydbd
SyslogFacility=daemon
SyslogLevel=err
Environment=LD_LIBRARY_PATH=/opt/ydb/lib
ExecStart=/opt/ydb/bin/ydbd server --grpc-port 2136 --ic-port 19002 --mon-port 8766 --yaml-config  /opt/ydb/cfg/config.yaml --tenant /Root/testdb --node-broker <node1.ydb.tech>:2135 --node-broker <node2.ydb.tech>:2135 --node-broker <node3.ydb.tech>:2135
LimitNOFILE=65536
LimitCORE=0
LimitMEMLOCK=32212254720

[Install]
WantedBy=multi-user.target

Where <nodeN.ydb.tech> is the FQDN of the servers running the static nodes.

1. Start the YDB dynamic node for the /Root/testdb database:

sudo systemctl start ydbd-testdb

1. Run additional dynamic nodes on other servers to ensure database availability.