Cluster Expansion

You can expand a YDB cluster by adding new nodes to the cluster configuration. Below are the necessary actions for expanding a YDB cluster installed manually on virtual machines or physical servers. Cluster expansion in a Kubernetes environment is performed by adjusting the YDB controller settings for Kubernetes.

Expanding a YDB cluster does not require suspending user access to databases. When expanding the cluster, its components are restarted to apply configuration changes, which in turn may lead to the need to retry transactions running on the cluster. Transaction retries are performed automatically by applications using YDB SDK capabilities for error control and operation retry.

Preparing New Servers

When placing new static or dynamic cluster nodes on new servers that were not previously part of the expanded YDB cluster, each new server must install YDB software according to the procedures described in the cluster deployment instructions. In particular, it is necessary to:

create an account and group in the operating system for the YDB service
install YDB software
prepare and place the corresponding TLS key and certificate on the server
copy the current YDB cluster configuration file to the server

TLS certificates used on new servers must comply with field filling requirements, and be signed by a trusted certificate authority used on existing servers of the expanded YDB cluster.

Adding Dynamic Nodes

Adding dynamic nodes allows increasing available computing resources (processor cores and RAM) for executing user queries by the YDB cluster.

To add a dynamic node to the cluster, it is sufficient to start the process serving this node, passing it the path to the cluster configuration folder, the name of the served database, and addresses of any three static cluster nodes in the command line parameters, as shown in the cluster deployment instructions.

After successfully adding a dynamic node to the cluster, information about it will be available on the cluster monitoring page in the embedded UI.

To remove a dynamic node from the cluster, it is sufficient to stop the dynamic node process.

Adding Static Nodes

Adding static nodes allows increasing throughput when performing input-output operations and increasing available capacity for data storage in the YDB cluster.

To add static nodes to the cluster, you need to perform the following sequence of actions:

Format disks that will be used for storing YDB data using the procedure described for the cluster deployment stage.
Obtain an authentication token for executing administrative commands using YDB CLI, for example:
```
ydb -e grpcs://<node1.ydb.tech>:2135 -d /Root --ca-file ca.crt \
    --user root auth get-token --force > token-file
```
The example command above uses the following parameters:
- node1.ydb.tech — FQDN of any server hosting static cluster nodes
- 2135 — grpcs service port number for static nodes
- ca.crt — certificate authority certificate file name
- root — username with administrative rights
- token-file — name of the file where the authentication token is saved for subsequent use
When executing the above command, YDB CLI will request a password for authenticating the specified user.

Get the current cluster configuration by executing the following command on any cluster node:

ydb --token-file token-file --ca-file ca.crt -e grpcs://<node1.ydb.tech>:2135 \
    admin cluster config fetch > config.yaml

Correct the cluster configuration file, including in the configuration a description of the added nodes (in the hosts section) and disks used on them (in the host_configs section).
Allow the YDB cluster to use disks on new static nodes for data storage by executing the following command on any cluster node:
```
export LD_LIBRARY_PATH=/opt/ydb/lib
ydb --token-file ydbd-token-file --ca-file ca.crt -e grpcs://<node1.ydb.tech>:2135 \
    admin cluster config replace -f config.yaml
echo $?
```
The example command above uses the following parameters:
- ydbd-token-file — name of the previously obtained authentication token file
- 2135 — grpcs service port number for static nodes
- ca.crt — certificate authority certificate file name
If the above command returns a configuration version verification error, this means that the current config version is outdated, and you need to get a new one from the cluster by repeating step 3. Example configuration version verification error message:
```
ErrorDescription: "ConfigVersion mismatch ConfigVersionProvided# 0 ConfigVersionExpected# 1"
```
Create an empty directory /opt/ydb/cfg on each new machine for the cluster to work with configuration. If multiple nodes are started on one machine, use the same directory. By executing a special command on each new machine, initialize this directory using any static cluster node as the configuration source.
```
sudo mkdir -p /opt/ydb/cfg
sudo chown -R ydb:ydb /opt/ydb/cfg
ydb admin node config init --config-dir /opt/ydb/cfg --seed-node <node.ydb.tech:2135>
```
Start processes serving new static cluster nodes on the corresponding servers.
Ensure that new static nodes are displayed on the cluster monitoring page in the embedded UI.
Add additional storage groups to one or more databases by executing commands of the following type on any cluster node:
```
export LD_LIBRARY_PATH=/opt/ydb/lib
/opt/ydb/bin/ydbd -f ydbd-token-file --ca-file ca.crt -s grpcs://`hostname -f`:2135 \
    admin database /Root/testdb pools add ssd:1
echo $?
```
The example command above uses the following parameters:
- ydbd-token-file - name of the previously obtained authentication token file
- 2135 - grpcs service port number for static nodes
- ca.crt - certificate authority certificate file name
- /Root/testdb - full path to the database
- ssd:1 - storage pool name and number of allocated storage groups
Ensure that added storage groups are displayed on the cluster monitoring page in the embedded UI.

Removing static nodes from the YDB cluster is performed according to the documented decommissioning procedure.

In case of damage and inability to repair a server running a static cluster node, it is necessary to place the unavailable static node on a new server containing a similar or greater number and volume of disks.

Was the article helpful?

Configuration parameters

State Storage move