---
metadata:
  - name: generator
    content: Diplodoc Platform v5.52.0
alternate:
  - https://ydb.tech/docs/en/troubleshooting/performance/schemas/overloaded-shards.md?version=v26.1
  - https://ydb.tech/docs/ru/troubleshooting/performance/schemas/overloaded-shards.md?version=v26.1
  - href: en/troubleshooting/performance/schemas/overloaded-shards.md
    type: text/markdown
    title: Markdown version
  - href: ../../../llms.txt
    type: text/markdown
    title: llms.txt
sourcePath: en/core/troubleshooting/performance/schemas/overloaded-shards.md
---
> **Documentation Index:** Fetch the complete configuration index at https://ydb.tech/docs/en/llms.txt

# Overloaded shards

[Data shards](https://ydb.tech/docs/en/concepts/glossary.md?version=v26.1#data-shard) serving [row-oriented tables](https://ydb.tech/docs/en/concepts/datamodel/table.md?version=v26.1#row-oriented-tables) may become overloaded for the following reasons:

* A table is created without the [AUTO_PARTITIONING_BY_LOAD](https://ydb.tech/docs/en/concepts/datamodel/table.md?version=v26.1#AUTO_PARTITIONING_BY_LOAD) clause.

    In this case, YDB does not split overloaded shards.

    Data shards are single-threaded and process queries sequentially. Each data shard can accept up to 10,000 operations. Accepted queries wait for their turn to be executed. So the longer the queue, the higher the latency.

    If a data shard already has 10000 operations in its queue, new queries will return an "overloaded" error. Retry such queries using a randomized exponential back-off strategy. For more information, see [Overloaded errors](https://ydb.tech/docs/en/troubleshooting/performance/queries/overloaded-errors.md?version=v26.1).

* A table was created with the [AUTO_PARTITIONING_MAX_PARTITIONS_COUNT](https://ydb.tech/docs/en/concepts/datamodel/table.md?version=v26.1#AUTO_PARTITIONING_MAX_PARTITIONS_COUNT) setting and has already reached its partition limit.

* An inefficient [primary key](https://ydb.tech/docs/en/concepts/glossary.md?version=v26.1#primary-key) that causes an imbalance in the distribution of queries across shards. A typical example is ingestion with a monotonically increasing primary key, which may lead to the overloaded "last" partition. For example, this could occur with an autoincrementing primary key using the serial data type.

## Diagnostics

<!-- The include is added to allow partial overrides in overlays  -->
<!-- source: en/troubleshooting/performance/schemas/_includes/overloaded-shards-diagnostics.md -->
1. Use the Embedded UI or Grafana to see if the YDB nodes are overloaded:

    - In the **[DB overview](https://ydb.tech/docs/en/reference/observability/metrics/grafana-dashboards.md?version=v26.1#dboverview)** Grafana dashboard, analyze the **Overloaded shard count** chart.

        ![](_assets/overloaded-shards-dashboard.png)

        The chart indicates whether the YDB cluster has overloaded shards, but it does not specify which table's shards are overloaded.

        {% note tip %}

        Use Grafana to set up alert notifications when YDB data shards get overloaded.

        {% endnote %}


    - In the [Embedded UI](https://ydb.tech/docs/en/reference/embedded-ui/index.md?version=v26.1):

        1. Go to the **Databases** tab and click on the database.

        1. On the **Navigation** tab, ensure the required database is selected.

        1. Open the **Diagnostics** tab.

        1. Open the **Top shards** tab.

        1. In the **Immediate** and **Historical** tabs, sort the shards by the **CPUCores** column and analyze the information.

        ![](_assets/partitions-by-cpu.png)

        Additionally, the information about overloaded shards is provided as a system table. For more information, see [History of overloaded partitions](https://ydb.tech/docs/en/dev/system-views.md?version=v26.1#top-overload-partitions).

1. To pinpoint the schema issue, use the [Embedded UI](https://ydb.tech/docs/en/reference/embedded-ui/index.md?version=v26.1) or [YDB CLI](https://ydb.tech/docs/en/reference/ydb-cli/index.md?version=v26.1):

    - In the [Embedded UI](https://ydb.tech/docs/en/reference/embedded-ui/index.md?version=v26.1):

        1. On the **Databases** tab, click on the database.

        1. On the **Navigation** tab, select the required table.

        1. Open the **Diagnostics** tab.

        1. On the **Describe** tab, navigate to `root > PathDescription > Table > PartitionConfig > PartitioningPolicy`.

            ![Describe](_assets/describe.png)

        1. Analyze the **PartitioningPolicy** values:

            - `SizeToSplit`
            - `SplitByLoadSettings`
            - `MaxPartitionsCount`

            If the table does not have these options, see [Recommendations for table configuration](https://ydb.tech/docs/en/troubleshooting/performance/schemas/overloaded-shards.md?version=v26.1#table-config).

        {% note info %}

        You can also find this information on the **Diagnostics > Info** tab.

        {% endnote %}


    - In the [YDB CLI](https://ydb.tech/docs/en/reference/ydb-cli/index.md?version=v26.1):

        1. To retrieve information about the problematic table, run the following command:

            ```bash
            ydb scheme describe <table_name>
            ```

        2. In the command output, analyze the **Auto partitioning settings**:

            - `Partitioning by size`
            - `Partitioning by load`
            - `Max partitions count`

            If the table does not have these options, see [Recommendations for table configuration](https://ydb.tech/docs/en/troubleshooting/performance/schemas/overloaded-shards.md?version=v26.1#table-config).

1. Analyze whether primary key values increment monotonically:

    - Check the data type of the primary key column. `Serial` data types are used for autoincrementing values.

    - Check the application logic.

    - Calculate the difference between the minimum and maximum values of the primary key column. Then compare this value to the number of rows in a given table. If these values match, the primary key might be incrementing monotonically.

    If primary key values do increase monotonically, see [Recommendations for the imbalanced primary key](https://ydb.tech/docs/en/troubleshooting/performance/schemas/overloaded-shards.md?version=v26.1#pk-recommendations).
<!-- endsource: en/troubleshooting/performance/schemas/_includes/overloaded-shards-diagnostics.md -->

## Recommendations

### For table configuration {#table-config}

Consider the following solutions to address shard overload:

* If the problematic table is not partitioned by load, enable partitioning by load.

    {% note tip %}

    A table is not partitioned by load, if you see the `Partitioning by load: false` line on the **Diagnostics > Info** tab in the **Embedded UI** or the  `ydb scheme describe` command output.

    {% endnote %}

* If the table has reached the maximum number of partitions, increase the partition limit.

    {% note tip %}

    To determine the number of partitions in the table, see the `PartCount` value on the **Diagnostics > Info** tab in the **Embedded UI**.

    {% endnote %}


Both operations can be performed by executing an [`ALTER TABLE ... SET`](https://ydb.tech/docs/en/yql/reference/syntax/alter_table/set.md?version=v26.1) query.


### For the imbalanced primary key {#pk-recommendations}

Consider modifying the primary key to distribute the load evenly across table partitions. You cannot change the primary key of an existing table. To do that, you will have to create a new table with the modified primary key and then migrate the data to the new table.

{% note info %}

Also, consider changing your application logic for generating primary key values for new rows. For example, use hashes of values instead of values themselves.

{% endnote %}

## Example

For a practical demonstration of how to follow these instructions, see [Overloaded shard example](https://ydb.tech/docs/en/troubleshooting/examples/schemas/overloaded-shard-simple-case.md?version=v26.1).