Vector load

Allows you to test YDB vector search performance and recall using exact and approximate search. Supports both global and filtered vector indexes.

The workload supports importing vectors from a real dataset (e.g., Wikipedia embeddings) or generating synthetic random vectors. After loading data, you can build a vector index, run search queries to measure recall and performance, and clean up the workload tables.

Command structure

ydb [global options...] workload vector [options...] <subcommand>

Subcommands:

vector                YDB vector workload.
├─ init                 Create and initialize tables for the workload
├─ import               Fill workload tables with data
│   ├─ files              Import vectors from files
│   └─ generator          Generate random vectors
├─ build-index          Create and initialize a vector index for the workload
├─ drop-index           Drop the vector index created for the workload
├─ run                  Run YDB vector workload
│   ├─ select            Search vectors and measure performance/recall
│   └─ upsert            Insert or update vector rows in the table
└─ clean                Drop tables created for load testing

Initializing the workload

Create the table for the workload:

ydb workload vector init

Available options

The created table has the following schema:

• id Uint64 NOT NULL — primary key;
• embedding String — serialized embedding vector;
• prefix Uint64 NOT NULL (only when --prefixed is specified) — filter column for prefixed indexes.

Loading data

After initialization, load data into the table. There are two subcommands: files to import vectors from an existing dataset and generator to generate synthetic random vectors.

After import is complete, a vector index named index is automatically built on the embedding column (unless --index-type None is specified).

Importing from files

Import vectors from files (CSV, TSV, or Parquet, optionally gzip-compressed). The dataset must contain id and embedding columns. Any additional columns are ignored.

For CSV/TSV files, embeddings must be encoded as a list of floats, e.g., "[ 1.0, 2.0, 3.0 ]". For Parquet files, embeddings can be a list of float32 values or already serialized YDB binary embeddings.

Example:

ydb workload vector import files

Available options

Common parameters of the import command

Generating synthetic data

Generate random vectors and load them into the table. Vector components are sampled from a uniform distribution and serialized into YDB binary embedding format.

ydb workload vector import generator

Available options

Common parameters of the import command

Building a vector index

If the table was loaded with --index-type None, or if you want to build an additional index with different parameters, you can build a vector index on an existing table using the build-index command.

ydb workload vector build-index --distance cosine

Available options

Dropping a vector index

Drop a previously built vector index.

ydb workload vector drop-index

Available options

Running the workload

Run load testing using one of two modes: select (vector search queries) or upsert (inserting new vector rows).

Search workload

Executes vector search queries against the indexed table. Optionally measures recall by comparing approximate search results with exact (full-scan) results, then runs a performance benchmark.

ydb workload vector run select --recall

Available options

Common parameters for all load types

Upsert workload

Continuously inserts new vector rows into the table, generating random embeddings on the fly.

ydb workload vector run upsert

Available options

Common parameters for all load types

Cleaning up

Drop tables created for load testing:

ydb workload vector clean

Test algorithm

The run select mode performs the following stages:

1. A test set is generated:
   • If a table with a test set is specified via --query-table, the first --targets rows are selected from it in primary key order.
   • Otherwise, --targets random rows are selected from the main table.
2. Recall measurement is performed (if --recall is specified):
   • Two queries are executed for each item in the test set.
   • The first query performs an exact vector search (full scan) based on vector distance, forming the R_exact result set.
   • The second query performs an approximate vector search using the vector index, forming the R_approx result set.
   • If a filtered vector index is selected, both queries scan only the entries with matching filter column values.
   • Recall of the approximate search is calculated using the formula $\frac{|R_{approx} \bigcap R_{exact}|}{|R_{exact}|}$ (where |A| is the number of elements in the set A and A ∩ B is the intersection of sets A and B).
3. Performance measurement is performed:
   • Indexed search queries are run for a specified duration and with a specified number of parallel threads.
   • Each query is executed for a random entry from the test set.
   • The average number of requests per second (RPS) and response time percentiles are calculated.

Usage examples

Example with generated data

1. Initialize the workload table:

   ydb workload vector init
   

2. Generate and load synthetic vectors:

   ydb workload vector import generator --rows 100000 --distance cosine
   

3. Run the search workload:

   ydb workload vector run select --recall
   

   Output example:

   Recall: 0.8950
   Window      Txs Txs/Sec Retries Errors  p50(ms) p95(ms) p99(ms) pMax(ms)
   1            100 100     0       0       5       8       12      15
   2             98 98      0       0       5       9       13      16
   ...
   
   Total       Txs Txs/Sec Retries Errors  p50(ms) p95(ms) p99(ms) pMax(ms)
   10          980 98.0    0       0       5       9       14      18
   

   Column descriptions:

   • Window — Sequential number of the time window (e.g., each second or fixed interval).
   • Txs — Number of transactions successfully completed in this window (or total across all windows in the bottom section).
   • Txs/Sec — Transaction rate per second for the given window (or average for total).
   • Retries — Number of automatic retries performed due to temporary errors (e.g., conflicts or throttling).
   • Errors — Number of unrecoverable errors encountered.
   • p50(ms) — Median (50th percentile) transaction latency in milliseconds.
   • p95(ms) — 95th percentile latency (milliseconds).
   • p99(ms) — 99th percentile latency (milliseconds).
   • pMax(ms) — Maximum observed latency within the window (or overall maximum for total).

4. Run the upsert workload:

   ydb workload vector run upsert
   

5. Clean up:

   ydb workload vector clean
   

Example with an external dataset

1. Prepare tables and load data. See the recipe Vector index with external dataset loading.

   Examples of creating vector indexes are available on the Vector Indexes documentation page.

2. Run the search workload:

   ydb -e grpc://hostname:2135 -d /Root/testdb workload vector run select \
       --table wikipedia --index idx_vector \
       --query-table wikipedia_sample --recall
   

   Output example:

   Recall: 0.8950
   Window      Txs Txs/Sec Retries Errors  p50(ms) p95(ms) p99(ms) pMax(ms)
   1            100 100     0       0       5       8       12      15
   2             98 98      0       0       5       9       13      16
   ...
   
   Total       Txs Txs/Sec Retries Errors  p50(ms) p95(ms) p99(ms) pMax(ms)
   10          980 98.0    0       0       5       9       14      18
   

Remarks

• The search workload generates SQL queries that select nearest rows by vector distance from table --table using index --index.
• You don't need to specify column names (embedding, filter, primary key columns) or the distance function — they are automatically extracted from the index definition.
• Random selection of test vectors from --table works only if its primary key is numeric.
• If --query-table is specified, it must have the same vector and filter column names as --table.
• Recall measurement (--recall) is performed as a separate stage before the main performance test and shows the average overlap ratio (from 0 to 1) of indexed search results with full-scan search results across all selected test vectors.

Preparing a test sample from a large table

In this case, you can prepare a test set by selecting random rows from the main table manually. Example query to create the table for test samples:

CREATE TABLE vector_index_sample (
    id Uint64 NOT NULL,
    prefix Uint64 NOT NULL,
    embedding String NOT NULL,
    PRIMARY KEY (id)
);

A query to fill it with approximately 1000 rows from a large table:

INSERT INTO vector_index_sample
SELECT id, prefix, embedding FROM large_table
WHERE RandomNumber(id) < 0xFFFFFFFFFFFFFFFF / <number_of_rows_in_table> * 1000;