---
metadata:
  - name: generator
    content: Diplodoc Platform v5.52.0
alternate:
  - https://ydb.tech/docs/en/concepts/query_execution/federated_query/architecture.md?version=v25.4
  - https://ydb.tech/docs/ru/concepts/query_execution/federated_query/architecture.md?version=v25.4
  - href: en/concepts/query_execution/federated_query/architecture.md
    type: text/markdown
    title: Markdown version
  - href: ../../../llms.txt
    type: text/markdown
    title: llms.txt
sourcePath: en/core/concepts/query_execution/federated_query/architecture.md
---
> **Documentation Index:** Fetch the complete configuration index at https://ydb.tech/docs/en/llms.txt

# Federated Query Processing System Architecture

## External Data Sources and External Tables

A key element of the federated query processing system in YDB is the concept of an [external data source](https://ydb.tech/docs/en/concepts/datamodel/external_data_source.md?version=v25.4). Such sources can include relational DBMS, object storage, and other data storage systems. When processing a federated query, YDB streams data from external systems and allows performing the same range of operations on them as on local data.

To work with data located in external systems, YDB must have information about the internal structure of this data (e.g., the number, names, and types of columns in tables). Some sources provide such metadata along with the data itself, whereas for other unschematized sources, this metadata must be provided externally. This latter purpose is served by [external tables](https://ydb.tech/docs/en/concepts/datamodel/external_table.md?version=v25.4).

Once external data sources and (if necessary) external tables are registered in YDB, the client can proceed to describe federated queries.

## Connectors {#connectors}

While executing federated queries, YDB needs to access external data storage systems over the network, for which it uses their client libraries. Including such dependencies negatively affects the codebase size, compilation time, and binary file size of YDB, as well as the product's overall stability.

The list of supported data sources for federated queries is constantly expanding. The most popular sources, such as S3, are natively supported by YDB. However, not all users require support for all sources simultaneously. Support can be optionally enabled using _connectors_ - special microservices implementing a unified interface for accessing external data sources.

The functions of connectors include:

* Translating YQL queries into queries in the language specific to the external source (e.g., into another SQL dialect or HTTP API calls).
* Establishing network connections with data sources.
* Converting data retrieved from external sources into a columnar format in [Arrow IPC Stream](https://arrow.apache.org/docs/format/Columnar.html#serialization-and-interprocess-communication-ipc) format, supported by YDB.

![YDB Federated Query Architecture](_assets/architecture.png "YDB Federated Query Architecture" =640x)

Thus, connectors form an abstraction layer that hides the specifics of external data sources from YDB. The concise connector interface makes it easy to expand the list of supported sources with minimal changes to YDB's code.

Users can deploy [one of the ready-made connectors](https://ydb.tech/docs/en/devops/deployment-options/manual/federated-queries/connector-deployment.md?version=v25.4) or write their own implementation in any programming language according to the [gRPC specification](https://github.com/ydb-platform/ydb/tree/main/ydb/library/yql/providers/generic/connector/api/service/protos).

## List of Supported External Data Sources {#supported-datasources}

| Source | Support |
|--------|---------|
| [ClickHouse](https://clickhouse.com/) | Via connector [fq-connector-go](../../../devops/deployment-options/manual/federated-queries/connector-deployment.md#fq-connector-go) |
| [Greenplum](https://www.greenplum.org/) | Via connector [fq-connector-go](../../../devops/deployment-options/manual/federated-queries/connector-deployment.md#fq-connector-go) |
| [Microsoft SQL Server](https://learn.microsoft.com/en-us/sql/?view=sql-server-ver16) | Via connector [fq-connector-go](../../../devops/deployment-options/manual/federated-queries/connector-deployment.md#fq-connector-go) |
| [MySQL](https://www.mysql.com/) | Via connector [fq-connector-go](../../../devops/deployment-options/manual/federated-queries/connector-deployment.md#fq-connector-go) |
| [PostgreSQL](https://www.postgresql.org/) | Via connector [fq-connector-go](../../../devops/deployment-options/manual/federated-queries/connector-deployment.md#fq-connector-go) |
| [S3](https://aws.amazon.com/s3/) | Built into `ydbd` |
| [YDB](https://ydb.tech/) | Via connector [fq-connector-go](../../../devops/deployment-options/manual/federated-queries/connector-deployment.md#fq-connector-go) |