Import and export data using federated queries
Note
When importing/exporting data from/to S3 in Parquet format, you must consider the mapping of YQL and Apache Arrow types.
Data import
Federated queries allow you to import data from connected external sources into YDB database tables. To import data, you need to use a read query from an external data source or external table and write to a YDB table.
For columnar tables, massively parallel import from an external source is supported when using UPSERT and INSERT statements — multiple worker threads read data from the external source in parallel and write to the table. For row-based tables, this functionality is under development.
| Operation | Write to row-based tables | Write to columnar tables |
|---|---|---|
| UPSERT | single-threaded | parallel |
| REPLACE | single-threaded | parallel |
| INSERT | single-threaded | parallel |
Tip
The recommended import options using federated queries are the UPSERT and REPLACE operations — the data import scenario is significantly optimized for them.
Example of importing data from a PostgreSQL table into a YDB table:
UPSERT INTO target_table
SELECT * FROM postgresql_datasource.source_table
For more details on creating external data sources and external tables, as well as reading data from them, see the articles:
Experimental data sources
Data export
Currently, data export using federated queries is supported only for S3-compatible storage; for more details, see the article Exporting data to S3 object storage.