--- metadata: - name: generator content: Diplodoc Platform v5.59.0 alternate: - https://ydb.tech/docs/en/reference/ydb-cli/export-import/import-nfs.md - https://ydb.tech/docs/ru/reference/ydb-cli/export-import/import-nfs.md - href: https://ydb.tech/docs/en/reference/ydb-cli/export-import/import-nfs.md type: text/markdown title: Markdown version - href: https://ydb.tech/docs/en/llms.txt rel: describedby sourcePath: en/core/reference/ydb-cli/export-import/import-nfs.md --- > **Documentation Index:** Fetch the complete configuration index at https://ydb.tech/docs/en/llms.txt # Import from NFS The `import nfs` command starts a server-side import process from the network file system ( [Network File System](https://en.wikipedia.org/wiki/Network_File_System), NFS) of the YDB cluster hosts, importing data and schema object information in the format described in the [File structure](https://ydb.tech/docs/en/reference/ydb-cli/export-import/file-structure.md) article: ```bash ydb [connection options] import nfs [options] ``` where [connection options] are [database connection options](https://ydb.tech/docs/en/reference/ydb-cli/connect.md#command-line-pars) Unlike the [`tools restore` command](https://ydb.tech/docs/en/reference/ydb-cli/export-import/tools-restore.md), the `import nfs` command always creates objects entirely, so for it to succeed, none of the imported objects (neither directories nor tables) must exist. If you need to load additional data into existing tables, use the [`tools restore` command](https://ydb.tech/docs/en/reference/ydb-cli/export-import/tools-restore.md) directly on the mounted NFS directory. ## Command line parameters {#pars} `[options]` — command parameters: ### NFS parameters {#nfs-params} The import from NFS command requires specifying a mounted directory (or subdirectory) common to all objects involved in the import. Since the import is performed asynchronously on all YDB hosts, the specified directory must be present on each YDB host and mounted in NFS. `--fs-path PATH`: path to the mounted directory (or subdirectory). ### Imported database schema objects {#objects} `--destination-path PATH`: Target directory for imported objects; the default value is the database root. `--include PATH`: Data schema objects to include in the import. Directories are traversed recursively. To include multiple objects, the parameter can be specified multiple times. If not specified, all exported objects are imported. `--exclude STRING`: Pattern ( [PCRE](https://www.pcre.org/original/doc/html/pcrepattern.html)) to exclude paths from the import. This parameter can be specified multiple times for different patterns. {% cut "Alternative method" %} For backward compatibility, an alternative way to specify the list of objects is supported: `--item STRING`: Description of the object to load. The `--item` parameter can be specified multiple times if you need to load several objects. If the `--item` or `--include` parameters are not specified, all objects present in the specified dump will be loaded. `STRING` is specified in the `=,...` format with the following required properties: - `source`, `src`, or `s` — path in NFS (relative to `fs-path`) with the imported directory or table. - `destination`, `dst`, or `d` — the path in the database for placing the imported directory or table. The final path element must not exist. All directories on the path will be created if they do not exist. Some features may be unavailable when using alternative syntax (in particular, encrypted backups or listing of export objects). {% endcut %} ### Additional parameters {#aux} | Parameter | Description | | --- | --- | | `--description STRING` | Text description of the operation, saved in the operation history. | | `--retries NUM` | Number of retry attempts the server will make.
Default value: `10`. | | `--skip-checksum-validation` | Skip the validation stage of [checksums](./file-structure.md#checksums) of imported objects. | | `--encryption-key-file PATH` | Path to the file containing the encryption key (only for encrypted exports). This file is binary and must contain the exact number of bytes corresponding to the key length in the selected encryption algorithm (16 bytes for `AES-128-GCM`, 32 bytes for `AES-256-GCM` and `ChaCha20-Poly1305`). The key can also be passed via the `YDB_ENCRYPTION_KEY` environment variable, in hexadecimal string representation. | | `--format STRING` | Result output format.
Valid values:
| ## Running the import {#exec} ### Server import operation flow {#server-import-workflow} 1. A server asynchronous import operation is created. 2. The server reads object metadata from files (`scheme.pb`, `metadata.json`, etc.) from the storage. 3. For each object, a new table is created in the database. Target paths **must not exist** — import cannot overwrite existing tables. 4. Data is loaded from files in parallel on all cluster nodes. ### Launch result {#result} Upon successful execution, the `import nfs` command outputs summary information about the queued import from NFS operation, in the format specified by the `--format` option. The actual import is performed asynchronously by the server. The summary information includes the operation ID, which can later be used to check the status and perform actions on the operation: - In the `pretty` output mode (default), the operation ID is shown in the id field highlighted with pseudographics: ```text ┌───────────────────────────────────────────┬───────┬─────... | id | ready | stat... ├───────────────────────────────────────────┼───────┼─────... | ydb://import/8?id=281474976788395&kind=fs | true | SUCC... ├╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴╴┴╴╴╴╴╴╴╴┴╴╴╴╴╴... | Items: ... ``` - In the `proto-json-base64` output mode, the identifier is in the "id" attribute: ```json {"id":"ydb://import/8?id=281474976788395&kind=fs","ready":true, ... } ``` ### Import status {#status} Data loading is performed in the background. You can get information about the status and progress of the loading by calling the `operation get` command, whose parameter must be the **quoted** operation identifier, for example: ```bash ydb -p quickstart operation get "ydb://import/8?id=281474976788395&kind=fs" ``` The output format `operation get` is also set by the `--format` option. Although the operation ID is in URL format, it is not guaranteed to be preserved in the future. It should be interpreted only as a string. The completion of the upload is tracked by the change of the "progress" attribute: - In the `pretty` output mode (default), a successfully completed operation is reflected by the value "Done" in the `progress` field highlighted with pseudographics: ```text ┌───── ... ──┬───────┬─────────┬──────────┬─... | id | ready | status | progress | ... ├──────... ──┼───────┼─────────┼──────────┼─... | ydb://... | true | SUCCESS | Done | ... ├╴╴╴╴╴ ... ╴╴┴╴╴╴╴╴╴╴┴╴╴╴╴╴╴╴╴╴┴╴╴╴╴╴╴╴╴╴╴┴╴... ... ``` - In the `proto-json-base64` output mode, a completed operation is reflected by the value `PROGRESS_DONE` of the `progress` attribute: ```json {"id":"ydb://...", ...,"progress":"PROGRESS_DONE",... } ``` ### Completing an import operation {#forget} After completing the import, use the `operation forget` command to remove the import from the list of operations: ```bash ydb -p quickstart operation forget "ydb://import/8?id=281474976788395&kind=fs" ``` ### List of import operations {#list} To get a list of import operations, use the `operation list import/nfs` command: ```bash ydb -p quickstart operation list import/nfs ``` The output format `operation list` is also set by the `--format` option. ## Examples {#examples} {% note info %} The examples use the `quickstart` profile. To learn more, see [Creating a profile to connect to a test database](https://ydb.tech/docs/en/reference/ydb-cli/profile/create.md#quickstart). {% endnote %} ### Importing to the database root {#example-full-db} Importing the contents of the `/mnt/nfs/backups/export1` directory on the file system into the database root: ```bash ydb -p quickstart import nfs \ --fs-path /mnt/nfs/backups/export1 ``` ### Importing multiple directories {#example-specific-dirs} Importing objects from the `dir1` and `dir2` directories of the export located in `/mnt/nfs/backups/export1` on the file system into the database directories with the same names: ```bash ydb -p quickstart import nfs \ --fs-path /mnt/nfs/backups/export1 \ --include dir1 --include dir2 ``` ### Importing an encrypted export {#example-encryption} Importing a single table that was exported along the `dir/my_table` path into the `dir1/dir/my_table` path from an encrypted export located in `/mnt/nfs/backups/export1` on the file system, using a secret key from the `~/my_secret_key` file. ```bash ydb -p quickstart import nfs \ --fs-path /mnt/nfs/backups/export1 --destination-path dir1 \ --include dir/my_table \ --encryption-key-file ~/my_secret_key ``` ### Getting operation IDs {#example-list-oneline} To get a list of import operation IDs in a format convenient for processing in bash scripts, you can use the [jq](https://stedolan.github.io/jq/download/) utility: ```bash ydb -p quickstart operation list import/nfs --format proto-json-base64 | jq -r ".operations[].id" ``` You will get output where each new line contains an operation ID, for example: ```text ydb://import/8?id=281474976789577&kind=fs ydb://import/8?id=281474976789526&kind=fs ydb://import/8?id=281474976788779&kind=fs ``` Using these IDs, you can, for example, run a loop to terminate all current operations: ```bash ydb -p quickstart operation list import/nfs --format proto-json-base64 | jq -r ".operations[].id" | while read line; do ydb -p quickstart operation forget $line;done ```