Project: YugabyteDB (App Definition and Development / Database)
File: migrate-steps.md

title: Steps to perform offline migration of your database using YugabyteDB Voyager headerTitle: Offline migration linkTitle: Offline migration headcontent: Steps for an offline migration using YugabyteDB Voyager description: Run the steps to ensure a successful offline migration using YugabyteDB Voyager. aliases: /preview/yugabyte-voyager/migrate-steps/ menu: preview_yugabyte-voyager: identifier: migrate-offline parent: migration-types weight: 102 type: docs The following page describes the steps to perform and verify a successful offline migration to YugabyteDB. | Phase | Step | Description | |:- |: | :| | PREPARE | | yb-voyager supports RHEL, CentOS, Ubuntu, and macOS, as well as airgapped and Docker-based installations. | | | | Create a new database user with READ access to all the resources to be migrated. | | | | Deploy a YugabyteDB database and create a user with superuser privileges. | | EXPORT | | Convert the database schema to PostgreSQL format using the `yb-voyager export schema` command. | | | | Generate a Schema&nbsp;Analysis&nbsp;Report using the `yb-voyager analyze-schema` command. The report suggests changes to the PostgreSQL schema to make it appropriate for YugabyteDB. | | | | Using the report recommendations, manually change the exported schema. | | | | Dump the source database to the target machine (where yb-voyager is installed), using the `yb-voyager export data` command. | | IMPORT | | Import the modified schema to the target YugabyteDB database using the `yb-voyager import schema` command. | | | | Import the data to the target YugabyteDB database using the `yb-voyager import data` command. | | | | Import indexes and triggers to the target YugabyteDB database using the `yb-voyager import schema` command with an additional `--post-snapshot-import` flag. | | | | Check if the offline migration is successful. | | END | | Clean up the migration information stored in the export directory and databases (source and target). | Before proceeding with migration, ensure that you have completed the following steps: . Review the . Review strategies. . . Create a new database user, and assign the necessary user permissions. <ul class="nav nav-tabs nav-tabs-yb"> <li > <a href="#postgresql" class="nav-link active" id="postgresql-tab" data-toggle="tab" role="tab" aria-controls="postgresql" aria-selected="true"> <i class="icon-postgres" aria-hidden="true"></i> PostgreSQL </a> </li> <li> <a href="#mysql" class="nav-link" id="mysql-tab" data-toggle="tab" role="tab" aria-controls="mysql" aria-selected="true"> <i class="icon-mysql" aria-hidden="true"></i> MySQL </a> </li> <li> <a href="#oracle" class="nav-link" id="oracle-tab" data-toggle="tab" role="tab" aria-controls="oracle" aria-selected="true"> <i class="icon-oracle" aria-hidden="true"></i> Oracle </a> </li> </ul> <div class="tab-content"> <div id="postgresql" class="tab-pane fade show active" role="tabpanel" aria-labelledby="postgresql-tab"> {{% includeMarkdown "./postgresql.md" %}} </div> <div id="mysql" class="tab-pane fade" role="tabpanel" aria-labelledby="mysql-tab"> {{% includeMarkdown "./mysql.md" %}} </div> <div id="oracle" class="tab-pane fade" role="tabpanel" aria-labelledby="oracle-tab"> {{% includeMarkdown "./oracle.md" %}} </div> </div> Prepare your target YugabyteDB database cluster by creating a database, and a user for your cluster. Create the target YugabyteDB database in your YugabyteDB cluster. The database name can be the same or different from the source database name. If the target YugabyteDB database name is not provided, yb-voyager assumes the target YugabyteDB database name to be `yugabyte`. If you do not want to import to the default `yugabyte` database, specify the name of the target YugabyteDB database using the `--target-db-name` argument of the `yb-voyager import`

```sql CREATE DATABASE targetdbname; ``` Create a user with role. For a local YugabyteDB cluster or YugabyteDB Anywhere, create a user and role with the superuser privileges using the following command: ```sql CREATE USER ybvoyager SUPERUSER PASSWORD 'password'; ``` For YugabyteDB Managed, create a user with role using the following command: ```sql CREATE USER ybvoyager PASSWORD 'password'; GRANT yb_superuser TO ybvoyager; ``` If you want yb-voyager to connect to the target YugabyteDB database over SSL, refer to . {{< warning title="Deleting the ybvoyager user" >}} After migration, all the migrated objects (tables, views, and so on) are owned by the `ybvoyager` user. You should transfer the ownership of the objects to some other user (for example, `yugabyte`) and then delete the `ybvoyager` user. Example steps to delete the user are: ```sql REASSIGN OWNED BY ybvoyager TO yugabyte; DROP OWNED BY ybvoyager; DROP USER ybvoyager; ``` {{< /warning >}} yb-voyager keeps all of its migration state, including exported schema and data, in a local directory called the export directory. Before starting migration, you should create the export directory on a file system that has enough space to keep the entire source database. Next, you should provide the path of the export directory as a mandatory argument (`--export-dir`) to each invocation of the yb-voyager command in an environment variable. ```sh mkdir $HOME/export-dir export EXPORT_DIR=$HOME/export-dir ``` The export directory has the following sub-directories and files: `reports` directory contains the generated Schema Analysis Report. `schema` directory contains the source database schema translated to PostgreSQL. The schema is partitioned into smaller files by the schema object type such as tables, views, and so on. `data` directory contains CSV (Comma Separated Values) files that are passed to the COPY command on the target YugabyteDB database. `metainfo` and `temp` directories are used by yb-voyager for internal bookkeeping. `logs` directory contains the log files for each command. Proceed with schema and data migration using the following steps: To begin, export the schema from the source database. Once exported, analyze the schema and apply any necessary manual changes. The `yb-voyager export schema` command extracts the schema from the source database, converts it into PostgreSQL format (if the source database is Oracle or MySQL), and dumps the SQL DDL files in the `EXPORT_DIR/schema/*` directories. {{< note title="Renaming index names for MySQL" >}} YugabyteDB Voyager renames the indexes for MySQL migrations while exporting the schema. MySQL supports two or more indexes to have the same name in the same database, provided they are for different tables. Similarly to PostgreSQL, YugabyteDB does not support duplicate index names in the same schema. To avoid index name conflicts during export schema, yb-voyager prefixes each index name with the associated table name. {{< /note >}} {{< note title="Usage for sourcedbschema" >}} The `sourcedbschema` argument specifies the schema of the source database. For MySQL, currently the `source-db-schema` argument is not applicable. For PostgreSQL, `source-db-schema` can take one or more schema names separated by comma. For Oracle, `source-db-schema` can take only one schema name and you can migrate only one schema at a

{{< /note >}} An example invocation of the command with required arguments is as follows: ```sh yb-voyager export schema --export-dir <EXPORT_DIR> \ --source-db-type <SOURCEDBTYPE> \ --source-db-host <SOURCEDBHOST> \ --source-db-user <SOURCEDBUSER> \ --source-db-password <SOURCEDBPASSWORD> \ # Enclose the password in single quotes if it contains special characters. --source-db-name <SOURCEDBNAME> \ --source-db-schema <SOURCEDBSCHEMA> # Not applicable for MySQL ``` Refer to for details about the arguments. The schema exported in the previous step may not yet be suitable for importing into YugabyteDB. Even though YugabyteDB is PostgreSQL compatible, given its distributed nature, you may need to make minor manual changes to the schema. The `yb-voyager analyze-schema` command analyses the PostgreSQL schema dumped in the step, and prepares a report that lists the DDL statements which need manual changes. An example invocation of the command An example invocation of the command with required arguments is as follows: ```sh yb-voyager analyze-schema --export-dir <EXPORT_DIR> --output-format <FORMAT> ``` The above command generates a report file under the `EXPORT_DIR/reports/` directory. Refer to for details about the arguments. Fix all the issues listed in the generated schema analysis report by manually editing the SQL DDL files from the `EXPORT_DIR/schema/*`. After making the manual changes, re-run the `yb-voyager analyze-schema` command. This generates a fresh report using your changes. Repeat these steps until the generated report contains no issues. To learn more about modelling strategies using YugabyteDB, refer to . {{< note title="Manual schema changes" >}} Include the primary key definition in the `CREATE TABLE` statement. Primary Key cannot be added to a partitioned table using the `ALTER TABLE` statement. {{< /note >}} Refer to the for a detailed list of limitations and suggested workarounds associated with the source databases when migrating to YugabyteDB Voyager. Dump the source data into the `EXPORT_DIR/data` directory using the `yb-voyager export data` command as follows: ```sh yb-voyager export data --export-dir <EXPORT_DIR> \ --source-db-type <SOURCEDBTYPE> \ --source-db-host <SOURCEDBHOST> \ --source-db-user <SOURCEDBUSER> \ --source-db-password <SOURCEDBPASSWORD> \ # Enclose the password in single quotes if it contains special characters. --source-db-name <SOURCEDBNAME> \ --source-db-schema <SOURCEDBSCHEMA> # Not applicable for MySQL ``` Note that the `source-db-schema` argument is required for PostgreSQL and Oracle, and is not applicable for MySQL. Refer to for details about the arguments. The options passed to the command are similar to the command. To export only a subset of the tables, pass a comma-separated list of table names in the `--table-list` argument. {{< note title="Sequence migration considerations" >}} Sequence migration consists of two steps: sequence creation and setting resume value (resume value refers to the `NEXTVAL` of a sequence on a source database). A sequence object is generated during export schema and the resume values for sequences are generated during export data. These resume values are then set on the target YugabyteDB database just after the data is imported for all tables. Note that there are some special cases involving sequences such as the following: In MySQL, auto-increment column is migrated to YugbayteDB as a normal column with a sequence attached to

For PostgreSQL, `SERIAL` datatype and `GENERATED AS IDENTITY` columns use sequence object internally, so resume values for them are also generated during data export. {{< /note >}} Run the `yb-voyager export data status --export-dir <EXPORT_DIR>` command to get an overall progress of the export data operation. Refer to for details about the arguments. For MySQL and Oracle, you can optionally speed up data export by setting the environment variable `BETAFASTDATA_EXPORT=1` when you run `export data` using yb-voyager. Consider the following caveats before using the feature: You need to perform additional steps when you . Some data types are unsupported. For a detailed list, refer to . `--parallel-jobs` argument (specifies the number of tables to be exported in parallel from the source database at a time) will have no effect. In MySQL RDS, writes are not allowed during the data export process. For Oracle where sequences are not attached to a column, resume value generation is unsupported. Import the schema using the `yb-voyager import schema` command. {{< note title="Usage for targetdbschema" >}} The `targetdbschema` argument specifies the schema of the target YugabyteDB database and is applicable only for MySQL and Oracle. `yb-voyager` imports the source database into the `public` schema of the target YugabyteDB database. By specifying `--target-db-schema` argument during import, you can instruct `yb-voyager` to create a non-public schema and use it for the schema/data import. {{< /note >}} An example invocation of the command with required arguments is as follows: ```sh yb-voyager import schema --export-dir <EXPORT_DIR> \ --target-db-host <TARGETDBHOST> \ --target-db-user <TARGETDBUSER> \ --target-db-password <TARGETDBPASSWORD> \ # Enclose the password in single quotes if it contains special characters. --target-db-name <TARGETDBNAME> \ --target-db-schema <TARGETDBSCHEMA> # MySQL and Oracle only ``` Refer to for details about the arguments. yb-voyager applies the DDL SQL files located in the `$EXPORT_DIR/schema` directory to the target YugabyteDB database. If yb-voyager terminates before it imports the entire schema, you can rerun it by adding the `--ignore-exist` option. {{< note title="Importing indexes and triggers" >}} Because the presence of indexes and triggers can slow down the rate at which data is imported, by default `import schema` does not import indexes and triggers (with the exception of UNIQUE indexes, to avoid any issues during import of schema because of foreign key dependencies on the index). You should complete the data import without creating indexes and triggers. After data import is complete, create indexes and triggers using the `import schema` command with an additional `--post-snapshot-import` flag. {{< /note >}} After you have successfully exported the source data and imported the schema in the target YugabyteDB database, you can import the data using the `yb-voyager import data` command with required arguments as follows: ```sh yb-voyager import data --export-dir <EXPORT_DIR> \ --target-db-host <TARGETDBHOST> \ --target-db-user <TARGETDBUSER> \ --target-db-password <TARGETDBPASSWORD> \ # Enclose the password in single quotes if it contains special characters. --target-db-name <TARGETDBNAME> \ --target-db-schema <TARGETDBSCHEMA> \ # MySQL and Oracle only. --parallel-jobs <NUMBEROFJOBS> ``` By default, yb-voyager creates C/2 connections where C is the total number of cores in the cluster. You can change the default number of connections using the `--parallel-jobs` argument. If yb-voyager fails to determine the number of cores in the cluster, it defaults to 2 connections per

Refer to for details about the arguments. yb-voyager splits the data dump files (from the `$EXPORTDIR/data` directory) into smaller batches. yb-voyager concurrently ingests the batches such that all nodes of the target YugabyteDB database cluster are used. This phase is designed to be restartable_ if yb-voyager terminates while the data import is in progress. After restarting, the data import resumes from its current state. {{< tip title="Importing large datasets" >}} When importing a very large database, run the import data command in a `screen` session, so that the import is not terminated when the terminal session stops. If the `yb-voyager import data` command terminates before completing the data ingestion, you can re-run it with the same arguments and the command will resume the data import operation. {{< /tip >}} Run the `yb-voyager import data status --export-dir <EXPORT_DIR>` command to get an overall progress of the data import operation. Refer to for details about the arguments. Import indexes and triggers using the `import schema` command with an additional `--post-snapshot-import` flag as follows: ```sh yb-voyager import schema --export-dir <EXPORT_DIR> \ --target-db-host <TARGETDBHOST> \ --target-db-user <TARGETDBUSER> \ --target-db-password <TARGETDBPASSWORD> \ # Enclose the password in single quotes if it contains special characters. --target-db-name <TARGETDBNAME> \ --target-db-schema <TARGETDBSCHEMA> \ # MySQL and Oracle only --post-snapshot-import true ``` Refer to for details about the arguments. After the schema and data import is complete, manually run validation queries on both the source and target YugabyteDB database to ensure that the data is correctly migrated. For example, you can validate the databases by running queries to check the row count of each table. {{< warning title = "Caveat associated with rows reported by import data status" >}} Suppose you have a scenario where, or command fails. To resolve this issue, you delete some of the rows from the split files. After retrying, the import data command completes successfully. In this scenario, command reports incorrect imported row count; because it doesn't take into account the deleted rows. For more details, refer to the GitHub issue . {{< /warning >}} To complete the migration, you need to clean up the export directory (export-dir) and Voyager state (Voyager-related metadata) stored in the target YugabyteDB database. Run the `yb-voyager end migration` command to perform the clean up, and to back up the schema, data, migration reports, and log files by providing the backup related flags (mandatory) as follows: ```sh yb-voyager end migration --export-dir <EXPORT_DIR> \ --backup-log-files <true, false, yes, no, 1, 0> \ --backup-data-files <true, false, yes, no, 1, 0> \ --backup-schema-files <true, false, yes, no, 1, 0> \ --save-migration-reports <true, false, yes, no, 1, 0> \ --backup-dir <BACKUP_DIR> ``` After you run end migration, you will not be able to continue further. If you want to back up the schema, data, log files, and the migration reports (`analyze-schema` report, `export data status` output, or `import data status` output) for future reference, use the `--backup-dir` argument, and provide the path of the directory where you want to save the backup content (based on what you choose to back up). Refer to for more details on the arguments.