Project: YugabyteDB (App Definition and Development / Database)
File: live-fall-back.md

title: Steps to perform live migration of your database with fall-back using YugabyteDB Voyager headerTitle: Live migration with fall-back linkTitle: Live migration with fall-back headcontent: Steps for a live migration with fall-back using YugabyteDB Voyager description: Steps to ensure a successful live migration with fall-back using YugabyteDB Voyager. menu: preview_yugabyte-voyager: identifier: live-fall-back parent: migration-types weight: 104 techPreview: /preview/releases/versioning/#feature-availability type: docs When migrating a database, it's prudent to have a backup strategy in case the new database doesn't work as expected. A fall-back approach involves streaming changes from the YugabyteDB (target) database back to the source database after the cutover operation, enabling you to cutover to the source database at any point. A fall-back approach allows you to test the system end-to-end. This workflow is especially important in heterogeneous migration scenarios, in which source and target databases are using different engines. {{< warning title="Important" >}} This workflow has the potential to alter your source database. Make sure you fully understand the implications of these changes before proceeding. {{< /warning >}} Before starting a live migration, you set up the and database. During migration, yb-voyager replicates the snapshot data along with new changes exported from the source database to the target YugabyteDB database, as shown in the following illustration: At , applications stop writing to the source database and start writing to the target YugabyteDB database. After the cutover process is complete, Voyager keeps the source database synchronized with changes from the target YugabyteDB database as shown in the following illustration: Finally, if you need to switch back to the source database (because the current YugabyteDB system is not working as expected), you can database. The following illustration describes the workflow for live migration using YB Voyager with the fall-back option. | Phase | Step | Description | | :- | : | :| | PREPARE | | yb-voyager supports RHEL, CentOS, Ubuntu, and macOS, as well as airgapped and Docker-based installations. | | | | Create a new database user with READ and WRITE (for fall-back) access to all the resources to be migrated. | | | | Deploy a YugabyteDB database and create a user with necessary privileges. | | SCHEMA | | Convert the database schema to PostgreSQL format using the `yb-voyager export schema` command. | | | | Generate a Schema&nbsp;Analysis&nbsp;Report using the `yb-voyager analyze-schema` command. The report suggests changes to the PostgreSQL schema to make it appropriate for YugabyteDB. | | | | Using the report recommendations, manually change the exported schema. | | | | Import the modified schema to the target YugabyteDB database using the `yb-voyager import schema` command. | | LIVE MIGRATION | Start | Start the phases: export data first, followed by import data to target and archive changes simultaneously. | | | | The export data command first exports a snapshot and then starts continuously capturing changes from the source.| | | | The import data command first imports the snapshot, and then continuously applies the exported change events on the target. | | | | After the snapshot import is complete, import indexes and triggers to the target YugabyteDB database using the `yb-voyager import schema` command with an additional `--post-snapshot-import` flag. | | | | Continuously archive migration changes to limit disk

| | CUTOVER TO TARGET | | Perform a cutover (stop streaming changes) when the migration process reaches a steady state where you can stop your applications from pointing to your source database, allow all the remaining changes to be applied on the target YugabyteDB database, and then restart your applications pointing to YugabyteDB. | | | | Monitor the wait status using the command. | | | | Check if the live migration is successful on both the source and the target databases. | | | | Run PL/SQL commands to ensure that triggers and foreign key checks are disabled so the data can be imported correctly from the target YugabyteDB database to the source database. | | (OPTIONAL) CUTOVER TO SOURCE | | Perform a cutover (stop streaming changes) when the migration process reaches a steady state where you can stop your applications from pointing to your target YugabyteDB database, allow all the remaining changes to be applied on the source database, and then restart your applications pointing to the source database. | | | | Monitor the wait status using the command. | | | | Run PL/SQL commands to re-enable the triggers and foreign keys on the source database. | | | | Check if the live migration is successful on both the source and the target databases. | | END | | Clean up the migration information stored in export directory and databases (source and target). | Before proceeding with migration, ensure that you have completed the following steps: . Review the . Review strategies. . . Create a new database user, and assign the necessary user permissions. <ul class="nav nav-tabs-alt nav-tabs-yb custom-tabs"> <li> <a href="#oracle" class="nav-link active" id="oracle-tab" data-toggle="tab" role="tab" aria-controls="oracle" aria-selected="true"> <i class="icon-oracle" aria-hidden="true"></i> Oracle </a> </li> <li > <a href="#pg" class="nav-link" id="pg-tab" data-toggle="tab" role="tab" aria-controls="pg" aria-selected="false"> <i class="icon-postgres" aria-hidden="true"></i> PostgreSQL </a> </li> </ul> <div class="tab-content"> <div id="oracle" class="tab-pane fade show active" role="tabpanel" aria-labelledby="oracle-tab"> {{< tabpane text=true >}} {{% tab header="Standalone Oracle Container Database" %}} Ensure that your database log_mode is `archivelog` as follows: ```sql SELECT LOG_MODE FROM V$DATABASE; ``` ```output LOG_MODE ARCHIVELOG ``` If log_mode is NOARCHIVELOG (that is, not enabled), run the following command: ```sql sqlplus /nolog SQL>alter system set dbrecoveryfiledestsize = 10G; SQL>alter system set dbrecoveryfiledest = '<oraclepath>/oradata/recovery_area' scope=spfile; SQL> connect / as sysdba SQL> Shutdown immediate SQL> Startup mount SQL> Alter database archivelog; SQL> Alter database open; ``` Create the tablespaces as follows: Connect to Pluggable database (PDB) as sysdba and run the following command: ```sql CREATE TABLESPACE logminertbs DATAFILE '/opt/oracle/oradata/ORCLCDB/ORCLPDB1/logminertbs.dbf' SIZE 25M REUSE AUTOEXTEND ON MAXSIZE UNLIMITED; ``` Connect to Container database (CDB) as sysdba and run the following command: ```sql CREATE TABLESPACE logminertbs DATAFILE '/opt/oracle/oradata/ORCLCDB/logminertbs.dbf' SIZE 25M REUSE AUTOEXTEND ON MAXSIZE UNLIMITED; ``` Run the following commands from CDB as sysdba: ```sql CREATE USER c##ybvoyager IDENTIFIED BY password DEFAULT TABLESPACE logminer_tbs QUOTA UNLIMITED ON logminer_tbs CONTAINER=ALL; GRANT CREATE SESSION TO c##ybvoyager CONTAINER=ALL; GRANT SET CONTAINER TO c##ybvoyager CONTAINER=ALL; GRANT SELECT ON V_$DATABASE to c##ybvoyager CONTAINER=ALL; GRANT FLASHBACK ANY TABLE TO c##ybvoyager CONTAINER=ALL; GRANT SELECT ANY TABLE TO c##ybvoyager CONTAINER=ALL; GRANT SELECTCATALOGROLE TO c##ybvoyager CONTAINER=ALL; GRANT EXECUTECATALOGROLE TO c##ybvoyager CONTAINER=ALL; GRANT SELECT ANY TRANSACTION TO c##ybvoyager CONTAINER=ALL; GRANT LOGMINING TO c##ybvoyager CONTAINER=ALL; GRANT CREATE TABLE TO c##ybvoyager CONTAINER=ALL; GRANT LOCK ANY TABLE TO c##ybvoyager CONTAINER=ALL; GRANT CREATE SEQUENCE TO c##ybvoyager CONTAINER=ALL; GRANT EXECUTE ON DBMS_LOGMNR TO c##ybvoyager CONTAINER=ALL; GRANT EXECUTE ON DBMSLOGMNRD TO c##ybvoyager CONTAINER=ALL; GRANT SELECT ON V_$LOG TO c##ybvoyager CONTAINER=ALL; GRANT SELECT ON V$LOGHISTORY TO c##ybvoyager CONTAINER=ALL; GRANT SELECT ON V$LOGMNRLOGS TO c##ybvoyager CONTAINER=ALL; GRANT SELECT ON V$LOGMNRCONTENTS TO c##ybvoyager CONTAINER=ALL; GRANT SELECT ON V$LOGMNRPARAMETERS TO c##ybvoyager CONTAINER=ALL; GRANT SELECT ON V_$LOGFILE TO c##ybvoyager CONTAINER=ALL; GRANT SELECT ON V$ARCHIVEDLOG TO c##ybvoyager CONTAINER=ALL; GRANT SELECT ON V$ARCHIVEDEST_STATUS TO c##ybvoyager CONTAINER=ALL; GRANT SELECT ON V_$TRANSACTION TO c##ybvoyager CONTAINER=ALL; GRANT SELECT ON V_$MYSTAT TO c##ybvoyager CONTAINER=ALL; GRANT SELECT ON V_$STATNAME

c##ybvoyager CONTAINER=ALL; ``` Enable supplemental logging in the database as follows: ```sql ALTER DATABASE ADD SUPPLEMENTAL LOG DATA; ALTER DATABASE ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS; ``` Create `ybvoyager_metadata` schema or user, and tables for voyager to use during migration as follows: ```sql CREATE USER ybvoyager_metadata IDENTIFIED BY "password"; GRANT CONNECT, RESOURCE TO ybvoyager_metadata; ALTER USER ybvoyager_metadata QUOTA UNLIMITED ON USERS; CREATE TABLE ybvoyagermetadata.ybvoyagerimportdataeventchannelsmetainfo ( migration_uuid VARCHAR2(36), channel_no INT, lastappliedvsn NUMBER(19), num_inserts NUMBER(19), num_updates NUMBER(19), num_deletes NUMBER(19), PRIMARY KEY (migrationuuid, channelno) ); CREATE TABLE ybvoyagermetadata.ybvoyagerimportedeventcountbytable ( migration_uuid VARCHAR2(36), table_name VARCHAR2(250), channel_no INT, total_events NUMBER(19), num_inserts NUMBER(19), num_updates NUMBER(19), num_deletes NUMBER(19), PRIMARY KEY (migrationuuid, tablename, channel_no) ); ``` Create a writer role for the source schema for Voyager to be able to write the changes from the target YugabyteDB database to the source database (in case of a fall-back): ```sql CREATE ROLE <SCHEMANAME>writer_role; BEGIN FOR R IN (SELECT owner, objectname FROM allobjects WHERE owner=UPPER('<SCHEMANAME>') and objecttype ='TABLE' MINUS SELECT owner, tablename from allnestedtables where owner = UPPER('<SCHEMANAME>')) LOOP EXECUTE IMMEDIATE 'GRANT SELECT, INSERT, UPDATE, DELETE, ALTER on '||R.owner||'."'||R.objectname||'" to <SCHEMANAME>writerrole'; END LOOP; END; / DECLARE v_sql VARCHAR2(4000); BEGIN FOR tablerec IN (SELECT tablename FROM alltables WHERE owner = 'YBVOYAGERMETADATA') LOOP vsql := 'GRANT ALL PRIVILEGES ON YBVOYAGERMETADATA.' || tablerec.tablename || ' TO <SCHEMANAME>writer_role'; EXECUTE IMMEDIATE v_sql; END LOOP; END; / GRANT CREATE ANY SEQUENCE, SELECT ANY SEQUENCE, ALTER ANY SEQUENCE TO <SCHEMANAME>writer_role; ``` Assign the writer role to the source database user as follows: ```sql GRANT <SCHEMANAME>writer_role TO c##ybvoyager; ``` {{% /tab %}} {{% tab header="RDS Oracle" %}} Ensure that your database log_mode is `archivelog` as follows: ```sql SELECT LOG_MODE FROM V$DATABASE; ``` ```output LOG_MODE ARCHIVELOG ``` If log_mode is NOARCHIVELOG (that is, not enabled), run the following command: ```sql exec rdsadmin.rdsadminutil.setconfiguration('archivelog retention hours',24); ``` Connect to your database as an admin user, and create the tablespaces as follows: ```sql CREATE TABLESPACE logminer_tbs DATAFILE SIZE 25M AUTOEXTEND ON MAXSIZE UNLIMITED; ``` Run the following commands connected to the admin or privileged user: ```sql CREATE USER ybvoyager IDENTIFIED BY password DEFAULT TABLESPACE logminer_tbs QUOTA UNLIMITED ON logminer_tbs; GRANT CREATE SESSION TO YBVOYAGER; begin rdsadmin.rdsadminutil.grantsys_object( pobjname => 'V_$DATABASE', p_grantee => 'YBVOYAGER', p_privilege => 'SELECT'); end; / GRANT FLASHBACK ANY TABLE TO YBVOYAGER; GRANT SELECT ANY TABLE TO YBVOYAGER; GRANT SELECTCATALOGROLE TO YBVOYAGER; GRANT EXECUTECATALOGROLE TO YBVOYAGER; GRANT SELECT ANY TRANSACTION TO YBVOYAGER; GRANT LOGMINING TO YBVOYAGER; GRANT CREATE TABLE TO YBVOYAGER; GRANT LOCK ANY TABLE TO YBVOYAGER; GRANT CREATE SEQUENCE TO YBVOYAGER; begin rdsadmin.rdsadminutil.grantsys_object( pobjname => 'DBMS_LOGMNR', p_grantee => 'YBVOYAGER', p_privilege => 'EXECUTE', pgrantoption => true); end; / begin rdsadmin.rdsadminutil.grantsys_object( pobjname => 'DBMSLOGMNRD', p_grantee => 'YBVOYAGER', p_privilege => 'EXECUTE', pgrantoption => true); end; / begin rdsadmin.rdsadminutil.grantsys_object( pobjname => 'V_$LOG', p_grantee => 'YBVOYAGER', p_privilege => 'SELECT'); end; / begin rdsadmin.rdsadminutil.grantsys_object( pobjname => 'V$LOGHISTORY', p_grantee => 'YBVOYAGER', p_privilege => 'SELECT'); end; / begin rdsadmin.rdsadminutil.grantsys_object( pobjname => 'V$LOGMNRLOGS', p_grantee => 'YBVOYAGER', p_privilege => 'SELECT'); end; / begin rdsadmin.rdsadminutil.grantsys_object( pobjname => 'V$LOGMNRCONTENTS', p_grantee => 'YBVOYAGER', p_privilege => 'SELECT'); end; / begin rdsadmin.rdsadminutil.grantsys_object( pobjname => 'V$LOGMNRPARAMETERS', p_grantee => 'YBVOYAGER', p_privilege => 'SELECT'); end; / begin rdsadmin.rdsadminutil.grantsys_object( pobjname => 'V_$LOGFILE', p_grantee => 'YBVOYAGER', p_privilege => 'SELECT'); end; / begin rdsadmin.rdsadminutil.grantsys_object( pobjname => 'V$ARCHIVEDLOG', p_grantee => 'YBVOYAGER', p_privilege => 'SELECT'); end; / begin rdsadmin.rdsadminutil.grantsys_object( pobjname => 'V$ARCHIVEDEST_STATUS', p_grantee => 'YBVOYAGER', p_privilege => 'SELECT'); end; / begin rdsadmin.rdsadminutil.grantsys_object( pobjname => 'V_$TRANSACTION', p_grantee => 'YBVOYAGER', p_privilege => 'SELECT'); end; / begin rdsadmin.rdsadminutil.grantsys_object( pobjname => 'V_$MYSTAT', p_grantee => 'YBVOYAGER', p_privilege => 'SELECT'); end; / begin rdsadmin.rdsadminutil.grantsys_object( pobjname => 'V_$STATNAME', p_grantee => 'YBVOYAGER', p_privilege => 'SELECT'); end; / ``` Enable supplemental logging in the database as follows: ```sql exec

begin rdsadmin.rdsadminutil.altersupplemental_logging( p_action => 'ADD', p_type => 'ALL'); end; / ``` Create `ybvoyager_metadata` schema or user, and tables for voyager to use during migration as follows: ```sql CREATE USER ybvoyager_metadata IDENTIFIED BY "password"; GRANT CONNECT, RESOURCE TO ybvoyager_metadata; ALTER USER ybvoyager_metadata QUOTA UNLIMITED ON USERS; CREATE TABLE ybvoyagermetadata.ybvoyagerimportdataeventchannelsmetainfo ( migration_uuid VARCHAR2(36), channel_no INT, lastappliedvsn NUMBER(19), num_inserts NUMBER(19), num_updates NUMBER(19), num_deletes NUMBER(19), PRIMARY KEY (migrationuuid, channelno) ); CREATE TABLE ybvoyagermetadata.ybvoyagerimportedeventcountbytable ( migration_uuid VARCHAR2(36), table_name VARCHAR2(250), channel_no INT, total_events NUMBER(19), num_inserts NUMBER(19), num_updates NUMBER(19), num_deletes NUMBER(19), PRIMARY KEY (migrationuuid, tablename, channel_no) ); ``` Create a writer role for the source schema for Voyager to be able to write the changes from the target YugabyteDB database to the source database (in case of a fall-back): ```sql CREATE ROLE <SCHEMANAME>writer_role; BEGIN FOR R IN (SELECT owner, objectname FROM allobjects WHERE owner=UPPER('<SCHEMANAME>') and objecttype ='TABLE' MINUS SELECT owner, tablename from allnestedtables where owner = UPPER('<SCHEMANAME>')) LOOP EXECUTE IMMEDIATE 'GRANT SELECT, INSERT, UPDATE, DELETE, ALTER on '||R.owner||'."'||R.objectname||'" to <SCHEMANAME>writerrole'; END LOOP; END; / DECLARE v_sql VARCHAR2(4000); BEGIN FOR tablerec IN (SELECT tablename FROM alltables WHERE owner = 'YBVOYAGERMETADATA') LOOP vsql := 'GRANT ALL PRIVILEGES ON YBVOYAGERMETADATA.' || tablerec.tablename || ' TO <SCHEMANAME>writer_role'; EXECUTE IMMEDIATE v_sql; END LOOP; END; / GRANT CREATE ANY SEQUENCE, SELECT ANY SEQUENCE, ALTER ANY SEQUENCE TO <SCHEMANAME>writer_role; ``` Assign the writer role to the source database user as follows: ```sql GRANT <SCHEMANAME>writer_role TO ybvoyager; ``` {{% /tab %}} {{< /tabpane >}} </div> <div id="pg" class="tab-pane fade" role="tabpanel" aria-labelledby="pg-tab"> {{< tabpane text=true >}} {{% tab header="Standalone PostgreSQL" %}} ybvoyager requires `wallevel` to be logical. You can check this using following the steps: Run the command `SHOW wal_level` on the database to check the value. If the value is anything other than logical, run the command `SHOW config_file` to know the path of your configuration file. Modify the configuration file by uncommenting the parameter `wal_level` and set the value to logical. Restart PostgreSQL. Check that the replica identity is FULL (`f`) for all tables on the database. Check the replica identity using the following query: ```sql SELECT n.nspname, relname, relreplident FROM pgclass c JOIN pgnamespace n on c.relnamespace = n.oid WHERE n.nspname in (<SCHEMA_LIST>) and relkind = 'r'; SCHEMALIST used is a comma-separated list of schemas, for example, SCHEMALIST 'abc','public', 'xyz'. ``` Change the replica identity of all tables if the tables have an identity other than FULL (`f`), using the following query: ```sql DO $$ DECLARE r Record; BEGIN FOR r IN (SELECT tableschema, '"' || tablename || '"' as tname FROM informationschema.tables WHERE tableschema IN (<SCHEMALIST>) AND table_type = 'BASE TABLE') LOOP EXECUTE 'ALTER TABLE ' || r.tableschema || '.' || r.tname || ' REPLICA IDENTITY FULL'; END LOOP; END $$; SCHEMALIST used is a comma-separated list of schemas, for example, SCHEMALIST 'abc','public', 'xyz'. ``` Create user `ybvoyager` for the migration using the following command: ```sql CREATE USER ybvoyager PASSWORD 'password' REPLICATION; ``` Switch to the database that you want to migrate as follows: ```sql \c <database_name> ``` Grant the `USAGE` permission to the `ybvoyager` user on all schemas of the database as follows: ```sql SELECT 'GRANT USAGE ON SCHEMA ' || schemaname || ' TO ybvoyager;' FROM informationschema.schemata; \\gexec ``` The preceding `SELECT` statement generates a list of `GRANT USAGE` statements which are then executed by `psql` because of the `\\gexec` switch. The `\\gexec` switch works for PostgreSQL v9.6 and later. For earlier versions, you'll have to manually execute the `GRANT USAGE ON SCHEMA schema_name TO ybvoyager` statement, for each schema in the source PostgreSQL

Grant `SELECT` permission on all the tables and sequences as follows: ```sql SELECT 'GRANT SELECT ON ALL TABLES IN SCHEMA ' || schemaname || ' TO ybvoyager;' FROM informationschema.schemata; \\gexec SELECT 'GRANT SELECT ON ALL SEQUENCES IN SCHEMA ' || schemaname || ' TO ybvoyager;' FROM informationschema.schemata; \\gexec ``` Create a replication group as follows: ```sql CREATE ROLE replication_group; ``` Add the original owner of the table to the group as follows: ```sql GRANT replicationgroup TO <originalowner>; ``` Add the user `ybvoyager` to the replication group as follows: ```sql GRANT replication_group TO ybvoyager; ``` Transfer ownership of the tables to the role `replication_group` as follows: ```sql DO $$ DECLARE r Record; BEGIN FOR r IN (SELECT tableschema, '"' || tablename || '"' as tname FROM informationschema.tables WHERE tableschema IN (<SCHEMALIST>)) LOOP EXECUTE 'ALTER TABLE ' || r.tableschema || '.' || r.tname || ' OWNER TO replication_group'; END LOOP; END $$; SCHEMALIST used is a comma-separated list of schemas, for example, SCHEMALIST 'abc','public', 'xyz'. ``` Grant `CREATE` privilege on the source database to `ybvoyager` as follows: ```sql GRANT CREATE ON DATABASE <database_name> TO ybvoyager; --required to create a publication. ``` The `ybvoyager` user can now be used for migration. Provide write privileges on all tables in the schemas to the Voyager user to be able to write the changes from target YugabyteDB to the source database (in case of a fall-back). ```sql SELECT 'GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA ' || schemaname || ' TO ybvoyager;' FROM informationschema.schemata where schemaname IN (<SCHEMALIST>) ; \\gexec SCHEMALIST used is a comma-separated list of schemas, for example, SCHEMALIST 'abc','public', 'xyz'. ``` {{% /tab %}} {{% tab header="RDS PostgreSQL" %}} ybvoyager requires `wallevel` to be logical. This is controlled by a database parameter `rds.logical_replication` which needs to be set to 1. You can check this using following the steps: Run the command `SHOW rds.logical_replication` on the database to check whether the parameter is set. If the parameter is not set, you can change the parameter value to 1 from the RDS console of the database; navigate to Configuration > Parameter group > `rds.logical_replication`. If the `rds.logical_replication` errors out (after the change), create a new parameter group with the value as 1, and assign it to the database instance from the Modify option on the RDS console. Restart RDS. Check that the replica identity is FULL (`f`) for all tables on the database. Check the Replica identity for all the tables on the database as follows: ```sql SELECT n.nspname, relname, relreplident FROM pgclaspgclass c JOIN pg_namespace n on c.relnamespace = n.oids WHERE n.nspname in (<SCHEMA_LIST>) and relkind = 'r'; SCHEMALIST used is a comma-separated list of schemas, for example, SCHEMALIST 'abc','public', 'xyz'. ``` Change the replica identity of all tables if the tables have an identity other than FULL (`f`), using the following query: ```sql DO $$ DECLARE r Record; BEGIN FOR r IN (SELECT tableschema, '"' || tablename || '"' as tname FROM informationschema.tables WHERE tableschema IN (<SCHEMALIST>) AND table_type = 'BASE TABLE') LOOP EXECUTE 'ALTER TABLE ' || r.tableschema || '.' || r.tname || ' REPLICA IDENTITY FULL'; END LOOP; END $$; SCHEMALIST used is a comma-separated list of schemas, for example, SCHEMALIST 'abc','public',

``` Create user `ybvoyager` for the migration using the following command: ```sql CREATE USER ybvoyager PASSWORD 'password'; GRANT rds_replication to ybvoyager; ``` Switch to the database that you want to migrate as follows: ```sql \\c <database_name> ``` Grant the `USAGE` permission to the `ybvoyager` user on all schemas of the database as follows: ```sql SELECT 'GRANT USAGE ON SCHEMA ' || schemaname || ' TO ybvoyager;' FROM informationschema.schemata; \\gexec ``` The preceding `SELECT` statement generates a list of `GRANT USAGE` statements which are then executed by `psql` because of the `\\gexec` switch. The `\\gexec` switch works for PostgreSQL v9.6 and later. For older versions, you'll have to manually execute the `GRANT USAGE ON SCHEMA schema_name TO ybvoyager` statement, for each schema in the source PostgreSQL database. Grant `SELECT` permission on all the tables and sequences as follows: ```sql SELECT 'GRANT SELECT ON ALL TABLES IN SCHEMA ' || schemaname || ' TO ybvoyager;' FROM informationschema.schemata; \\gexec SELECT 'GRANT SELECT ON ALL SEQUENCES IN SCHEMA ' || schemaname || ' TO ybvoyager;' FROM informationschema.schemata; \\gexec ``` Create a replication group as follows: ```sql CREATE ROLE replication_group; ``` Add the original owner of the table to the group as follows: ```sql GRANT replicationgroup TO <originalowner>; ``` Add the user `ybvoyager` to the replication group as follows: ```sql GRANT replication_group TO ybvoyager; ``` Transfer ownership of the tables to the role <replication_group> as follows: ```sql DO $$ DECLARE r Record; BEGIN FOR r IN (SELECT tableschema, '"' || tablename || '"' as tname FROM informationschema.tables WHERE tableschema IN (<SCHEMALIST>)) LOOP EXECUTE 'ALTER TABLE ' || r.tableschema || '.' || r.tname || ' OWNER TO replication_group'; END LOOP; END $$; SCHEMALIST used is a comma-separated list of schemas, for example, SCHEMALIST 'abc','public', 'xyz'. ``` Grant `CREATE` privilege on the source database to `ybvoyager` as follows: ```sql GRANT CREATE ON DATABASE <database_name> TO ybvoyager; --required to create a publication. ``` The `ybvoyager` user can now be used for migration. Provide write privileges on all tables in the schemas to the Voyager user to be able to write the changes from target YugabyteDB to the source database (in case of a fall-back): ```sql SELECT 'GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA ' || schemaname || ' TO ybvoyager;' FROM informationschema.schemata where schemaname IN (<SCHEMALIST>) ; \\gexec SCHEMALIST used is a comma-separated list of schemas, for example, SCHEMALIST 'abc','public', 'xyz'. ``` {{% /tab %}} {{< /tabpane >}} </div> If you want yb-voyager to connect to the source database over SSL, refer to . {{< note title="Connecting to Oracle instances" >}} You can use only one of the following arguments to connect to your Oracle instance: --source-db-schema (Schema name of the source database.) --oracle-db-sid (Oracle System Identifier you can use while exporting data from Oracle instances.) --oracle-tns-alias (TNS (Transparent Network Substrate) alias configured to establish a secure connection with the server.) {{< /note >}} Prepare your target YugabyteDB database cluster by creating a database, and a user for your cluster. {{<note title="Important">}} Add the following flags to the cluster before starting migration, and revert them after the migration is complete. For the target YugabyteDB versions `2.18.5.1` and `2.18.6.0` (or later minor versions), set the following flag: ```sh ysqlpgconfcsv = ybmaxquerylayer_retries=0 ``` For all the other target YugabyteDB versions, set the following flags: ```sh ysqlmaxreadrestartattempts = 0 ysqlmaxwriterestartattempts = 0 ``` Turn off the isolation level on the target YugabyteDB cluster during the migration. {{</note>}} Create the target YugabyteDB database in your YugabyteDB cluster. The database name can be the same or different from the source database name. If you don't provide the target YugabyteDB database name during import, yb-voyager assumes the target YugabyteDB database name is `yugabyte`. To specify the target YugabyteDB database name during import, use the `--target-db-name` argument with the `yb-voyager import` commands. ```sql CREATE DATABASE targetdbname; ``` Create a user with

For a local YugabyteDB cluster or YugabyteDB Anywhere, create a user and role with the superuser privileges using the following command: ```sql CREATE USER ybvoyager SUPERUSER PASSWORD 'password'; ``` For YugabyteDB Managed, create a user with role using the following command: ```sql CREATE USER ybvoyager PASSWORD 'password'; GRANT yb_superuser TO ybvoyager; ``` If you want yb-voyager to connect to the target YugabyteDB database over SSL, refer to . {{< warning title="Deleting the ybvoyager user" >}} After migration, all the migrated objects (tables, views, and so on) are owned by the `ybvoyager` user. You should transfer the ownership of the objects to some other user (for example, `yugabyte`) and then delete the `ybvoyager` user. Example steps to delete the user are: ```sql REASSIGN OWNED BY ybvoyager TO yugabyte; DROP OWNED BY ybvoyager; DROP USER ybvoyager; ``` {{< /warning >}} yb-voyager keeps all of its migration state, including exported schema and data, in a local directory called the export directory. Before starting migration, you should create the export directory on a file system that has enough space to keep the entire source database. Next, you should provide the path of the export directory as a mandatory argument (`--export-dir`) to each invocation of the yb-voyager command in an environment variable. ```sh mkdir $HOME/export-dir export EXPORT_DIR=$HOME/export-dir ``` The export directory has the following sub-directories and files: `reports` directory contains the generated Schema Analysis Report. `schema` directory contains the source database schema translated to PostgreSQL. The schema is partitioned into smaller files by the schema object type such as tables, views, and so on. `data` directory contains CSV (Comma Separated Values) files that are passed to the COPY command on the target YugabyteDB database. `metainfo` and `temp` directories are used by yb-voyager for internal bookkeeping. `logs` directory contains the log files for each command. Proceed with schema and data migration using the following steps: To begin, export the schema from the source database. Once exported, analyze the schema and apply any necessary manual changes. The `yb-voyager export schema` command extracts the schema from the source database, converts it into PostgreSQL format (if the source database is Oracle or MySQL), and dumps the SQL DDL files in the `EXPORT_DIR/schema/*` directories. {{< note title="Usage for sourcedbschema" >}} The `sourcedbschema` argument specifies the schema of the source database. For Oracle, `source-db-schema` can take only one schema name and you can migrate only one schema at a time. {{< /note >}} An example invocation of the command with required arguments is as follows: ```sh yb-voyager export schema --export-dir <EXPORT_DIR> \ --source-db-type <SOURCEDBTYPE> \ --source-db-host <SOURCEDBHOST> \ --source-db-user <SOURCEDBUSER> \ --source-db-password <SOURCEDBPASSWORD> \ # Enclose the password in single quotes if it contains special characters. --source-db-name <SOURCEDBNAME> \ --source-db-schema <SOURCEDBSCHEMA> ``` Refer to for details about the arguments. The schema exported in the previous step may not yet be suitable for importing into YugabyteDB. Even though YugabyteDB is PostgreSQL compatible, given its distributed nature, you may need to make minor manual changes to the schema. The `yb-voyager analyze-schema` command analyses the PostgreSQL schema dumped in the step, and prepares a report that lists the DDL statements which need manual changes. An example invocation of the command with required arguments is as follows: ```sh yb-voyager analyze-schema --export-dir <EXPORT_DIR> --output-format <FORMAT> ``` The preceding command generates a report file under the `EXPORT_DIR/reports/` directory. Refer to for details about the arguments. Fix all the issues listed in the generated schema analysis report by manually editing the SQL DDL files from the `EXPORT_DIR/schema/*`. After making the manual changes, re-run the `yb-voyager analyze-schema` command. This generates a fresh report using your changes. Repeat these steps until the generated report contains no

To learn more about modelling strategies using YugabyteDB, refer to . {{< note title="Manual schema changes" >}} Include the primary key definition in the `CREATE TABLE` statement. Primary Key cannot be added to a partitioned table using the `ALTER TABLE` statement. {{< /note >}} Import the schema using the `yb-voyager import schema` command. {{< note title="Usage for targetdbschema" >}} `yb-voyager` imports the source database into the `public` schema of the target YugabyteDB database. By specifying `--target-db-schema` argument during import, you can instruct `yb-voyager` to create a non-public schema and use it for the schema/data import. {{< /note >}} An example invocation of the command with required arguments is as follows: ```sh yb-voyager import schema --export-dir <EXPORT_DIR> \ --target-db-host <TARGETDBHOST> \ --target-db-user <TARGETDBUSER> \ --target-db-password <TARGETDBPASSWORD> \ # Enclose the password in single quotes if it contains special characters.. --target-db-name <TARGETDBNAME> \ --target-db-schema <TARGETDBSCHEMA> ``` Refer to for details about the arguments. yb-voyager applies the DDL SQL files located in the `$EXPORT_DIR/schema` directory to the target YugabyteDB database. If yb-voyager terminates before it imports the entire schema, you can rerun it by adding the `--ignore-exist` option. {{< note title="Importing indexes and triggers" >}} Because the presence of indexes and triggers can slow down the rate at which data is imported, by default `import schema` does not import indexes (except UNIQUE indexes to avoid any issues during import of schema because of foreign key dependencies on the index) and triggers. You should complete the data import without creating indexes and triggers. Only after data import is complete, create indexes and triggers using the `import schema` command with an additional `--post-snapshot-import` flag. {{< /note >}} Begin exporting data from the source database into the `EXPORT_DIR/data` directory using the yb-voyager export data from source command with required arguments as follows: ```sh yb-voyager export data from source --export-dir <EXPORT_DIR> \ --source-db-type <SOURCEDBTYPE> \ --source-db-host <SOURCEDBHOST> \ --source-db-user <SOURCEDBUSER> \ --source-db-password <SOURCEDBPASSWORD> \ # Enclose the password in single quotes if it contains special characters. --source-db-name <SOURCEDBNAME> \ --source-db-schema <SOURCEDBSCHEMA> \ --export-type snapshot-and-changes ``` The export data from source command first ensures that it exports a snapshot of the data already present on the source database. Next, you start a streaming phase (CDC phase) where you begin capturing new changes made to the data on the source after the migration has started. Some important metrics such as the number of events, export rate, and so on, is displayed during the CDC phase similar to the following: ```output | | -- | | Metric | Value | | | -- | | Total Exported Events | 123456 | | Total Exported Events (Current Run) | 123456 | | Export Rate(Last 3 min) | 22133/sec | | Export Rate(Last 10 min) | 21011/sec | | | -- | ``` Note that the CDC phase will start only after a snapshot of the entire table-set is completed. Additionally, the CDC phase is restartable. So, if yb-voyager terminates when data export is in progress, it resumes from its current state after the CDC phase is restarted. Some data types are unsupported. For a detailed list, refer to . For Oracle where sequences are not attached to a column, resume value generation is unsupported. `--parallel-jobs` argument (specifies the number of tables to be exported in parallel from the source database at a time) has no effect on live migration. Refer to for details about the arguments of an export operation. The options passed to the command are similar to the

To export only a subset of the tables, pass a comma-separated list of table names in the `--table-list` argument. Run the `yb-voyager get data-migration-report --export-dir <EXPORT_DIR>` command to get a consolidated report of the overall progress of data migration concerning all the databases involved (source and target). Refer to for details about the arguments. After you have successfully imported the schema in the target YugabyteDB database, you can start importing the data using the yb-voyager import data to target command as follows: ```sh yb-voyager import data to target --export-dir <EXPORT_DIR> \ --target-db-host <TARGETDBHOST> \ --target-db-user <TARGETDBUSER> \ --target-db-password <TARGETDBPASSWORD> \ # Enclose the password in single quotes if it contains special characters. --target-db-name <TARGETDBNAME> \ --target-db-schema <TARGETDBSCHEMA> \ # Oracle only. --parallel-jobs <NUMBEROFJOBS> ``` Refer to for details about the arguments. For the snapshot exported, yb-voyager splits the data dump files (from the $EXPORT_DIR/data directory) into smaller batches. yb-voyager concurrently ingests the batches such that all nodes of the target YugabyteDB database cluster are used. After the snapshot is imported, a similar approach is employed for the CDC phase, where concurrent batches of change events are applied on the target YugabyteDB database cluster. Some important metrics such as the number of events, ingestion rate, and so on, is displayed during the CDC phase similar to the following: ```output | -- | -- | | Metric | Value | | -- | -- | | Total Imported events | 272572 | | Events Imported in this Run | 272572 | | Ingestion Rate (last 3 mins) | 14542 events/sec | | Ingestion Rate (last 10 mins) | 14542 events/sec | | Time taken in this Run | 0.83 mins | | Remaining Events | 4727427 | | Estimated Time to catch up | 5m42s | | -- | -- | ``` The entire import process is designed to be restartable if yb-voyager terminates while the data import is in progress. If restarted, the data import resumes from its current state. {{< note title="Note">}} The arguments `table-list` and `exclude-table-list` are not supported in live migration. For details about the arguments, refer to the . {{< /note >}} {{< tip title="Importing large datasets" >}} When importing a very large database, run the import data to target command in a `screen` session, so that the import is not terminated when the terminal session stops. If the `yb-voyager import data to target` command terminates before completing the data ingestion, you can re-run it with the same arguments and the command will resume the data import operation. {{< /tip >}} Run the following command with required arguments to get a consolidated report of the overall progress of data migration concerning all the databases involved (source and target). ```sh yb-voyager get data-migration-report --export-dir <EXPORT_DIR> \ --target-db-password <TARGETDBPASSWORD> \ --source-db-password <SOURCEDBPASSWORD> ``` Refer to for details about the arguments. Import indexes and triggers on the target YugabyteDB database after the `import data to target` has completed the following tasks: The exported snapshot has been completely imported on the target. All the events accumulated on local disk by during the snapshot import phase and have caught up in the CDC phase (you can monitor the timeline based on `Estimated Time to catch up` metric). After the preceding steps are completed, you can start importing indexes and triggers in parallel with the `import data to target` command using the `import schema` command with an additional `--post-snapshot-import` flag as follows: ```sh yb-voyager import schema --export-dir <EXPORT_DIR> \ --target-db-host <TARGETDBHOST> \ --target-db-user <TARGETDBUSER> \ --target-db-password <TARGETDBPASSWORD> \ # Enclose the password in single quotes if it contains special

--target-db-name <TARGETDBNAME> \ --target-db-schema <TARGETDBSCHEMA> \ --post-snapshot-import true ``` If any of the CREATE INDEX DDLs fail in the preceding command, retry the command with the argument `--ignore-exist` to ignore already created indexes and create new ones instead. ```sh yb-voyager import schema --export-dir <EXPORT_DIR> \ --target-db-host <TARGETDBHOST> \ --target-db-user <TARGETDBUSER> \ --target-db-password <TARGETDBPASSWORD> \ # Enclose the password in single quotes if it contains special characters. --target-db-name <TARGETDBNAME> \ --target-db-schema <TARGETDBSCHEMA> \ --post-snapshot-import true \ --ignore-exist true ``` Refer to for details about the arguments. As the migration continuously exports changes on the source database to the `EXPORT-DIR`, the disk utilization continues to grow indefinitely over time. To limit usage of all the disk space, optionally, you can use the `archive changes` command as follows: ```sh yb-voyager archive changes --export-dir <EXPORT-DIR> --move-to <DESTINATION-DIR> ``` Refer to for details about the arguments. During cutover, you switch your application over from the source database to the target YugabyteDB database. Keep monitoring the metrics displayed for export data from source and import data to target processes. After you notice that the import of events is catching up to the exported events, you are ready to perform a cutover. You can use the "Remaining events" metric displayed in the import data to target process to help you determine the cutover. Perform the following steps as part of the cutover process: Quiesce your source database, that is stop application writes. Perform a cutover after the exported events rate ("Export rate" in the metrics table) drops to 0 using the following command: ```sh yb-voyager initiate cutover to target --export-dir <EXPORT_DIR> --prepare-for-fall-back true ``` Refer to for details about the arguments. As part of the cutover process, the following occurs in the background: The initiate cutover to target command stops the export data from source process, followed by the import data to target process after it has imported all the events to the target YugabyteDB database. The command automatically starts capturing changes from the target YugabyteDB database. Note that the process transforms to an `export data from target` process, so if it gets terminated for any reason, you need to restart the process using the `export data from target` command as suggested in the `import data to target` output. {{<note title="Event duplication">}} The `export data from target` command may result in duplicated events if you restart Voyager, or there is a change in the YugabyteDB database server state. Consequently, the command may display additional events that have been exported from the target YugabyteDB database, and imported into the source database. For such situations, it is recommended to manually verify data in the source or target database to ensure accuracy and consistency. {{</note>}} The command automatically starts applying changes (captured from the target YugabyteDB) back to the source database. Note that the process transforms to a `import data to source` process, so if it gets terminated for any reason, you need to restart the process using `import data to source` command as suggested in the `export data from source` output. Wait for the cutover process to complete. Monitor the status of the cutover process using the following command: ```sh yb-voyager cutover status --export-dir <EXPORT_DIR> ``` Refer to for details about the arguments. If there are in the migration, refresh them manually after cutover. Verify your migration. After the schema and data import is complete, the automated part of the database migration process is considered complete. You should manually run validation queries on both the source and target YugabyteDB database to ensure that the data is correctly migrated. A sample query to validate the databases can include checking the row count of each

{{< warning title = "Caveat associated with rows reported by get data-migration-report" >}} Suppose you have the following scenario: or command fails. To resolve this issue, you delete some of the rows from the split files. After retrying, the import data to target command completes successfully. In this scenario, the command reports an incorrect imported row count because it doesn't take into account the deleted rows. For more details, refer to the GitHub issue . {{< /warning >}} Disable triggers and foreign-key constraints on the source database to ensure that changes from the target YugabyteDB database can be imported correctly to the source database using the following PL/SQL commands on the source schema as a privileged user:<br> {{< tabpane text=true >}} {{% tab header="Oracle" %}} Use the following PL/SQL commands to disable triggers, and disable referential constraints on the source: ```sql --disable triggers BEGIN FOR R IN (SELECT owner, objectname FROM allobjects WHERE owner=UPPER('<SCHEMANAME>') and objecttype ='TABLE' MINUS SELECT owner, tablename from allnestedtables where owner = UPPER('<SCHEMANAME>')) LOOP EXECUTE IMMEDIATE 'ALTER TABLE '||R.owner||'."'||R.object_name||'" DISABLE ALL TRIGGERS'; END LOOP; END; / --disable referential constraints BEGIN FOR c IN (SELECT tablename, constraintname FROM user_constraints WHERE constrainttype IN ('R') AND OWNER = '<SCHEMANAME>') LOOP EXECUTE IMMEDIATE 'ALTER TABLE ' || c.tablename || ' DISABLE CONSTRAINT ' || c.constraintname; END LOOP; END; / ``` {{% /tab %}} {{% tab header="PostgreSQL" %}} Use the following PL/SQL commands to disable triggers, and drop foreign-key constraints on the source: ```sql --disable triggers DO $$ DECLARE r RECORD; BEGIN FOR r IN SELECT tableschema, '"' || tablename || '"' AS t_name FROM information_schema.tables WHERE table_type = 'BASE TABLE' AND tableschema IN (<SCHEMALIST>) LOOP EXECUTE 'ALTER TABLE ' || r.tableschema || '.' || r.tname || ' DISABLE TRIGGER ALL'; END LOOP; END $$; SCHEMALIST used is a comma-separated list of schemas, for example, SCHEMALIST 'abc','public', 'xyz'. --drop referential constraints DO $$ DECLARE fk RECORD; BEGIN FOR fk IN SELECT conname, conrelid::regclass AS table_name FROM pg_constraint JOIN pgclass ON conrelid = pgclass.oid JOIN pgnamespace ON pgnamespace.oid = pg_class.relnamespace WHERE contype = 'f' AND pgnamespace.nspname IN (<SCHEMALIST>) LOOP EXECUTE 'ALTER TABLE ' || fk.table_name || ' DROP CONSTRAINT ' || fk.conname; END LOOP; END $$; SCHEMALIST used is a comma-separated list of schemas, for example, SCHEMALIST 'abc','public', 'xyz'. ``` {{% /tab %}} {{< /tabpane >}} During this phase, switch your application over from the target YugabyteDB database back to the source database. Perform this step only if the target YugabyteDB database is not working as expected. Keep monitoring the metrics displayed for `export data from target` and `import data to source` processes. After you notice that the import of events to the source database is catching up to the exported events from the target YugabyteDB database, you are ready to cutover. You can use the "Remaining events" metric displayed in the `import data to source` process to help you determine the cutover. Perform the following steps as part of the cutover process: Quiesce your target YugabyteDB database, that is stop application writes. Perform a cutover after the exported events rate ("Export rate" in the metrics table) drops to 0 using the following command: ```sh yb-voyager initiate cutover to source --export-dir <EXPORT_DIR> ``` Refer to for details about the arguments. The `initiate cutover to source` command stops the `export data from target` process, followed by the `import data to source` process after it has imported all the events to the source database. Wait for the cutover process to

Monitor the status of the cutover process using the following command: ```sh yb-voyager cutover status --export-dir <EXPORT_DIR> ``` Refer to for details about the arguments. Note that for Oracle migrations, restoring sequences after cutover on the source-replica database is currently unsupported, and you need to restore sequences manually. Re-enable triggers and foreign-key constraints on the source database using the following PL/SQL commands on the source schema as a privileged user: {{< tabpane text=true >}} {{% tab header="Oracle" %}} ```sql --enable triggers BEGIN FOR R IN (SELECT owner, objectname FROM allobjects WHERE owner=UPPER('<SCHEMANAME>') and objecttype ='TABLE' MINUS SELECT owner, tablename from allnestedtables where owner = UPPER('<SCHEMANAME>')) LOOP EXECUTE IMMEDIATE 'ALTER TABLE '||R.owner||'."'||R.object_name||'" ENABLE ALL TRIGGERS'; END LOOP; END; / --enable referential constraints BEGIN FOR c IN (SELECT tablename, constraintname FROM user_constraints WHERE constrainttype IN ('R') AND OWNER = '<SCHEMANAME>' ) LOOP EXECUTE IMMEDIATE 'ALTER TABLE ' || c.tablename || ' ENABLE CONSTRAINT ' || c.constraintname; END LOOP; END; / ``` {{% /tab %}} {{% tab header="PostgreSQL" %}} Use the following PL/SQL to enable the triggers and create foreign key constraints back before using the source again. ```sql --disable triggers DO $$ DECLARE r RECORD; BEGIN FOR r IN SELECT tableschema, '"' || tablename || '"' AS t_name FROM information_schema.tables WHERE table_type = 'BASE TABLE' AND tableschema IN (<SCHEMALIST>) LOOP EXECUTE 'ALTER TABLE ' || r.tableschema || '.' || r.tname || ' ENABLE TRIGGER ALL'; END LOOP; END $$; SCHEMALIST used is a comma-separated list of schemas, for example, SCHEMALIST 'abc','public', 'xyz'. --create referential constraints --you can use schema dump from source which is use to import schema on target YugabyteDB (with the modifications if made in schema migration phase), one copy of the pure form of that dump is stored in `$EXPORT_DIR/temp/schema.sql`. ``` {{% /tab %}} {{< /tabpane >}} Verify your migration. After the schema and data import is complete, the automated part of the database migration process is considered complete. You should manually run validation queries on both the source and target databases to ensure that the data is correctly migrated. A sample query to validate the databases can include checking the row count of each table. To complete the migration, you need to clean up the export directory (export-dir), and Voyager state ( Voyager-related metadata) stored in the target YugabyteDB database and source database. Run the `yb-voyager end migration` command to perform the clean up, and to back up the schema, data, migration reports, and log files by providing the backup related flags (mandatory) as follows: ```sh yb-voyager end migration --export-dir <EXPORT_DIR> \ --backup-log-files <true, false, yes, no, 1, 0> \ --backup-data-files <true, false, yes, no, 1, 0> \ --backup-schema-files <true, false, yes, no, 1, 0> \ --save-migration-reports <true, false, yes, no, 1, 0> \ --backup-dir <BACKUP_DIR> ``` Note that after you end the migration, you will not be able to continue further. If you want to back up the schema, data, log files, and the migration reports (`analyze-schema` report and `get data-migration-report` output) for future reference, the command provides an additional argument `--backup-dir`, using which you can pass the path of the directory where the backup content needs to be saved (based on what you choose to back up). Refer to for more details on the arguments. In addition to the Live migration , the following additional limitations apply to the fall-back feature: Fall-back is unsupported with a YugabyteDB cluster running on YugabyteDB Managed. SSL Connectivity is unsupported for export or streaming events from YugabyteDB during `export data from target`. In the fall-back phase, you need to manually disable (and subsequently re-enable if required) constraints/indexes/triggers on the source