AWS Database Migration Service (Part 2 of 2)
Tags: aws, database, migration, cloud
Published: 2022-08-25 14:38:25

In the [previous article](https://dev.to/aws-builders/aws-database-migration-service-part-1-of-2-3e6f) we went over the high-level approach around using [AWS DMS](https://aws.amazon.com/dms/) and then created the replication instance on which your migration processing will run and then created the source and target endpoints that manage the connection to the source and target databases. The last step is to create the database migration task.

## Creating your Database Migration Task

So far, we have defined the resource set that is going to do the work as well as the places where data will be coming from and where it will be going. There is one area that we have not yet defined, and that is the database migration task. This task defines the work that will be done. As part of this task, you can specify which tables to use, define any special processing, configure logging, etc. Let’s take a look at creating one of these tasks.

First, go into the _Database migration tasks_ screen in the AWS DMS console and then click the **Create task** button. This will bring up the creation screen, with the first section being _Task configuration_. This section allows you to:

- Provide a _Task identifier_ or name for the task
- Select the _Replication instance_ to use
- Select the _Source database_ endpoint
- Select the _target database_ endpoint
- Select the _Migration type_

The Migration type is where you tell DMS the kind of work that you want this task to perform. There are three different options that you can select. The first is to _Migrate existing data_. Using this as a migration type means that you’re looking to do a one-time copy of the data and would be ideal for doing that one-time migration. The next option is to _Migrate existing data and replicate ongoing changes_. The name pretty much describes what is going on with this approach, and is most appropriate when you need to run both the source and target systems in parallel but want them to stay as updated as possible. This approach is especially common in Data Lake scenarios where data is being moved from a transactional system to an analytics or reporting system. The last migration type option is to _Replicate data changes only_ where you replicate any changes in data but do not perform that one-time migration.

The next major section to complete when creating a migration task is the _Task settings_. Task settings control the behavior of your task and can be configured through a _Wizard_ or through a _JSON editor_. We will use the wizard mode so that we can more easily talk about the major settings.

The first item to configure is the _Target table preparation mode_, or how DMS should be preparing the tables at the target endpoint. There are three options, _Do nothing_, _Drop tables on target_, and _Truncate_. When you select the “do nothing” option then target tables will not be affected. Any tables that do not exist will be created. When you select to drop the tables, then DMS will drop and recreate all affected tables. Truncating means that all tables and metadata remain, but all of the data is removed.

The next item to configure is _Include LOB columns in replication_. LOB are large objects and you have the option as to whether or not you want to include those object columns in the target data. You have three options, the first of which is _Don’t include LOB columns_, and the second of which is _Full LOB mode_; both of which are rather straightforward. The third option is _Limited LOB mode_. In this mode, DMS will truncate each LOB to a defined size, the _Maximum LOB size (kb)_ value.

You then can configure whether you want to _Enable validation_. Checking this box will cause DMS to compare the source and target data immediately after the full load is performed. This ensures your data is migrated correctly, but it takes additional time to perform, and thus increases cost. You next can _Enable CloudWatch logs_. There are also some advanced task settings, but we won’t go into those as part of this discussion.

The next section is _Table mappings_. This section is where you define the rules about what data is moved and how it is moved. At a high-level, you will create a _Selection rule_, which determines the data that you wish to replicate, and then you can create a _Transformation rule_ that modifies the selected data before it is provided to the destination endpoint. The table mappings section also gives you the opportunity to use a Wizard approach or a JSON editor to enter all table mappings in JSON. We will walk through using the wizard.

The first step is to select the **Add new selection rule** button. This expands the selection rule section as shown in Figure 1.

![Figure 1. Creating selection rules for a database migration task](https://i0.wp.com/billthevestguy.com/wp-content/uploads/2022/08/image-9.png)_Figure 1. Creating selection rules for a database migration task_

Expanding the _Schema_ drop-down will show that there is only one option – to _Enter a schema_. Selecting this option will add another textbox in which you can provide the _Source name_. This allows you to limit, by schema, the data that is being selected. You can enter % to select all schemas in the database or enter the schema name. You do the same for the Source table name, entering % if you want all the tables replicated. Once you have those defined, you then select the appropriate _Action_, to either _Include_ or _Exclude_ the items that fit your selection criteria. You can create as many rules as desired, however, you must always have at least one rule with an **include** action.

Once you have the selection rule configured you can _Add column filter_. This allows you to limit the number and type of records. A column filter requires the _Column name_, one or more _Conditions_, and then one or more comparative values. You have the following options for the conditions:

- Less than or equal to
- Greater than or equal to
- Equal to
- Not equal to
- Equal to or between to values
- Not between two values
- Null
- Not null

You can create any number of column filters per each selection rule.

Once you have completed your selection rule you can then add one or more _Transformation rules_. These rules allow you to change or transform schema, table, or column names of some or all the items that you have selected. Since we are simply copying the database across, we do not need to add any of these, especially since any changes will likely break our code!

Your next option is to determine whether you want to _Enable premigration assessment run_. This will warn you of any potential migration issues. Checking the box will expand the UI and present you with a set of Assessments to run as shown in Figure 2.

![Figure 2. Enabling premigration assessment run on a scheduled task](https://i0.wp.com/billthevestguy.com/wp-content/uploads/2022/08/image-10.png)_Figure 2. Enabling premigration assessment run on a scheduled task_

Once you have all of your selection and transformation rules created, you can select to _Start migration task_ either _Automatically on Create_, the default, or _Manually later_. Lastly, add any tags that you desire and click the **Create task** button.

This will bring you back to the database migration tasks list screen where you will see your task being created. Once created you can either start the task manually or allow it to run itself if so configured. You will be able to watch the table count move from _Tables queued_ to _Tables Loading_ to _Tables loaded_ as they are processed. Returning to the AWS DMS Dashboard will show that there is 1 Load complete as shown in Figure 3.

![Figure 3. Dashboard showing completed migration task](https://i0.wp.com/billthevestguy.com/wp-content/uploads/2022/08/image-11.png)_Figure 3. Dashboard showing completed migration task_

For those cases where you simply want to migrate data sets with minimal changes other than perhaps renaming some columns, the Database Migration Service works like a dream. Relatively painless to setup and powerful enough to move data between servers, even servers that are of dissimilar types, such as where we just copied data from SQL Server to Amazon Aurora. However, there is a tool that will help you move more disparate data between different database engines. We will take a look at that tool next.