AWS Data Migration Service (DMS) - MySQL to S3 Bucket services
Tags: aws, database, community
Published: 2023-02-25 19:20:00

**The article contains two parts:**

**Part- 1**
Some configuration for the migration:
- Create an IAM role to allow access to the target
- Create S3 bucket to be the target endpoint.

**Part-2**
- Margate the MySQL Database to S3 bucket.




**outline**

- Margate the MySQL Database to S3 bucket
- Data migration service architecture
- Creation of Data Migration Service (DMS)

**Margate the MySQL Database to S3 bucket**

AWS Database Migration Service (AWS DMS) is a managed migration and replication service that helps move your database and analytics workloads to AWS quickly, securely, and with minimal downtime and zero data loss. AWS DMS supports migration between 20-plus database and analytics engines, such as Oracle to Amazon Aurora MySQL-Compatible Edition, MySQL to Amazon Relational Database (RDS) for MySQL, Microsoft SQL Server to Amazon Aurora PostgreSQL-Compatible
Edition, MongoDB to Amazon DocumentDB (with MongoDB compatibility), Oracle to Amazon Redshift, and Amazon Simple Storage Service (S3).

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xy6p8reexnet6kwyrayf.png)

**Migration use cases:**

**• Move to managed databases:**
To migrate data from legacy or on-premises databases to managed cloud services through a streamlined migration process.

**• Remove licensing costs and accelerate business growth:**
To update purpose built databases to innovate and build faster for any use case at scale at one tenth of the cost.

**• Replicate ongoing changes:**
Create redundancies of business-critical databases and data stores to minimize downtime and protect against any data loss.

**• Improve integration with data lakes:**
Build data lakes and perform real-time processing on change data from your data stores.

**Data migration service architecture**

The Data Migration Service architecture consists of some configurations:

**• Replication instance
• Database Endpoint
• Database Migration Tasks**

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/eeu36g082z0mliq4zmy5.png)

**• Replication instance**

AWS DMS uses a replication instance to connect to your source data store, read the source data, and format the data for consumption by the target data store. A replication instance also loads the data into the target data store. Most of this processing happens in memory.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ix72nrdns0k5et0y3duk.png)
The creation of DNS architecture replication instance is similar to how we choose an ec2 instance typically we give a very small number of parameters and then our replication instance would be up and running we choose the size of the instance like how we choose an easy instance size and very few parameters whether you want a single ac or multi-ac and then your replication instance would be up and running.

**• Database Endpoint**

This is the stage we specify the connection details of the source and target database.

**- A data source:** is an initial location where a database is created or where physical information is first digitized, however even the most refined data may serve as a source, as long as another process accesses and utilizes it.

**- Target database:** is the database to which you are moving the data or new changes.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vmaszmv5tdgaremo9yz3.png)

**• Database Migration Tasks:**

An AWS Database Migration Service (AWS DMS) task is where all the work happens. You specify what tables (or views) and schemas to use for your migration and any special processing, such as logging requirements, control table data, and error handling.

A task can consist of three major phases:

o Migration of existing data (Full load)
o The application of cached changes
o Ongoing replication (Change Data Capture)


**![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3jzuwk7mretksewevjrb.png)**

Before creating the data migration we need to :
1- Create an **IAM role** to allow access to the target:

To open the IAM console :
- From the services of the your AWS account choose the IAM service :
• Choose **Roles**, then **Create role**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6r0xcw3ac44g6bmlrwgw.png)
• Under **Trusted entity type**, select AWS service.
• From the **Use cases for other AWS services** dropdown list, choose DMS.
• then choose **Next**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1ibbtwr1p7g5rd9p3an6.png)
• On the **Add permissions** tab, choose the permission **Amazons3FullAccess**.
• Choose **Next**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k8tak3032etqhdi243jq.png)

• For **Role name**, enter a unique name for your role, such as s3 migration-role.
• For **Description**, enter descriptive text .

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zdkpntluwvbv0sokil4d.png)
The permission is add

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8w6ndfr65lae0ahunsyg.png)
• Then choose **Create role**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rabc6jkjyhbij3gcxkgu.png)
• The Role is successfully created

And we can add another permissions to the role by :
• choose the role that we are created:
• Select **Add permissions** then **Attach polices**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ilgm00v866l9lfo8wvry.png)
• Search for the polices we need (AmazonRDSFullAccess) and choose it by click the check box.
• After add the polices click on **Add polices**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ja8hjgr1tg857k07eckd.png)
• Notice that all policies have been successfully attached to our role.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5k6bsy10gidu6znditbo.png)
• Copy the **ARN** for the role that will be used in configuring the migration task.




**2- Create the S3 bucket to migrate the data to it:**

Open the S3 console:
• Choose the **Create bucket**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gslljhz37rumv7po9rbi.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/170kiu4eb2k569ctvf7k.png)
• Do some configuration and then choose the **Create bucket**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xsnbx2yoxeg06z8d03yw.png)

The bucket is created and after opening it notice that it is empty

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6yl09cbz5ab4ye7yrou8.png)


**Creation the AWS Database Migration Service**

To begin creation of AWS database migration service:

Open the AWS Database Migration Service console:
• From **(Services )** select the Database Migration Service.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fx4cpv49tvwdybkuinkz.png)

 In the DMS the first step is create an instance replication :

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/727u5xxs8zz2j9b3aawt.png)
• Click on **Create replication instance**

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ij2ezsf4hf5pl93cgia2.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/sxxmk69z1998qyhlp0sp.png)

**In the instance configuration:**

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kbwn59l7rotmcw05n5wz.png)

**• In storage:**

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lenduc51oshjh3d3c2fu.png)
**• In connectivity and security:**

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/is6rd8wxd3u38syvpye5.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/391d92rxga6k8wk399mb.png)
• The replication instance is creating:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3kuegf5noew30zp3joul.png)
• And after some minutes it will be available


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cp1yt4elcz7o8381gbwe.png)

** The second step of AWS Database Migration Service is to create the endpoint:**

In this configuration, you specify the source endpoint for the data and the target endpoint to which the data will be migrated.

• Click on the **Endpoint**, then click on **Create endpoint**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hu2v1btey5ujgxddq8ci.png)

For the endpoint type:
• First, choose Source endpoint.
• If the source endpoint is RDS database instance, select the checkbox.
• Select the RDS instance from which you want to migrate the data.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ey1hhv86lbhc9brfvqn1.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/81ljx5gg750ona4awmsm.png)
• additional endpoint setting, using wizard or editor.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cfwdai612lzrhrn0n8gu.png)
• Add **tag** to the end point (optional)

To test the running of the end point, click on **Run test**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1ablu2eme2g5me3yb2xt.png)
The source endpoint is created .

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/c59cuthee1zzvt9zo45l.png)
Now to create the target endpoint, click again on **Create endpoint**.
• In the endpoint type, select Target endpoint.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6vfyz20mk2ydbmflgdtd.png)

In the target endpoint configuration:
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tu9yrcs73cbqefu31ebt.png)

To test the running of the end point, click on **Run test**.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jxuj22st4db2phd0vy1f.png)

The target endpoint is created .
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/eypc1dqg0jfx92n3gcmm.png)

** The third step of AWS Database Migration Service is to create the database migration task:**

• Click in the left side on **Database migration tasks**.
• Click on **Create task**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lm9hqpeqmqauw9l9gkya.png)

• In the creation of database migration task:
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qoxwwjpuekes7n5fjlfb.png)

You can configure some setting for the migration task:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hzdiy7ph28mzbm9b8648.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b1a5b4p4ww9xkutbcldi.png)

**In migration task startup configuration:**

• You can start the migration automatically on create or manually later
• Add tag (optional)
• Click on **Create task** .


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1ta8aq5a1jyt1zsm7tye.png)
• The migration task has been created, and is in the **creating state**.
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/35hqbn3zi1zhmtzus6la.png)
• Then after refresh it is ready.
• Then after a few minutes, the migrate task is in the running state.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0zowo77pue48g2kt4zh9.png)
• If we go back to the s3 bucket we should be able to see which subfolders the database was migrated.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yh3k1qh6ox3jq80cnpln.png)


References:
[1]https://aws.amazon.com/dms/


[2]https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.html