Migrate Cognito user pool with AWS Lambda and CDK for IaC
Published: 2023-01-31 08:11:50



During the last week me and my teammates had the objective to change the sign in flow of the mobile application to allow a sign in with preferred username. One little obstacle was, that Cognito attributes were immutable and we will need to create a new Cognito to fulfill the requirement. This led to the need for migration of our user-base.

![](https://cdn-images-1.medium.com/max/2006/1*m8Em-oZLjph-XR8C0oGLDg.png)

There are two possible types of user pool migrations:

 1. Migrate user one at a time, only when the user sign in. This option is slower and requires all users from the user-base to sign in. The advantage is that the password hash is also migrated. It is a valuable option for disaster recovery, because it allows you to have Cognito in another region with same user base.

 2. Batch user migration has the downside, that users are required to perform password reset after the migration. The advantage is, that the batch migration is faster and is easy to perform

This article is primarily for the second option, where we perform the batch user migration. Below is the diagram of the solution:

![List users attribute and write them to csv. Upload the list to S3 bucket](https://cdn-images-1.medium.com/max/2000/1*pXBOyN9LseKejpnOvStGnw.jpeg)

A lambda function will perform listUsers api call to get users attributes. After that the function will save the output in S3 as csv. To import users there is a requirement, that the Cognito headers are present in the csv. To find out the required headers, you can use following command:
{% gist https://gist.github.com/mnanchev/b350b70303514958392ecbbefbaa4b38.js %}
All the headers are required to be present as column in the csv. Some of the values in the csv could be empty. The bare minimum values, that should be filled for a user are:

* name

* email

* email_verified — should be set to true. After the imports user should reset their password. The import in the new Cognito will fail if this field is empty or false

* phone_number_verfied — should be set to true or false, even when it is not in use

* cognito:mfa_enabled — should be set to false before the export

* cognito:username — In our case this was required field to preserve as unique sequence, because it was used in the MongoDB to save user specific values.

I would recommend you to start with one user import to see how it is working. Below is a sample:
{% gist https://gist.github.com/mnanchev/c4c22294a7cba7b150887d1dcc051141.js %}
The full script, that i used to migrate was developed in Python. I have added some comments in each function:
{% gist https://gist.github.com/mnanchev/0345f1ccf4ccae939215e4569f1ac0e5.js %}
To deploy it I used cdk as infrastructure as code. One of the benefits of cdk is the usage of high level constructs, which make the work with it easy and fast:
{% gist https://gist.github.com/mnanchev/041eba475b66580347b4c153abcf5ef1.js %}
After the lambda was invoked, you an object CognitoUsers.csv will be present in the bucket. To import the users in the new Cognito follow following procedure:

 1. Go to Cognito in the management console

 2. Select “Manage User Pools” and select the name of the new userpool

 3. Go to “Users and groups” , which is below the “General settings”

 4. Select the “Import users” button

 5. “Create import job”, it is required, that you created a role or the role will be created during the process with permissions to write to Cloudwatch logs — arn:aws:iam::XXX:role/service-role/Cognito-UserImport-Role. You will select the csv file, that was downloaded from S3

 6. After the job is created you should click on the start.

![Create Users import job with the csv, downloaded from the S3 bucket](https://cdn-images-1.medium.com/max/2000/1*SMfLrmD3I47RGv8ltNGQ6w.png)

![Click on the start button to invoke the job](https://cdn-images-1.medium.com/max/2392/1*akmdFUi_cGggShQZ9m5fig.png)

![Check the status regularly until the job succeed](https://cdn-images-1.medium.com/max/2402/1*0B1C7RhWjJ0OmZLYtupfAQ.png)

**Summary: Until this task, I was convinced, that the export and migration was hard task, simply, because all transfers out of the cloud were not so easy and there is always the vendor lock-in and the specifics of some of the services. I was pleasantly surprised, that the Cognito user migration was easy,at least between two Cognitos, and I finished it for a couple of hours, even with the change of the attributes. My next task is improve the whole process by automating also the import with the help of a Step Function**
[**Configuring User Pool Attributes**](https://docs.aws.amazon.com/cognito/latest/developerguide/user-pool-settings-attributes.html)
[**既存のログイン認証基盤からCognitoに移行するために調べてわかったこと - Qiita**](https://qiita.com/SuyamaDaichi/items/70a152b0ad8d7f669d0a)

[https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/cognito-idp.html](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/cognito-idp.html)
[**Creating the User Import .csv File**](https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-pools-using-import-tool-csv-header.html)