Akshay Mishra Akshay Mishra

Overview of Migration from Mongo DB to AWS DynamoDB

March 11, 2022 | Comments(1) |

In recent times, we have seen a rise in the volume of data used by the IT industry for analytics, processing, etc. The traditional way of storing the data is using a relational database and an on-premises server. However, this type of setup will handle limited requests because of limited infrastructure capabilities.

Data nowadays is becoming dynamic, unstructured, and rapidly changing. So, to fulfill this kind of data required, we can use a no-SQL database. We use programmable languages to query your data instead of using SQL queries for the CRUD (Create, read, update, delete) actions. Hence the No-SQL Database is in sudden demand due to its lightness and capabilities required to overcome the rapidly changing data and handle the vast amount of complex unstructured data.

Two extensive services helping us today are Mongo DB and AWS DynamoDB. Both are No-SQL databases with similar capabilities, and their advantages like MongoDB is platform-agnostic, but AWS DynamoDB is a managed AWS service. On the other side, MongoDB is limited to the infrastructure it is hosted on. In contrast, AWS DynamoDB is highly scalable, provides high throughput with low latency, and supports global tables.

Some of the key terms which are similar but differently named are shown below.

AWS MongoDB

If you are seeking to improve your data handling capability you might be interested in migrating from MongoDB to AWS DynamoDB to enjoy a highly scalable managed infrastructure.

Today we will be using an AWS service known as AWS Data Migration Service to migrate the data.

AWS Database Migration Service is beneficial for migration from either on-premises, EC2 instance, Standalone servers, or AWS RDS to an AWS backed Database service with almost negligible downtime.

I have used a ubuntu 18.04 OS server to deploy my source on-premises MongoDB Database. I assume you have some data in your collection in the MongoDB database node, which is now ready to be sent to Amazon DynamoDB.

Let us get Started

AWS DMS offers two modes for migration

  • Document Mode: In this mode, all the data in the collection is transferred as a single column in the DynamoDB table
  • Table Mode: Here, DMS scan some specified documents in the table to create a schema which then helps the DMS to put them as fields in DynamoDB

We will be using the ‘table’ mode for the migration

  1. Let us start by creating a role in your AWS account which will allow the DMS service to have access to DynamoDB

  • Go to IAM, then create a role
    MongoDB
  • Choose DMS
  • Click next permissions and add DynamoDB full access policy to the role
    MongoDB
  • Use tags if required and name the role as DMS-dynamo-role
  • Click Save

2. Let us go to DMS (Database migration service) in AWS Console

  • We will be required to create a replication server in a VPC that will perform the migration
  • Choose the name of your choice
  • Choose VPC where you want to launch
  • Choose the instance type: here, we have chosen t3.micro
  • Choose Dev workload (Single AZ) for Demo purpose for production workload you can opt for multi-AZ
    MongoDB Migration

3. Jump to your MongoDB database and provide the permissions

These permissions are required for the Replication server to access the DB

  • Use Admin to create the below role
  • Use your Database then to create a user with the role above
    MongoDB

4. Now, we will be creating Source and Target endpoints

These endpoints will be required for the DMS task to transfer the data from source DB to DynamoDB

  • Create Source endpoint
  • Give Source endpoint name
  • Server Name as public Ip of MongoDB machine
  • Name the DB which has the required permissions we created in the earlier step
  • The endpoint will be created as below
    MongoDB
  • Create Target endpoint
  • Give Target endpoint name
  • Choose Target engine as DynamoDB
  • The endpoint will be created as below
    MongoDB

5. Test the endpoint connection for the Source DB connection. It will show as Successful if configured correctly
MongoDB Migration

6. Create a Database migration task in the DMS console

  • Create Task
    MongoDB
  • Give the task a name
  • Select the replication server we created
  • Choose Source and target endpoints
  • Choose table mode
  • Just as we complete creating task
  • Click on create and Database migration will take place. Time consumed for the task to complete will depend on the size of the data migrated
    MongoDB

You can now go to DynamoDB in the AWS console and see your tables created
MongoDB

Conclusion

Migrating your No-SQL data to AWS DynamoDB is an up-gradation of infrastructure that will help provide single-digit millisecond performance. In addition, your data is now globally available for better throughput, which also supports encryption of your data at rest. For your CRUD operations, you can also follow up AWS documentation to merge your current application with AWS DynamoDB using APIs, SDKs, etc.

Big enterprises like Disney, Zoom, Dropbox are taking advantage of the seamless experience provided by AWS DynamoDB.

About CloudThat

We at CloudThat, are AWS (Amazon Web Services) Advanced Consulting Partner and Training partner and Microsoft gold partner, helping people to develop knowledge on cloud and help their businesses to aim for higher goals using best in industry cloud computing practices and knowledge.

Feel free to drop a comment or any queries that you have regarding AWS services, cloud and data migration, consulting and we will get back to you quickly. To get started, go through our Expert Advisory page and Managed Services Package that is CloudThat‘s offerings.

One Response to “Overview of Migration from Mongo DB to AWS DynamoDB”

Leave a Reply