Skip to content

Migrating data from MongoDB 3 to 5

Back up, copy, restore, and validate your MongoDB databases. If you have a large set of data to migrate, consider the process in Migrating large-scale data from MongoDB 3 to 5 for a more efficient approach.

Before you begin

Before migrating your data, check the verification section below to understand how data migration is validated. Note that 'Mongo' refers to MongoDB 3, and 'Mongo5' to MongoDB 5 in this article.

  • Back up your Mongo data.
  • Ensure that Mongo5 is already installed.

  • Check that Mongo and Mongo5 pods are in running status.

    $ kubectl -n connections get pod | grep mongo

    |mongo-0|2/2|Running|0|11h| |mongo5-0|2/2|Running|0|11h|

  • Ensure that the user account has the necessary access rights to perform the steps in this task.

  • Note that you will use the following PV storage components to backup and restore MongoDB data:

    Component Pack service Required storage component Directories
    MongoDB 3 (for backup) Mongo /pv-connections/mongo-node-0/data/db
    MongoDB 5 (to restore) Mongo5 /pv-connections/mongo5-node-0/data/db

Note: To avoid losing new data while migrating the existing data, run this task during a maintenance window.

Migrate MongoDB data

  1. Copy the backup from Mongo to the new Mongo5 database.

    Note: The following steps depend on the type of file system used, for example NFS, EFS, and the like.

    1. Ensure that you have the necessary permissions to complete the following steps.

    2. Go to the location which was configured as the PV storage component directory on your Mongo database, for example: /mnt/pv-connections/mongo-node-0/data/db/backups or NFS master: /pv-connections/mongo-node-0/data/db/backups

    3. Copy the Mongo backup folder to the Mongo5 directory, for example: cp -rfp /pv-connections/mongo-node-0/data/db/backups /pv-connections/mongo5-node-0/data/db

  2. Restore data with x509 Authentication activated.

    1. Connect to a Mongo5 pod by running the following command:

      kubectl exec -ti -n connections $(kubectl get pods -n connections |grep mongo5-0|awk '{print $1}') -- bash

    2. Restore the database:

      mongorestore --ssl --sslPEMKeyFile /etc/mongodb/x509/user_admin.pem --sslCAFile /etc/mongodb/x509/mongo-CA-cert.crt --host $(hostname -f) --authenticationDatabase '$external' --authenticationMechanism MONGODB-X509 --username 'C=IE,ST=Ireland,L=Dublin,O=IBM,OU=Connections-Middleware-Clients,CN=admin,emailAddress=admin@mongodb' /data/db/backups/catalog-bkp1

      This command returns the following output:

      o/p: 
……………
……………
2022-08-19T05:47:12.443+0000    44 document(s) restored successfully. 0 document(s) failed to restore.

Verify migration

  1. Verify that the data is migrated successfully.

    1. Connect to a Mongo5 pod by running the following command (if not already):

      kubectl exec -ti -n connections $(kubectl get pods -n connections |grep mongo5-0|awk '{print $1}') -- bash

    2. Connect to a Mongo5 daemon:

      mongosh --tls --tlsCertificateKeyFile /etc/mongodb/x509/user_admin.pem --tlsCAFile /etc/mongodb/x509/mongo-CA-cert.crt --host $(hostname -f)  --authenticationDatabase '$external' --authenticationMechanism MONGODB-X509

      This command returns the following output:

      o/p: Using MongoDB:        5.0.14
Using Mongosh:        1.6.0

    3. Verify that the Connections databases inside Mongo5 are created during the restoration process:

      show databases

      This command returns the following output:

      o/p: 
AppReg               268.00 KiB
ITM                  168.00 KiB
admin                172.00 KiB
boards-app           392.00 KiB
boards-licence       128.00 KiB
boards-notification  316.00 KiB
boards-user          388.00 KiB
cnx-slack-servicedb   32.00 KiB
commtpldb             44.00 KiB
config               260.00 KiB
datamigrationdb        1.23 MiB
local                  3.09 MiB
profiledb            816.00 KiB
relationshipdb         2.08 MiB

    4. Verify collections:

      use commtpldb
show collections

      This command returns the following output:

      o/p: 
CommTemplate
TemplateCategory

  2. Verify the data population on the screen from one of the components. The following verification procedure uses the community-template-service as an example to demonstrate the data migration on the interface.

    1. The deployment community-template-service uses Mongo3. Before starting to migrate data from Mongo3 to Mongo5, the following three templates are created as follows:

      Examples of community templates

      As Mongo3 is in use, these templates are stored in the Mongo3 database.

    2. Switch deployment community-template-service to use Mongo5.

      kubectl -n connections edit deployment community-template-service

Edit the following to mongo5 :

        - name: MONGODB_HOST
          value: mongo5

......

        - name: MONGO_RS_MEMBERS_HOSTS
          valueFrom:
            configMapKeyRef:
              key: mongo5-rs-members-hosts
              name: connections-env

    3. Since there is no data created and migrated in Mongo5 yet, the database used for community-template-service is blank.

      Screenshot of a blank community template database

    4. Perform the previous data migration procedure and verify the community-template-service screen. If the steps are correctly done, the community-template-service screen should populate the data from Mongo3, as seen in step 5a.

Parent topic: Steps to install or upgrade to Component Pack 8