Guide for Upgrading to MongoDB 5.0 and Rocket.Chat 4.0 on Docker
The Docker way to change the deprecated database storage engine MMAPv1 to WiredTiger on MongoDB, as well as upgrading MongoDB and Rocket.Chat.
Rocket.Chat 4.0.0 has recently been released. The upgrade process from the previous version is not quite simple to be very honest.
Even though you can make the transition from an old 3.x.y version of Rocket.Chat, earlier versions of MongoDB, namely, 3.6 and 4.0, have been deprecated for use with the popular chat server since the new release. MongoDB 3.4 support has also been removed. These changes were first confirmed on pull request #22907 merged two weeks earlier.
In this tutorial, I've documented a complete step-by-step process to make a successful upgrade to Rocket.Chat including upgrading your MongoDB database version to the most recent 5.0.
In addition, there is also a mandatory necessity to change the database storage engine from MMAPv1 (also deprecated) to WiredTiger. Only then can you continue upgrading to newer versions of Rocket.Chat. That procedure, has also been carefully covered in detail within this comprehensive guide.
How To Upgrade Rocket.Chat 3.x.y to the all new 4.0.0
Let's see what you need to upgrade Rocket.Chat to the most recent version with Docker on a Linux server.
Prerequisites
Since this document is particularly focused on simplifying the migration and upgrade process, I will be assuming an existing Rocket.Chat configuration (with mmap storage engine on MongoDB) based on the following deployment guide:
Avimanyu Bandyopadhyay
All necessary requirements have been detailed in the above guide. Aside from familiarity with Linux commands, knowing the basics of Docker Compose will be helpful here. But the essential requirements relevant to this guide are:
- A Rocket.Chat 3.x.y based Docker deployment running live with at least MongoDB 3.4 database.
- Nginx reverse proxy setup used to self-host the above Rocket.Chat server.
Step 0: Backup a database dump of your existing MongoDB
For your data safety, it's better to create and save your existing MongoDB database first:
mkdir ~/backup
cd ~/rocketchat
docker-compose exec mongo mongodump --archive=/dump/mmap --gzip
cp data/dump/mmap ~/backup/mongo-mmap-dump.gzip
I'd also suggest you have a complete backup of your Rocket.Chat instance as well.
Avimanyu Bandyopadhyay
You will now learn the following three things:
- How to change the deprecated database storage engine mmap to wiredTiger
- Upgrade your MongoDB instance
- Upgrade your Rocket.Chat instance
Step 1: Stop your existing Rocket.Chat system including all its services
This step is particularly important because you do not want any data corruption on your database - MongoDB. Just to be sure, performing a migration and upgrade on a running database instance better be avoided, even through scaling.
docker-compose stop
Unlike, docker-compose down, the above command will only stop your containers without removing them, thus keeping downtime as low as possible.
Step 2: Download the "MongoDB mmap to wiredTiger migration" repository
Using git, fetch the official migration repo into a separate location on your server. Think of this as a migration toolkit provided for your convenience from the Rocket.Chat developers.
git clone https://github.com/RocketChat/docker-mmap-to-wiredtiger-migration ~/rocketchat-migration
Step 3: Copy the "docker" directory from the downloaded repo
This directory holds the Dockerfile of the custom migrator image. For the newly revised Docker Compose file, you need this in your existing compose folder:
cp -r ~/rocketchat-migration/docker ~/rocketchat/docker
Step 4: Backup your existing Docker Compose file
Here, I've named it docker-compose.old.yml. You can use it as a reference when you use the new configuration (discussed in the next step):
mv ~/rocketchat/docker-compose.yml ~/rocketchat/docker-compose.old.yml
Step 5: Use the new migration based Docker Compose file
The newly downloaded migration repo also includes a new Docker Compose file for you to begin the process easily. However, in our case, I've had to follow the below configuration since it is based on a reverse proxy setup:
version: "3.7"
services:
rocketchat:
image: rocketchat/rocket.chat:3.18.2
command: >
bash -c
"for (( ; ; )); do
node main.js &&
s=$$? && break || s=$$?;
echo \"Could not start Rocket.Chat. Waiting 5 secs...\";
sleep 5;
done; (exit $$s)"
restart: on-failure
volumes:
- ./uploads:/app/uploads
environment:
- PORT=3000
- ROOT_URL=https://chat.domain.com
- LETSENCRYPT_HOST=chat.domain.com
- VIRTUAL_HOST=chat.domain.com
- MONGO_URL=mongodb://mongo:27017/rocketchat
- MONGO_OPLOG_URL=mongodb://mongo:27017/local
- Accounts_UseDNSDomainCheck=false
depends_on:
- mongo
networks:
- net
- rocket
mongo:
image: mongo:4.0
restart: on-failure
volumes:
- ./data/db:/data/db
- ./data/dump:/dump
command: >
bash -c
"while [ ! -f /data/db/WiredTiger ]; do
echo \"wiredTiger migration hasn't started yet. Waiting 30 secs...\";
sleep 30;
done;
docker-entrypoint.sh mongod --oplogSize 128 --replSet rs0 --storageEngine=wiredTiger;"
depends_on:
- migrator
networks:
- rocket
migrator:
build: ./docker/
volumes:
- ./data/db:/data/db
networks:
- rocket
mongo-init-replica:
image: mongo:4.0
command: >
bash -c
"for (( ; ; )); do
mongo mongo/rocketchat --eval \"
rs.initiate({
_id: 'rs0',
members: [ { _id: 0, host: 'localhost:27017' } ]})\" &&
s=$$? && break || s=$$?;
echo \"Could not reach MongoDB. Waiting 5 secs ...\";
sleep 5;
done; (exit $$s)"
depends_on:
- mongo
networks:
- rocket
networks:
net:
external: true
rocket:
internal: true
Please do not take the above as the final configuration. There are many more steps and changes remaining. You need the above config only for this step. I will discuss the further changes later.
The migrator service will only run once when it finds an mmap storage engine. It would change it to wiredTiger. Otherwise, it would exit itself. The mongo service would also behave in the same manner. Let's continue to the next step to see how.
Step 6: Build the migrator image to start the migration
The following command will build the migrator image and begin the migration process. It will also start up the containers again. Rocket.Chat will wait for the database to finish initialization before starting up finally.
docker-compose up --build -d
To keep a watch on the whole process, you can use the Docker Compose logs command to monitor the complete procedure:
docker-compose logs -f migrator
Wait for the above command to complete until it drops to the terminal prompt with an exit code 0 after the wiredTiger instance performs a graceful shut down:
rocketchat_migrator_1 exited with code 0
Now, monitor the database:
docker-compose logs -f mongo
Give it some time to initialize and finally check your Rocket.Chat instance:
docker-compose logs -f rocketchat
Wait until you notice the "SERVER RUNNING" message. It would be something like this:
rocketchat_1 | ➔ +--------------------------------------------------+
rocketchat_1 | ➔ | SERVER RUNNING |
rocketchat_1 | ➔ +--------------------------------------------------+
rocketchat_1 | ➔ | |
rocketchat_1 | ➔ | Rocket.Chat Version: 3.18.2 |
rocketchat_1 | ➔ | NodeJS Version: 12.22.1 - x64 |
rocketchat_1 | ➔ | MongoDB Version: 4.0.18 |
rocketchat_1 | ➔ | MongoDB Engine: wiredTiger |
rocketchat_1 | ➔ | Platform: linux |
rocketchat_1 | ➔ | Process Port: 3000 |
rocketchat_1 | ➔ | Site URL: https://chat.domain.com |
rocketchat_1 | ➔ | ReplicaSet OpLog: Enabled |
rocketchat_1 | ➔ | Commit Hash: 2ffdd13795 |
rocketchat_1 | ➔ | Commit Branch: HEAD |
rocketchat_1 | ➔ | |
rocketchat_1 | ➔ +--------------------------------------------------+
As you can see above, MongoDB Engine: wiredTiger denotes that your Rocket.Chat instance is now using the WiredTiger storage engine and no longer the deprecated MMAPv1. This means it is now future-ready for any upcoming application updates. But wait, MongoDB Version: 4.0.18 is also deprecated. This can be noticed if you upgrade to Rocket.Chat 4.0.0 without upgrading MongoDB to at least version 4.2:
rocketchat_1 | +--------------------------------------------------+
rocketchat_1 | | SERVER RUNNING |
rocketchat_1 | +--------------------------------------------------+
rocketchat_1 | | |
rocketchat_1 | | Rocket.Chat Version: 4.0.0 |
rocketchat_1 | | NodeJS Version: 12.22.1 - x64 |
rocketchat_1 | | MongoDB Version: 4.0.18 |
rocketchat_1 | | MongoDB Engine: wiredTiger |
rocketchat_1 | | Platform: linux |
rocketchat_1 | | Process Port: 3000 |
rocketchat_1 | | Site URL: https://chat.domain.com |
rocketchat_1 | | ReplicaSet OpLog: Enabled |
rocketchat_1 | | Commit Hash: 2ffdd13795 |
rocketchat_1 | | Commit Branch: HEAD |
rocketchat_1 | | |
rocketchat_1 | +--------------------------------------------------+
rocketchat_1 | +----------------------------------------------------------------------+
rocketchat_1 | | DEPRECATION |
rocketchat_1 | +----------------------------------------------------------------------+
rocketchat_1 | | |
rocketchat_1 | | YOUR CURRENT MONGODB VERSION (4.0.18) IS DEPRECATED. |
rocketchat_1 | | IT WILL NOT BE SUPPORTED ON ROCKET.CHAT VERSION 5.0.0 AND GREATER, |
rocketchat_1 | | PLEASE UPGRADE MONGODB TO VERSION 4.2 OR GREATER |
rocketchat_1 | | |
rocketchat_1 | +----------------------------------------------------------------------+
In case you are using MongoDB 3.4, also note again that Rocket.Chat 4.0.0 does NOT support it. It is better to also upgrade to the latest version of MongoDB and prevent future hiccups.
Step 7: Upgrade MongoDB and Rocket.Chat
Let's suppose you are running MongoDB version 3.4 and Rocket.Chat 3.18.2. To upgrade to version 5.0, the preferred method is to do the upgrade in increments. So, sequentially, you'd upgrade to 3.6, 4.0, 4.2 and 4.4 (prefinal). Finally, you can upgrade to version 5.0. Let us see how:
Step 7a: Stop the instances first
docker-compose stop
Step 7b: Update the Docker Compose file
Inside the mongo and mongo-init-replica service definitions within your Docker Compose file, revise the image version to the second next incremental version as discussed above. So, if you're running MongoDB 3.4, change it version 3.6. Once you've saved it, redeploy the updated config for the changes to take effect:
docker-compose up -d
The MongoDB container will now be recreated based on version 3.6.
Step 7c: Enable features that persist data incompatible with earlier versions of MongoDB
This step is not optional. To be able to upgrade to the next version (>3.6) you must also use the following command on your MongoDB container:
docker-compose exec mongo bash -c 'mongo --eval "db.adminCommand( { setFeatureCompatibilityVersion: \"3.6\" } )"'
To confirm the command was successful, you should see an output like:
MongoDB shell version v3.6.23
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("fb1e57ff-d2ec-4ac0-8187-c1ae8a36418b") }
MongoDB server version: 3.6.23
{
"ok" : 1,
"$clusterTime" : {
"clusterTime" : Timestamp(1633455129, 1),
"signature" : {
"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
"keyId" : NumberLong(0)
}
},
"operationTime" : Timestamp(1633455129, 1)
}
Make sure you see "ok" : 1,. It confirms you enabled the setting. Now go back to step 7a and repeat the process until you have upgraded to version 4.2 and enabled feature compatibility.
Step 7d: Upgrade to Rocket.Chat 4.0.0
Congratulations, you are now fully ready for upgrading to the all new 4.0.0 version because you are now on MongoDB 4.2! You can use scaling to spin up a fresh new Rocket.Chat instance and minimize downtime. Revise the image version to 4.0.0 in the Docker Compose file and you're all set:
docker-compose up -d --scale rocketchat=2 --no-recreate
sleep 20 && docker rm -f rocketchat_rocketchat_1
docker-compose up -d --scale rocketchat=1 --no-recreate
You can also confirm from the logs about the new change:
rocketchat_2 | +--------------------------------------------------+
rocketchat_2 | | SERVER RUNNING |
rocketchat_2 | +--------------------------------------------------+
rocketchat_2 | | |
rocketchat_2 | | Rocket.Chat Version: 4.0.0 |
rocketchat_2 | | NodeJS Version: 12.22.1 - x64 |
rocketchat_2 | | MongoDB Version: 4.2.17 |
rocketchat_2 | | MongoDB Engine: wiredTiger |
rocketchat_2 | | Platform: linux |
rocketchat_2 | | Process Port: 3000 |
rocketchat_2 | | Site URL: https://chat.domain.com |
rocketchat_2 | | ReplicaSet OpLog: Enabled |
rocketchat_2 | | Commit Hash: 2ffdd13795 |
rocketchat_2 | | Commit Branch: HEAD |
rocketchat_2 | | |
rocketchat_2 | +--------------------------------------------------+
Step 8: We're at the End Game Now!
At this stage, it is advisable to resume the database version upgrade, because the new Rocket.Chat now supports MongoDB 5.0. So you can go back to step 7a once again and repeat the same process until 7c for MongoDB 5.0. That way, like Rocket.Chat, your MongoDB Docker instance is also ready for future upgrades. You can confirm from the logs about the final version:
rocketchat_2 | +--------------------------------------------------+
rocketchat_2 | | SERVER RUNNING |
rocketchat_2 | +--------------------------------------------------+
rocketchat_2 | | |
rocketchat_2 | | Rocket.Chat Version: 4.0.0 |
rocketchat_2 | | NodeJS Version: 12.22.1 - x64 |
rocketchat_2 | | MongoDB Version: 5.0.3 |
rocketchat_2 | | MongoDB Engine: wiredTiger |
rocketchat_2 | | Platform: linux |
rocketchat_2 | | Process Port: 3000 |
rocketchat_2 | | Site URL: https://chat.chmod777.ltd |
rocketchat_2 | | ReplicaSet OpLog: Enabled |
rocketchat_2 | | Commit Hash: 2ffdd13795 |
rocketchat_2 | | Commit Branch: HEAD |
rocketchat_2 | | |
rocketchat_2 | +--------------------------------------------------+
That's it then, you are all set! Enjoy your newly upgraded Rocket.Chat server!
Bonus notes
Considering that only Rocket.Chat would be scaled up for updates without downtime from now on usually, I kept the Docker Compose file as is and did not remove the migrator service definition or revise the mongo database service definition based on the original setting.
Once the wiredTiger storage engine is in charge, they are ignored and seem quite future-proof. Who knows, it might prove useful in future deployments and upgrades? Here's the log report for the migrator service when you stop and restart an already existent WiredTiger storage engine configuration:
Attaching to rocketchat_migrator_1
migrator_1 | Found /data/db/WiredTiger - assuming migration to WiredTiger already completed...
migrator_1 | Found /data/db/WiredTiger - assuming migration to WiredTiger already completed...
migrator_1 | Found /data/db/WiredTiger - assuming migration to WiredTiger already completed...
migrator_1 | Found /data/db/WiredTiger - assuming migration to WiredTiger already completed...
migrator_1 | Found /data/db/WiredTiger - assuming migration to WiredTiger already completed...
migrator_1 | Found /data/db/WiredTiger - assuming migration to WiredTiger already completed...
rocketchat_migrator_1 exited with code 0
The mongo service definition also behaves in the same manner and skips doing the migration based on the migrator when it finds that the WiredTiger storage engine is already in place.
Also, in my case, I upgraded the MongoDB database from version 4.0 onwards. But I did not have to run the feature compatibility command (step7c) for version 4.0 beforehand. Perhaps it was already enabled for future upgrades.
Summary
In this step-by-step tutorial, you learned how to change the deprecated database storage engine MMAPv1 to WiredTiger for MongoDB, upgrade the MongoDB database to the latest version and also the Rocket.Chat instance itself - all on Docker.
Hope this guide helps you in upgrading your RocketChat Docker instance. This will also be useful if you use MongoDB for some other application on Docker. If you have any feedback, queries, comments or suggestions, please share with us in the comment section below. Happy self-hosting :) !
DevOps Geek at Linux Handbook | Doctoral Researcher on GPU-based Bioinformatics & author of 'Hands-On GPU Computing with Python' | Strong Believer in the role of Linux & Decentralization in Science