Table of Contents

Description

I was using a custom solution running MongoDB in the Backend on Ubuntu 18.04 and recently decided to try out an

apt-get update
apt-get dist-upgrade

I got quickly reminded why doing such operations require a bit more planning ahead instead of the just do it way of thinking.

systemctl status mongod
● mongod.service loaded failed failed MongoDB Database Server

cat /var/log/mongodb/mongod.log

“ctx”:“initandlisten”,“msg”:“Storage engine to use detected by data files”,“attr”:{“dbpath”:"/var/lib/mongodb", Functions"storageEngine":“mmapv1”}}

Cannot start server with an unknown storage engine: mmapv1"}}

Luckily in IT, there’s always a solution for everything and a chance to reverse engineer what happened…

What happened ?

I was initially running Mongo 3.x and had the following configuration in /etc/mongod.conf

# Where and how to store data.
storage:
  dbPath: /var/lib/mongodb
  journal:
    enabled: true
#  engine:
  mmapv1:
    smallFiles: true

After opening the Release Notes of Mongo 4.2 (should have done this before upgrading) one thing jumped into my eyes:

Not so cool…

First attempt at fixing it…FAIL

I thought to myself…what if I just remove the mmapv1 definition from /etc/mongod.conf ?

# Where and how to store data.
storage:
  dbPath: /var/lib/mongodb
  journal:
    enabled: true

“Storage engine to use detected by data files”,“attr”:{“dbpath”:"/var/lib/mongodb",“storageEngine”:“mmapv1”}

Mongo detects the storage engine to use from its data files.
There’s no “magic automatic” conversion happening in the background.

Back to the drawing board…

Convert the Database Files to wiredTiger

By now I have already upgraded MongoDB to the v4 release and it seems that in order to convert the database one needs to:

  • do a mongodump
  • start with a fresh db
  • then change the storage engine
  • import the dump back

Sadly I already cannot start my database anymore as I upgraded…

What do to ?

Build the older version of Mongo

Clone the repository of Mongo:

cd /tmp
git clone https://github.com/mongodb/mongo.git
cd mongo
#get the v3 branch of it
git checkout v3.6

Build MongoDB but DO NOT INSTALL

apt install build-essential
apt install install libboost-filesystem-dev libboost-program-options-dev libboost-system-dev libboost-thread-dev
python2 -m pip install -r buildscripts/requirements.txt
pip2 install Cheeta
python2 buildscripts/scons.py mongod

Run the older Mongo

cd /tmp/mongo
./mongod -f /etc/mongod.conf

Dump the DB

mongodump
...
2023-05-03T14:05:03.216+0200    done dumping blabla.App1 (1 document)
2023-05-03T14:05:03.217+0200    done dumping something.Detached_Users (0 documents)
...

This by default creates a dump directory in /tmp where I was already with my shell.

Convert to wiredTiger

I used the official guide from here but cut down on some steps.

First I stopped the Mongo v3 that was running in the foreground from before (CTR+C).

Second I start the official/already installed v4 Mongo.

Remember that I had already removed the storage type mmapv1 definition from /etc/mongod.conf.

I had the following data directory for Mongo:

dbPath: /var/lib/mongodb

The contents of it need to go away as all the files inside are using mmapv1 storage type.

# better 2 x safe than sorry, even if we have a dump of the DB
cp -av /var/lib/mongodb /var/lib/mongodbbck
rm -f /var/lib/mongodb/*
rm -f /var/lib/mongodb/*/*

Start MongoDB

systemctl start mongod
systemctl status mongod

● mongod.service - MongoDB Database Server
   Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2023-05-03 14:17:23 CEST; 4s ago

All good so far :)

Import the dump I took previously

mongorestore /tmp/dump

2023-05-03T14:22:32.956+0200    456 document(s) restored successfully. 5 document(s) failed to restore.

Celebrate :)