Using LUKS2 for Encrypted Application Storage on Linux
Recently I am asked to add one more layer of protection of disk encryption to our data installed on a cloud development instance.
I decided to use LUKS2, the goals include:
Mount the decrypted filesystem under
/var/backendMove MongoDB data and middleware onto the encrypted volume
Measure performance impact
Understand how to expand the encrypted disk later
This article documents the complete process and some practical considerations I encountered along the way.
1. Disk layout
The additional data disk attached to the cloud instance is /dev/vdb:
First, create a GPT partition table and a partition covering almost the entire disk:
sudo parted /dev/vdb --script mklabel gpt mkpart primary 1MiB 100%
Resulting in partition of /dev/vdb1
The storage stack will eventually look like this:
/dev/vdb
└── /dev/vdb1
└── LUKS2
└── /dev/mapper/reframe_crypto
└── XFS/ext4
└── /var/backend
└── mongo/
2. Generate a temporary encryption key
I generated a random 256-bit value with OpenSSL:
KEYHEX=$(openssl rand -hex 32) \
&& printf "%s" "$KEYHEX" | sudo tee /run/luks.key >/dev/null
Why /run?
/run is normally backed by tmpfs, so its contents live in volatile memory and disappear after reboot. This is preferable to casually leaving an encryption key somewhere on the persistent root filesystem.
3. Create the LUKS2 container
Install cryptsetup:
sudo dnf install cryptsetup
Then initialize the partition as LUKS2:
sudo cryptsetup luksFormat \
--type luks2 \
--pbkdf argon2id \
/dev/vdb1 \
--key-file /run/luks.key
I explicitly selected:
LUKS2
PBKDF: Argon2id
LUKS itself manages the encrypted container and its key slots. Argon2id is the password-based key derivation function used here to make brute-force attacks against passphrases or key material more expensive.
4. Back up the LUKS header
Before going further, I backed up the LUKS header:
sudo cryptsetup luksHeaderBackup \
/dev/vdb1 \
--header-backup-file /run/reframe.luks.hdr
The LUKS header contains critical metadata required to unlock the encrypted volume. Damage to the header can make the data inaccessible.
Keep the backup to a a secure location separate from the encrypted disk.
5. Open the encrypted volume
Open the LUKS container and create a device-mapper mapping called reframe_crypto:
sudo cryptsetup open \
/dev/vdb1 \
reframe_crypto \
--key-file /run/luks.key
After the volume is successfully opened, the temporary key can be removed:
sudo rm -f /run/luks.key
The decrypted block device is now available as:
/dev/mapper/reframe_crypto
We can verify it with:
sudo cryptsetup status reframe_crypto
For example:
/dev/mapper/reframe_crypto is active and is in use.
type: LUKS2
cipher: aes-xts-plain64
keysize: 512 [bits]
device: /dev/vdb1
mode: read/write
A potentially confusing detail is:
cipher: aes-xts-plain64
keysize: 512 bits
This is not AES-512 because AES-XTS uses two AES keys. A 512-bit XTS key corresponds to two 256-bit AES keys.
6. Create the filesystem
Initially I formatted the decrypted device as ext4:
sudo mkfs.ext4 /dev/mapper/reframe_crypto
Then create the mount point:
sudo mkdir -p /var/backend
and mount it:
sudo mount /dev/mapper/reframe_crypto /var/backend
Later I decided to reformat it as XFS, which recommended by Mongo. Do notice this is reformat, Any existing data will be cleared.
First unmount it:
sudo umount /var/backend
Then create the XFS filesystem:
sudo mkfs.xfs -f /dev/mapper/reframe_crypto
and mount it again:
sudo mount /dev/mapper/reframe_crypto /var/backend
7. Quick sequential I/O test with dd
Before moving the application data, I did some simple disk tests.
For the root filesystem:
dd if=/dev/zero of=/tmp/testfile \
bs=1G count=1 oflag=direct
For the encrypted filesystem:
dd if=/dev/zero of=/var/backend/testfile \
bs=1G count=1 oflag=direct
Read tests:
dd if=/tmp/testfile of=/dev/null \
bs=1G iflag=direct
and:
dd if=/var/backend/testfile of=/dev/null \
bs=1G iflag=direct
Using direct I/O helps reduce the influence of the Linux page cache, although dd should still be considered a quick sanity check rather than a comprehensive storage benchmark.
8. Random I/O benchmark with fio
For a more realistic random I/O comparison, I installed fio:
sudo dnf install fio
I tested the normal filesystem first:
fio \
--name=randrw \
--ioengine=libaio \
--rw=randrw \
--bs=4k \
--size=1G \
--numjobs=4 \
--runtime=60 \
--group_reporting \
--directory=/tmp
Result:
READ: bw=4873KiB/s (4990kB/s)
WRITE: bw=4897KiB/s (5015kB/s)
Then I ran the same workload against the encrypted filesystem:
fio \
--name=randrw \
--ioengine=libaio \
--rw=randrw \
--bs=4k \
--size=1G \
--numjobs=4 \
--runtime=60 \
--group_reporting \
--directory=/var/backend
Result:
READ: bw=4359KiB/s (4463kB/s)
WRITE: bw=4385KiB/s (4490kB/s)
In this particular test, the encrypted filesystem delivered roughly 10% lower bandwidth.
I also checked whether the CPU include AES acceleration ability:
grep aes /proc/cpuinfo
Modern CPUs mostly com with hardware AES acceleration.
9. Move MongoDB onto the encrypted volume
Next I created a dedicated MongoDB directory:
sudo mkdir -p /var/backend/mongo
sudo chown mongod:mongod /var/backend/mongo
Then I changed MongoDB's storage path in:
/etc/mongod.conf
to
storage:
dbPath: /var/backend/mongo
After migrating the existing MongoDB data as required, MongoDB can be restarted:
sudo systemctl restart mongod && sleep 3 && sudo systemctl status mongod
10. Prevent services from starting before the disk is unlocked
This was an important operational consideration.
After a reboot, the LUKS volume is locked:
/dev/vdb1
↓
[LOCKED]
Therefore:
/var/backend
is not yet backed by the encrypted filesystem.
If MongoDB starts automatically before the LUKS volume is opened and mounted, it may see the ordinary /var/backend directory from the root filesystem instead.
That can lead to a particularly confusing situation:
/var/backend
appears to contain one set of files before the mount and another set after the encrypted filesystem is mounted.
Nothing has magically reappeared or disappeared because a Linux mount simply hides the contents of the original mount-point directory while another filesystem is mounted over it.
For this reason I disabled automatic MongoDB and PM2 auto startup in this setup:
sudo systemctl disable mongod
systemctl disable pm2-root.service
The intended boot sequence became:
Boot
↓
Obtain LUKS key
↓
cryptsetup open
↓
mount /var/backend
↓
start MongoDB
↓
start application processes
For example:
sudo cryptsetup open \
/dev/vdb1 \
reframe_crypto \
--key-file /run/luks.key
sudo rm -f /run/luks.key
sudo mount /dev/mapper/reframe_crypto /var/backend
sudo systemctl start mongod
pm2 resurrect
For a fully automated production system, I would prefer expressing these dependencies through crypttab, fstab, and systemd dependencies rather than relying on manual command ordering.
The key point is that the application must never start under the assumption that /var/backend is mounted when it is not.
another way is add "RequiresMountsFor" to mongo systemd unit file
[Unit]
RequiresMountsFor=/var/backend
11. Expanding the encrypted disk
Disk expansion with LUKS involves multiple layers.
Conceptually:
Physical/virtual disk
↓
Partition
↓
LUKS mapping
↓
Filesystem
Each relevant layer needs to know about the additional space.
First expand partition 1:
sudo growpart /dev/vdb 1
Then resize the active LUKS mapping:
sudo cryptsetup resize reframe_crypto
Finally, expand the filesystem.
For ext4:
sudo resize2fs /dev/mapper/reframe_crypto
For XFS, the command is different.
Because XFS grows while mounted, use:
sudo xfs_growfs /var/backend
While resize2fs which applies to ext filesystems, after reformatting the volume as XFS, xfs_growfs is the appropriate filesystem-level expansion command.
So for my final XFS setup the expansion sequence is:
sudo growpart /dev/vdb 1
sudo cryptsetup resize reframe_crypto
sudo xfs_growfs /var/backend
12. What I learned
For me, this is a good hands-on experience of implementing LUKS in a real application environment, integrating the encrypted storage cleanly into the application's boot sequence, data layout, backup strategy, and operational procedures. For real production environment, there will be more factors like how and where to keep the LUKS decryption key to think about, in the end, LUKS is more for keeping data at rest against offline access (like being cloned in EBS snapshot service) not online usage after unlocked.