Inplace Encryption and Decryption

Posted

I just moved back home from Dublin to Toronto and had a bunch of data I was moving with me. Most of my devices are encrypted already but my home server was not. For a variety of reasons I didn’t want to encrypt it at this time, however for mostly philosophical reasons I would prefer to have all of my data encrypted while traveling in case it was lost or stolen. So I wanted to find a solution that fit the following constraints.

  1. Didn’t require much storage overhead. My server has a 1T hard disk and the only other storage I had available was my laptop’s SSD at 500G. In practice I had about 200G free space which is less than the size of the data on the server.
  2. Losing data is better than revealing data. For my specific circumstances the data on the server was mostly replaceable. I would need to regenerate some keys but everything was backed up.

Overview

The basic concept is quite simple. We will be using the Linux kernel’s dm-crypt framework to do the encryption and decryption for us. The only trick that we perform is reading from one layer while writing to the other. This is clearly a hack but it works well.

One key component of the process is using the plain mode of dm-crypt which doesn’t write any metadata to the disk. This means that we can encrypt in place with no space overhead.

Process

Preparation

We will be using the following tools to get started.

Practice

I highly recommend that you perform some practice runs before running this on your disk. You can create an empty file as your practice block device.

You will need root on a linux machine in order to use the Linux device mapper functionality. You can either do this on a random machine you own, or in the USB live environment that you will be using for the actual process. The commands in this guide will work as a regular user with sudo where root is required.

To set up an example filesystem on a block device run the following commands.

blockdevice=~/test.dat
fallocate -l $((1024*1024*1024*2)) $blockdevice
mkfs.ext4 $blockdevice
mkdir -p inplace-mount
sudo mount $blockdevice inplace-mount
echo HelloGrepper | sudo dd of=inplace-mount/hello
sudo umount inplace-mount

Then continue on with the instructions. Once you have done it a couple of times you can repeat the steps on the real block device as root. Just set blockdevice=/dev/sdX.

Check Hash

We expect to arrive back at the exact same state we started at. To detect errors as soon as possible and be confident that everything was successful in the end we will take a starting checksum. Make note of this value as we will compare to it throughout the process.

% pv $blockdevice | sha1sum
2.00GiB 0:00:02 [ 791MiB/s] [================================>] 100%
53812df694cc269596c9a69f849acdcc8c440099  -

Identify Markers

To double-check that the encryption hasn’t completely failed we will identify a number of strings in the input data and verify that they are no longer present on the block device once encrypted.

The following regex is useful for the practice disk as it checks the contents of a known file and the name of a known directory. For your real block device find an expression which finds at least a couple of matches.

pv $blockdevice | strings | grep 'lost+found\|HelloGrepper'

Encrypt

Open Mapper

The first step is to setup the mapper. This will be where you set your passphrase. Note that we are using the plain mode. This mode stores no metadata and has no way to verify we have entered the correct password. I don’t recomended it for general use but since we don’t want any space overhead we can’t store any metadata on this disk, making plain our only option.

sudo cryptsetup open $blockdevice inplace --type=plain --verify-passphrase

Copy and Encrypt Data

This step will copy the data from the raw block device onto the mapper device. This will encrypt the data.

pv $blockdevice | sudo dd of=/dev/mapper/inplace

Close Mapper

Your data is now encrypted and the disk has been overwritten. All that is left is cleaning up.

sudo cryptsetup close /dev/mapper/inplace

Verify Device is Encrypted

Now is time to verify that we can’t see the plain-text data anymore. Use the strings you have identified in the Identify Markers stage.

pv $blockdevice | strings | grep 'lost+found\|HelloGrepper'

You should see no matches and grep should exit with status 1.

Decrypt

Once your data has been safely transported it is time to reverse the operation. This is basically the reverse process of the encrption.

Open Mapper

Open the mapper device.

sudo cryptsetup open $blockdevice inplace --type=plain

Verify Decryption

Before we make any writes to the block device we want to ensure that the decrypted data is identical to the data we started with. DO NOT SKIP THIS STEP otherwise you may corrupt your data.

% sudo pv /dev/mapper/inplace | sha1sum
2.00GiB 0:00:02 [ 791MiB/s] [================================>] 100%
53812df694cc269596c9a69f849acdcc8c440099  -

Check that the hash matches the starting value. If it does not, STOP NOW. Consider the following then rerun this step until you get a match:

If none of these solve your problem then you have likely lost data. Do not proceed further or you will only make the data harder to recover. If you think you have managed to resolve the problem run the check again to verify. Only continue with later steps once the checksum matches.

Copy and Decrypt Data

sudo pv /dev/mapper/inplace 1<>$blockdevice

Close Mapper

All that is left is some cleanup.

sudo cryptsetup close /dev/mapper/inplace

Final Verification

This is an optional step to check that everything has been copied correctly from the mapper back to the raw block device.

% pv $blockdevice | sha1sum
2.00GiB 0:00:02 [ 791MiB/s] [================================>] 100%
53812df694cc269596c9a69f849acdcc8c440099  -

Conclusion

Your disk is now in exactly the same state you started in. Next time be sure to use full disk encryption from the start.