Inplace Encryption and Decryption

Posted

I just moved back home from Dublin to Toronto and had a bunch of data I was moving with me. Most of my devices are encrypted already but my home server was not. For a variety of reasons I didn't want to encrypt it at this time, however for mostly philosophical reasons I would prefer to have all of my data encrypted while traveling in case it was lost or stolen. So I wanted to find a solution that fit the following constraints.

  1. Didn't require much storage overhead. My server has a 1T hard disk and the only other storage I had available was my laptop's SSD at 500G. In practice I had about 200G free space which is less than the size of the data on the server.
  2. Losing data is better than revealing data. For my specific circumstances the data on the server was mostly replaceable. I would need to regenerate some keys but everything was backed up.

Do not attempt these steps if you can't afford to lose your data. In-place encryption and decryption is fundamentally dangerous. Even if you follow these instructions perfectly there is a high risk of data loss should you lose power or have any hardware failures. Any failure during the encryption or decryption process will result in difficult to recover data. I CAN NOT BE HELD RESPONSIBLE FOR ANY ISSUES ARISING FROM THESE INSTRUCTIONS.

Overview

The basic concept is quite simple. We will be using the Linux kernel's dm-crypt framework to do the encryption and decryption for us. The only trick that we perform is reading from one layer while writing to the other. This is clearly a hack but it works well.

One key component of the process is using the plain mode of dm-crypt which doesn't write any metadata to the disk. This means that we can encrypt in place with no space overhead.

Process

Preparation

We will be using the following tools to get started.

Practice

I highly recommend that you perform some practice runs before running this on your disk. You can create an empty file as your practice block device.

You will need root on a linux machine in order to use the Linux device mapper functionality. You can either do this on a random machine you own, or in the USB live environment that you will be using for the actual process. The commands in this guide will work as a regular user with sudo where root is required.

To set up an example filesystem on a block device run the following commands.

head -c $((1024*1024*1024*2)) </dev/zero >test.dat
blockdevice=~/test.dat
mkfs.ext4 $blockdevice
mkdir -p inplace-mount
sudo mount $blockdevice inplace-mount
echo HelloGrepper | sudo dd of=inplace-mount/hello
sudo umount inplace-mount

Then continue on with the instructions. Once you have done it a couple of times you can repeat the steps on the real block device as root. Just set blockdevice=/dev/sdX.

Check Hash

We expect to arrive back at the exact same state we started at. To detect errors as soon as possible and be confident that everything was successful in the end we will take a starting checksum. Make note of this value as we will compare to it throughout the process.

% pv $blockdevice | sha1sum
2.00GiB 0:00:02 [ 791MiB/s] [================================>] 100%
53812df694cc269596c9a69f849acdcc8c440099  -

Identify Markers

To double-check that the encryption hasn't completely failed we will identify a number of strings in the input data and verify that they are no longer present on the block device once encrypted.

The following regex is useful for the practice disk as it checks the contents of a known file and the name of a known directory. For your real block device find an expression which finds at least a couple of matches.

pv $blockdevice | strings | grep 'lost+found\|HelloGrepper'

Encrypt

Open Mapper

The first step is to setup the mapper. This will be where you set your passphrase. Note that we are using the plain mode. This mode stores no metadata and has no way to verify we have entered the correct password. I don't recomended it for general use but since we don't want any space overhead we can't store any metadata on this disk, making plain our only option.

--type=plain assumes the default encryption parameters. Make sure you are using the exact same versions of all software for the encryption and decryption part of this guide as any change to the defaults will make the data unreadable. I recomend noting exactly what live image you used and ensuring that you can acquire more copies if needed.

sudo cryptsetup open $blockdevice inplace --type=plain --verify-passphrase

This command will likely warn you that there is an existing filesystem on the block device. This is intentional and we are accepting the risk so you can type YES.

Remember your password. Without it your data will be unrecoverable.

Copy and Encrypt Data

This step will copy the data from the raw block device onto the mapper device. This will encrypt the data.

If this step is interrupted your device will be partially encrypted and incredibly difficult to recover.

pv $blockdevice | sudo dd of=/dev/mapper/inplace

Close Mapper

Your data is now encrypted and the disk has been overwritten. All that is left is cleaning up.

sudo cryptsetup close /dev/mapper/inplace

Verify Device is Encrypted

Now is time to verify that we can't see the plain-text data anymore. Use the strings you have identified in the Identify Markers stage.

pv $blockdevice | strings | grep 'lost+found\|HelloGrepper'

You should see no matches and grep should exit with status 1.

Decrypt

Once your data has been safely transported it is time to reverse the operation. This is basically the reverse process of the encrption.

Open Mapper

Open the mapper device.

cryptsetup has no way to check if your password is correct so it will accept anything. Ensure that you do the following verification step before modifying the data otherwise an invaid or typoed pasword will lead to data corruption.

sudo cryptsetup open $blockdevice inplace --type=plain

Verify Decryption

Before we make any writes to the block device we want to ensure that the decrypted data is identical to the data we started with. DO NOT SKIP THIS STEP otherwise you may corrupt your data.

% sudo pv /dev/mapper/inplace | sha1sum
2.00GiB 0:00:02 [ 791MiB/s] [================================>] 100%
53812df694cc269596c9a69f849acdcc8c440099  -

Check that the hash matches the starting value. If it does not, STOP NOW. Consider the following then rerun this step until you get a match:

If none of these solve your problem then you have likely lost data. Do not proceed further or you will only make the data harder to recover. If you think you have managed to resolve the problem run the check again to verify. Only continue with later steps once the checksum matches.

Copy and Decrypt Data

If this step is interrupted your device will be only partially decrypted and incredibly difficult to recover.

sudo pv /dev/mapper/inplace 1<>$blockdevice

Close Mapper

All that is left is some cleanup.

sudo cryptsetup close /dev/mapper/inplace

Final Verification

This is an optional step to check that everything has been copied correctly from the mapper back to the raw block device.

$ pv $blockdevice | sha1sum
2.00GiB 0:00:02 [ 791MiB/s] [================================>] 100%
53812df694cc269596c9a69f849acdcc8c440099  -

Conclusion

Your disk is now in exactly the same state you started in. Next time be sure to use full disk encryption from the start.