Tag Archives: security

Keeping sensitive data on the crypto disks

2008-02-18Uncategorizedcrypto securityChristopher

Previously, I described how to create one or more crytpographic partitions. The data stored on those partitions is not retrievable without the 32-digit hexadecimal key that protects it, the key being constructed from a passphrase input by the user. It may seem that this is sufficient to protect sensitive data, making sure simply to create and edit your files only in that partition. However, there are some subtle details that have to be kept in mind.

Information stored on an unencrypted ext2 or ext3 partition has an unknown persistence. A file that was stored there, and later deleted, may be partially or fully recoverable at some time in the future. To be sure of the confidentiality of your data, you have to make sure that it has never been stored to an unencrypted partition.

If you start up your favourite text editor, telling it to create a new file in some place, let’s call it /crypto/sensitive.txt, and then start typing, you may expect that the data never lands on an unencrypted partition. However, there are at least four things to be careful of:

The editor may store information in your home directory, which may not be on the encrypted partition. It might store some of the file contents there, or it might store file metadata. Your editor may keep a table of filenames recently visited in /home, with information about the line number last visited. Your editor might be configured to store crash-recovery autosave files in a directory under your /home directory.
The editor may sometimes store the contents of a working buffer to a file in /tmp.
The computer may come under memory pressure, resulting in some of your data being sent to the swap device.
Your backups may not be as well protected as the files on the cryptographic disk.

The first two points are probably best addressed by ensuring that all of the directories writable by the unprivileged user are on cryptographic partitions. If you only have write permission to the crypto drives, you won’t store any files in plaintext. Note, however, that you typically need /tmp to exist and be writable during the bootup of your system, so that partition can’t be protected with a passphrase if you care about the system successfully performing an unattended reboot.

So, what do we do about /tmp? Well, one simple solution is an overmount. While you normally mount a partition onto an empty directory, it is legal to mount onto a directory that is not empty. The files that were present in that directory are mostly inaccessible after that (a process with access to file descriptors that it opened before the mount will still be able to operate on those files, but they will be invisible to new open operations by pathname).

We’re assuming you have at least one cryptographic partition. So, create a directory on that partition, let’s say /crypto/tmp. After you have formatted and mounted your cryptographic partition, run this command. You only have to do this once, the first time you set up cryptographic disks.

mkdir --mode=01777 /crypto/tmp

Now, you can add the following command to the end of the script in the previous post, the script that mounts your formatted disks:

mount --bind /crypto/tmp /tmp

After you’ve done this, the system will still boot up as usual, using its unencrypted /tmp partition. Then, the root user can run the script from the previous post, now modified to have this extra mount line on the end of it. After entering the passphrase the script will do its work and exit, at which time your /tmp partition will have been replaced with the one in /crypto. Note that if your system starts up in X, with a graphical login screen, you will have to restart it after you have overmounted /tmp, otherwise you will find that X programs fail to work at all. I usually restart X by issuing a simple “killall X” command, and letting the xdm or gdm program start it back up again. This is a lot of trouble, but all manner of things can be stored on your /tmp disk. Firefox will store downloaded files such as PDFs there when there is a helper application ready to use them.

That leaves us with swap. Encrypting the swap space is actually very easy:

# Encrypt the swap partition
hashed=`dd if=/dev/urandom bs=1 count=64 | md5sum | awk ' { print $1 } '`
dmsetup create SWP <<DONE
0 `blockdev --getsize /dev/hda6` crypt aes-plain $hashed 0 /dev/hda6 0
DONE
mkswap /dev/mapper/SWP
swapon /dev/mapper/SWP

This can run unattended during the bootup. It creates a random cryptographic key using /dev/urandom, a device especially designed to produce true random numbers even during a system bootup sequence. This random key is used to create an encrypted interface to /dev/hda6. It is formatted as a swap partition, and then enabled. A new key will be generated each time the system boots, so nothing in swap space will survive a reboot. Note that there do exist suspend-to-disk procedures for Linux that store a memory image on the swap partition. If you intend to use such a suspend system, you will have to ensure that it does not attempt to write to the cryptographic swap partition, or you’ll have to defer mounting the swap partition until the root user can enter a specific passphrase, thereby allowing you to preserve the contents across a reboot. If you’re supplying a passphrase to handle encryption on the swap space, you should not run mkswap, except the first time you set up the partition (think of mkswap as being a reformat).

The question of how to protect your backup copies of sensitive files is entirely dependent on what system you use for backups. You may be able to pipe your backups through the des binary, or you may be able to store the backups on encrypted filesystems, but there are too many variations for me to offer much advice here. The security of your backups is not something that can be ignored, as has been made all to obvious with the various data disclosure scares that occur with alarming regularity when shipments of tapes or CDs fail to arrive at their destinations.

UPDATE

See my followup article for a warning about a vulnerability in this technique.

Cryptographic mounts

2008-02-17Uncategorizedcrypto securityChristopher

Some of the data on my computers is stuff that I’d rather not let into the hands of a random stranger. Work-related files, proprietary data or source code, banking information, or other sensitive files. A laptop can go missing, an entire desktop computer can be carried away. It would be nice if the sensitive data were inaccessible in that event.

This leads us to cryptographic mounts. Partitions whose contents cannot be read without the knowledge of a secret that is not stored in the computer. I use a passphrase, but if you are the kind of person who memorizes 32 digit hexadecimal numbers, you can skip the passphrase. The appropriate features to enable in the kernel, either as modules or compiled directly in, are MD (the same subsystem that controls RAID) and two features in that subsystem, BLK_DEV_MD, and DM_CRYPT. You also need a cryptographic algorithm available. I use AES encryption on my partitions, but there are many others available. I have activated the CRYPTO_AES module, plus the appropriate architecture specific module, CRYPTO_AES_X86_64 for my desktop machine and CRYPTO_AES_586 for my laptop.

So, let’s say you have one or more blank partitions that you’d like to set up as a cryptographic partitions, all with the same passphrase. You start with this script:

#! /bin/sh
#

partition=/dev/sda6
partition2=/dev/sdc6
mtpt=/crypto
mapname1=Crypto1
mapname2=Crypto2

holdIFS="$IFS"
IFS=""

echo -n "Enter the passphrase: "
read -s oneline

IFS="$holdIFS"

{ hashed=`md5sum | awk ' { print $1 } '` ; }<<DONE
$oneline
DONE

dmsetup create $mapname1 <0 `blockdev --getsize $partition` crypt aes-plain $hashed 0 $partition 0
DONE
dmsetup create $mapname2 <0 `blockdev --getsize $partition2` crypt aes-plain $hashed 0 $partition2 0
DONE

What this script does is to prompt the user for a passphrase, without echoing it to the screen. Once the passphrase is entered, it is converted to a 32 character hexadecimal string with the MD5 program. I use a here document, marked with the << characters, because that way the hexadecimal string does not appear in the process status list. Simply using echo risks having the secret visible to any user who types ps at the correct moment. Then, the dmsetup program creates the cryptographic mapping, using the hex sequence as the cryptographic key.

You will have to change the values of the $partition and $partition2 variables to correspond to those on your system. Note that volume labels are unavailable, because the system can’t read the label off a cryptographic partition before the passphrase has been supplied.

Run this script, entering the passphrase. It’s important that you do this through the script, and not manually at the command line, because later you’ll modify the script to mount your cryptographic partitions, and you want to ensure that exactly the same code read your passphrase when you created the partitions as will read your passphrase when you try to mount the partitions after a reboot some time in the future.

When the script exits, you will have two new objects appearing in the /dev/mapper directory. In this case, they are /dev/mapper/Crypto1 and /dev/mapper/Crypto2. So, in this example, /dev/sda6 is the encrypted volume, and /dev/mapper/Crypto1 is the decrypted version of the same volume. You do all of your work on /dev/mapper/Crypto1. You format and mount that device, never /dev/sda6.

This command will create an ext3 filesystem with 0 bytes reserved for the superuser.

/sbin/mke2fs -j -m 0 /dev/mapper/Crypto1

Now, you can mount /dev/mapper/Crypto1 onto a mount point, and start copying files there as usual. Until you remove the cryptographic mapping, the data is available as a normal mounted partition. So, we now append some code to the script above to allow the partitions to be mounted by the root user after a reboot. Take the script above and add the following lines to the bottom:

/sbin/e2fsck -y /dev/mapper/$mapname1 || \
        { dmsetup remove $mapname1 ; echo "" ; echo "fsck failed"; exit 1; }

/sbin/e2fsck -y /dev/mapper/$mapname2 || \
        { dmsetup remove $mapname1; dmsetup remove $mapname2 ;\
             echo "" ; echo "fsck failed"; exit 1; }

mount -onodiratime /dev/mapper/$mapname1 $mtpt1 || \
        { dmsetup remove $mapname1 ; dmsetup remove $mapname2 ; \
            echo "" ; echo "Failed" ; exit 1 ; }

mount -onodiratime /dev/mapper/$mapname1 $mtpt2 || \
        { umount $mtpt ; \
            dmsetup remove $mapname1 ; dmsetup remove $mapname2 ; \
            echo "" ; echo "Failed" ; exit 1 ; }
echo ""

This runs fsck on the partitions, if necessary (remember, fstab can’t fsck these partitions because it doesn’t know the passphrase). Note that if you entered the wrong passphrase, you’ll find out at this point, when e2fsck fails to identify the partition as being an ext2 or ext3 partition.

It then manually mounts the cryptographic partitions onto the mountpoints in $mtpt1 and $mtpt2. In the event of a mount failure, it unmounts everything and removes the cryptographic mappings.

The next time the computer is rebooted, the root user will have to run this script and enter the correct passphrase before the data on those drives is readable. If somebody else obtains your laptop, any mounted cryptographic partitions will be unavailable if the computer is rebooted, or the drive removed from the laptop and inserted into another machine.

This is only half the story. In a later post I’ll describe the care you have to take to make sure your sensitive data does not wind up as readable plaintext somewhere on your filesystem.

What do we have running on that box?

2008-02-17Uncategorizedaudit login failure one size fits all securityChristopher

As I mentioned in my first post, when you install a distribution you sometimes have programs running without your knowledge. Because some users may need these features, you often get them as well.

Last week at the office, somebody came up to me to ask if I could figure out why it was no longer possible to log into one of the servers. This server has a history of flakiness, there’s probably a bad memory module on the board, and sometimes it becomes unresponsive. So, my co-worker, upon realizing that he couldn’t log in, had rebooted the computer. However, even after the reboot, he still couldn’t log on, either as his regular user through SSH, or as root on the console.

The first step, before getting out of my chair, was to telnet to port 22 on the box. I got a “connected” message, and a text string indicating that I was attached to an SSH daemon. This told me that the kernel was alive, it was accepting new connections and passing them to the appropriate processes, which were themselves able to make forward progress. So, the box wasn’t wedged. I went to the console, and tried to log in through the getty running on the text login screen. I entered ‘root’ at the username, and got a password prompt. When I entered the password and pressed ENTER, the getty process froze, and did not present me with a shell.

So, we have two very different authentication schemes that are failing to allow logins. The console doesn’t allow root logins. Something seemed to be interfering with the general activity of authentication. The first thought is that this might be a PAM problem, but it would be a strange one. We didn’t get authentication failure messages, we got a hang after authentication. Root’s credentials were stored on the local drive, so it wasn’t an LDAP issue, and in any case, the machine was on the network, and there weren’t LDAP problems anywhere else in the office.

When multiple independent programs fail together, the next thought is that there’s probably a full disk somewhere. If you fill up /tmp your system can start to behave very strangely. The login problems were a symptom of something, as yet unknown. So, the next thing to do is to check the hard drive to see if we had any full partitions. Because I didn’t know what else might be misbehaving, I wanted to avoid all of the startup jobs, so I rebooted the machine with an appended kernel parameter, “init=/bin/bash“. Instead of running the usual /sbin/init, and all of the various scripts under /etc/init.d, the computer would start up the kernel and then drop immediately to a root shell. No logins, no passwords, no startup scripts. I could then run ‘df‘ at the prompt, and confirm that there were no partitions within 5% of full (remember that a default ext2 format will reserve 5% of the blocks for root, so a disk that’s 96% full could actually be entirely full for some users). Checking with ‘df -i‘ showed that we had not run out of inodes either.

So, what’s next? I decided to reboot the machine into single-user mode so that I could easily modify files on the disk but still get onto the computer without a password. This is done by appending the parameter “S” on the kernel boot line. Again, I get a shell, but this time the disks are read-write mounted, and various services have started up. So, I modify the inittab. I replaced the getty on tty1 with /bin/bash. That means that when the computer is rebooted into multi-user mode, tty1 has a root shell while the other ttys are still running their gettys.

Reboot into the usual multi-user mode. I have a root shell on tty1. I run “ps ax“, and find the PID of the getty on tty2. Then, I run the command

strace -f -p

at the shell prompt of tty1. Changing virtual consoles to tty2 with the usual command, CTRL-ALT-F2, I am presented with a login prompt. I enter the username ‘root’, and enter the password. The program hangs. So, I change back to tty1 to see what strace has to say about the program. The last things the program did are on the screen. It opened a device called /dev/audit, did some things with it, then issued an ioctl() on the file descriptor. That ioctl was not returning to the caller, so the program was blocking waiting for a response from something associated with /dev/audit.

None of us had heard of /dev/audit, so it was time to do a bit of research. It turned out to be a package that was included in the RHEL distribution installed on that computer. There is communication between the device and a daemon. That daemon keeps logs, so I went to its logging directory to see what was there. I found 4 GB of data there. Apparently that had reached some sort of internal limit, and the daemon responded by forbidding further auditable actions until some of the logs were removed by the administrator. Logins, being auditable actions, were blocked.

So, delete all of the logs in the directory, and reboot the computer. Everything returned to normal.

Now, a logging function like this is very useful for some users. There are some people who must know exactly who logged into the machine, what database entries they accessed or modified, and so on. We are not such people. A service we never knew about, enabled for all because it is useful by some, wound up locking us out of our own machine.

It’s a security feature that logins are forbidden until the logs have been inspected and removed. If you’re going to design a function like this, then this is the correct way to go about it. Of course, it was very easy for me to overcome this security feature with access to the console, but that’s generally true. I probably would have set it up so that gettys are permitted to log in as root even when an audit failure occurs, but that level of flexibility may not be available, if the behaviour is driven by a special PAM module or a patched glibc.

There have been visitors to this page

Wordpress on cneufeld.ca

Moved here from Taiwan Yahoo blogs

Tag Archives: security

Keeping sensitive data on the crypto disks

Cryptographic mounts