wiki:Linux/Ubuntu/InstallToMultipleLVMsArbitraryEncryption
Last modified 9 years ago

  1. Install to Multiple LVMs with Arbitrary Encryption
    1. Layout
    2. Preparation
      1. Data Back-Up
    3. Volumes
      1. Physical Partitioning
      2. Logical Volume Management
        1. Create LVM Volumes
      3. Encryption
        1. Open Encrypted Volumes
      4. File-System Creation
    4. Installation
    5. Post-Installation Requirements
      1. Mount Target System
      2. Switch to Target
      3. Install Additional Packages
      4. Configure Custom Crypto Boot Process
      5. Modify Initial RAM-Disk Configuration
        1. Jaunty +
        2. Pre-Jaunty
      6. Update Initial RAM-Disk Image
      7. Leave the Target

Install to Multiple LVMs with Arbitrary Encryption

Note: This document is currently a work-in-progress

This is a development of my previous articles RAID-5 Encrypted with Logical Volume Management and Encrypted Logical Volume Management.

The aims are:

  • More disk capacity

In this article I'm reporting the steps taken when replacing the original hard-disk with one having double the capacity (200GB to 400GB) since it was beginning to get very cramped. At the same time I can re-organise the logical disk layout based on experience so it will be more flexible.

  • Encryption inside logical volumes

Where this scenario differs from the previous experiences is the use of logical volumes with the encrypted file-systems inside rather than encrypted raw disk partitions with all the logical volumes inside. I decided on this approach after realising that encryption was slowing down disk access in some areas of the disk - mostly in the /home/all/SourceCode/ directory where I routinely clone git/SVN/CVS/mercurial repositories and unpack and patch Ubuntu source packages. The key issue is the very large number of small files and the amount of time used in scanning/searching/indexing them by various tools I use. With this revised approach I can avoid using encryption for such logical volumes but still use it for other volumes (/, /var/, /home/, and so on).

  • Use Ext4 file-system

I have been impressed by the reports on the stability and performance of the ext4 file-system, especially when deleting large trees of files, and wanted to figure out how to install Intrepid to ext4 from the start.

  • More operating system versions

I want to take advantage of the additional capacity to allow me to operate three Ubuntu versions side-by-side (ubuntu-1, ubuntu, ubuntu+1 - e.g. Hardy, Intrepid, Jaunty) to make debugging of device driver issues easier.

Layout

The hard disk is configured with four partitions: 

sudo cfdisk -P s /dev/sda
Partition Table for /dev/sda

               First       Last
 # Type       Sector      Sector   Offset    Length   Filesystem Type (ID) Flag
-- ------- ----------- ----------- ------ ----------- -------------------- ----
 1 Primary           0    19551104*    63    19551105*Unknown (27)         None
 2 Primary    19551105*   77047739*     0    57496635 HPFS/NTFS (07)       Boot
 3 Primary    77047740*   77577884      0      530145 Linux (83)           None
 4 Primary    77577885   781417664      0   703839780 Linux LVM (8E)       None

Preparation

Data Back-Up

Because I'm replacing the hard-disk I don't need to do any data back-up. Data can be copied back from the previous disk-drive once the new system is operational.

Volumes

Physical Partitioning

In my case partitions 1 and 2 (will) contain the original Windows Vista installation. The original hard disk has the Windows Recovery installation in partition 1, which can recreate a clean Windows installation in partition 2.

Partition 3 is purely for the Linux kernel boot images and therefore doesn't need to be very large.

The rest of the disk is assigned to LVM (Logical Volume Management)

Logical Volume Management

When working from a live-CD LVM isn't installed so the first task is to install it: 

apt-get install lvm2

The kernel modules supporting disk-mapper and encryption need to be loaded. Prior to Jaunty both modules need loading but Jaunty includes the dm-mod module in the kernel image (part of the effort to reduce start-up times). Load the module(s): 

# not needed for Jaunty onwards
sudo modprobe dm-mod

Note: If the disk already contains LVM volumes they should be detected and 'loaded': 

sudo vgscan
sudo vgchange -ay

Create LVM Volumes

First assign partition 4 to LVM: 

sudo pvcreate /dev/sda4

Now create a volume group: 

sudo vgcreate Ubuntu /dev/sda4

Then create logical volumes that will contain the file-systems. I haven't allocated all the available space at this point since I shall be creating additional volumes for specific purposes (source-code archive, projects, media, etc.) and mounting them into the file-system once the system is installed: 

sudo lvcreate -L 10G -n Jaunty Ubuntu
sudo lvcreate -L 4G -n Jaunty_var Ubuntu
sudo lvcreate -L 20G -n home Ubuntu
sudo lvcreate -L 4G -n swap Ubuntu

Encryption

The disk-mapper encryption module needs loading: 

sudo modprobe dm-crypt

Now create encrypted volumes inside one or more of the LVMs. In my case I shall be encrypted all these volumes but the volumes I create later for package source-code and media will be unencrypted.

I use key-files instead of passwords for encrypted volumes. I choose an existing innocuous looking file that has plenty of pseudo-random content such as a digital media file that won't look out-of-place on a USB flash-memory device. I ensure the path to the file is the same on the USB key and on the running system so that one /etc/crypttab configuration works whether from the initrd image (the initial ram-disk) used by the kernel to unlock the start-up disks, or for unlocking other volumes that are required later. This avoids having to have the USB key plugged in after start-up. 

sudo cryptsetup --hash sha512 --key-size 256 --cipher aes-cbc-essiv:sha256 \
 luksFormat /dev/Ubuntu/Jaunty /media/USB/home/tj/Media/theme-song.mp3 
sudo cryptsetup --hash sha512 --key-size 256 --cipher aes-cbc-essiv:sha256 \ 
 luksFormat /dev/Ubuntu/Jaunty_var /media/USB/home/tj/Media/theme-song.mp3
sudo cryptsetup --hash sha512 --key-size 256 --cipher aes-cbc-essiv:sha256  \
 luksFormat /dev/Ubuntu/home /media/USB/home/tj/Media/theme-song.mp3

Open Encrypted Volumes

sudo cryptsetup luksOpen /dev/Ubuntu/Jaunty root --key-file /media/USB/home/tj/Media/theme-song.mp3
sudo cryptsetup luksOpen /dev/Ubuntu/Jaunty_var var --key-file /media/USB/home/tj/Media/theme-song.mp3
sudo cryptsetup luksOpen /dev/Ubuntu/home home --key-file /media/USB/home/tj/Media/theme-song.mp3

The volumes will now appear in /dev/mapper/ 

ls -l /dev/mapper/
total 0
crw-rw---- 1 root root  10, 61 2009-02-07 17:01 control
brw-rw---- 1 root disk 252,  6 2009-02-07 17:27 home
brw-rw---- 1 root disk 252,  4 2009-02-07 17:27 root
brw-rw---- 1 root disk 252,  2 2009-02-07 17:25 Ubuntu-home
brw-rw---- 1 root disk 252,  0 2009-02-07 17:25 Ubuntu-Jaunty
brw-rw---- 1 root disk 252,  1 2009-02-07 17:25 Ubuntu-Jaunty_var
brw-rw---- 1 root disk 252,  3 2009-02-07 17:11 Ubuntu-swap
brw-rw---- 1 root disk 252,  5 2009-02-07 17:27 var

The names that include "Ubuntu" are the LVM volumes. "home", "root" and "var" are the encrypted volumes 'inside' the LVM parent volumes. These encrypted volumes are where the file-systems are created.

File-System Creation

Partition 3 needs formatting with a file-system that GRUB can understand, so use ext3: 

sudo mkfs.ext3 -L boot /dev/sda3

The future swap volume needs preparing too (although the installer usually won't recognise it on pre-Jaunty releases): 

sudo mkswap -L swap /dev/Ubuntu/swap

The installation volumes are going to use ext4: 

sudo mkfs.ext4 -L Jaunty /dev/mapper/root
sudo mkfs.ext4 -L Jaunty_var /dev/mapper/var

But because the intention is to be able to share the user home directories across multiple releases the home volume will be ext3: 

sudo mkfs.ext3 -L home /dev/mapper/home

Installation

Start from the 'desktop' live-CD. Ensure the PC has Internet access (for installing additional packages later). Start a terminal (Applications > Accessories > Terminal).

Note: Where there are differences in the procedure for different OS releases I'll try to detail them here. The 'default' (standard instructions) are in the Jaunty section.

Run the installer (double-click on the Install icon on the desktop and at the partitioning stage choose Manual. From there, allocate the volumes to the appropriate mount-points for their intended purpose, select the appropriate file-system type (ext3, ext4, etc.) but do not choose to format them since you've already done that. Pre-Jaunty the swap volume sometimes doesn't show up (/dev/mapper/Ubuntu-swap) so don't try to assign the swap volume at this point.

Ensure that the /boot mount-point (in my case /dev/sda3) is correctly set otherwise the resulting system will be unable to start and won't even see the installation.

When prompted with warnings about swap and formatted partitions select "Continue".

Post-Installation Requirements

The target system needs to be modified before the system is restarted because the system is installed into LVM volumes which contain encryption and the kernel modules to support them need to be available during start-up. The installation process doesn't currently do that.

Mount Target System

All the file-systems that make up the target system need to be mounted together so they appear to be arranged as they would in the running system. Create a mount point and mount them all and add the kernel's proc dev and sys file-systems too: 

sudo mkdir -p /target
sudo mount /dev/mapper/root /target
sudo mount /dev/mapper/var /target/var
sudo mount /dev/mapper/home /target/home
sudo mount /dev/sda3 /target/boot
sudo mount --bind /proc /target/proc
sudo mount --bind /dev /target/dev
sudo mount --bind /sys /target/sys

Switch to Target

Now switch into the target system to modify settings as if it were the running system: 

sudo chroot /target

Install Additional Packages

LVM and cryptographic block devices are required for the system to start-up: 

apt-get install lvm2 cryptsetup

Configure Custom Crypto Boot Process

This a script I developed that is installed in the initial ram-disk and handles the unlocking of the encrypted volumes. It can be downloaded and installed directly: 

wget http://tjworld.net/raw-attachment/wiki/Linux/Ubuntu/HardyRAID5EncryptedLVM/crypto-usb-key.sh \
 -O /usr/local/sbin/crypto-usb-key.sh
chmod a+x /usr/local/sbin/crypto-usb-key.sh

Add entries to the crypt-table (/etc/crypttab) that tell cryptsetup which volumes to unlock and where to find the key: 

echo "root /dev/disk/by-uuid/$(vol_id --uuid /dev/Ubuntu/Jaunty) /home/tj/Media/theme-song.mp3 luks,keyscript=/usr/local/sbin/crypto-usb-key.sh" >> /etc/crypttab
echo "var /dev/disk/by-uuid/$(vol_id --uuid /dev/Ubuntu/Jaunty_var) /home/tj/Media/theme-song.mp3 luks,keyscript=/usr/local/sbin/crypto-usb-key.sh" >> /etc/crypttab
echo "home /dev/disk/by-uuid/$(vol_id --uuid /dev/Ubuntu/home) /home/tj/Media/theme-song.mp3 luks,keyscript=/usr/local/sbin/crypto-usb-key.sh" >> /etc/crypttab

Modify Initial RAM-Disk Configuration

Because the required modules aren't included in the kernel image they need to be added to the initrd (initial ram-disk) image that contains the initial boot-strap scripts, supporting binary executables and configuration files.

Some module-names are based upon the specific machine architecture and therefore the names are built using some minor shell helpers:

Jaunty +

ARCH=$(uname -m)
echo -e "aes_${ARCH}\naes_generic\ndm-crypt\nsha256_generic\n" | sudo tee -a /etc/initramfs-tools/modules

Pre-Jaunty

The dm-mod module (disk mapper) is also required: 

ARCH=$(uname -m)
echo -e "aes_${ARCH}\naes_generic\ndm-mod\ndm-crypt\nsha256_generic\n" | sudo tee -a /etc/initramfs-tools/modules

These commands add the required module names to the list of modules that should be included in the initrd image.

Update Initial RAM-Disk Image

Update all existing initrd images: 

sudo update-initramfs -k all -u

Leave the Target

exit