[[PageOutline]] = RAID-5 Encrypted with Logical Volume Management = These instructions in this tutorial should apply to Intrepid (and possibly Jaunty) as well as Hardy. Some slight adjustments might be necessary due to changes in various system libraries and tools. The key-script has been updated to work with Intrepid. These instructions show how to use the Ubuntu Hardy Desktop Live CD to build and install to a secure encrypted multi-disk RAID system that requires a key-file on an external USB memory stick in order to start. It works around several ''bugs'' in the Ubiquity installer in order to work with ''md'' software RAID arrays (thus avoiding the need to use the alternate text-based installer CD image). In [wiki:Linux/Ubuntu/HardyEncryptedLVM another article there are similar instructions for encrypted non-RAID systems] such as notebooks and laptops that have a single disk drive. The test system being configured has four 61GB parallel ATA disks on an IDE controller. Instead of using the Fastrak proprietary soft-RAID (which requires the Linux dmraid driver), the RAID is being managed completely by Linux. There will be three RAID arrays: * /dev/md0 RAID-1 (Mirrored), not encrypted. For /boot. 3GB (only needs 250MB). * /dev/md1 RAID-1 (Mirrored), not encrypted. For swap & hibernate. 3GB. * /dev/md2 RAID-5 (Striping with Parity), encrypted. Logical Volume Management. For /, /var, and /home. Remaining disk space. /boot contains the kernel image that GRUB loads via the BIOS and therefore cannot be encrypted or allocated via LVM (BIOS/GRUB do not know how to deal with encryption or LVM). It is placed in a RAID-1 mirror because when GRUB starts it has no way to read a software RAID-5 array. RAID-1 is usable because each disk/partition has identical data on it. GRUB will boot from either of the two drives in the RAID-1 array but is unaware of the mirror. The swap & hibernate partition uses a RAID-1 mirror too, sized based on the installed system RAM plus a margin (system has 2GB RAM), which is unencrypted (However, encryption is possible because the kernel has been loaded and started by GRUB at the point the hibernate image is read - I might add instructions for encrypting once I've had chance to test it). The mirrors are arranged so the disks are on different controller channels, which allows for concurrent access. Access to the Internet is required to install packages not on the LiveCD. Alternatively, download the .deb packages for [http://packages.ubuntu.com/hardy/mdadm mdadm], [http://packages.ubuntu.com/hardy/cryptsetup cryptsetup] and [http://packages.ubuntu.com/hardy/lvm2 lvm2] and make them available locally on a USB memory stick, CD, or floppy disk (they require about 1MB) and install them using: {{{ dpkg -i .deb }}} == Organisation == The physical system looks like this:[[BR]] [[Image(RAIDencryptedLVM-physical.png)]] The boot files and swap are in RAID-1 (mirrored) arrays that are accessible to GRUB and BIOS:[[BR]] [[Image(RAIDencryptedLVM-logical-RAID1.png)]] The operating system and user data are in an encrypted RAID-5 (striped plus parity) array:[[BR]] [[Image(RAIDencryptedLVM-logical-RAID5.png)]] == Boot from the Desktop Live CD == There is no operating system or other data on the disks so the Live CD environment is used to prepare the disks prior to installing Hardy. Once the OS has started open a terminal window (press Alt+F2 to start the '''Run Application''' dialog, type `gnome-terminal` and press the '''Run''' button). In the terminal window assume super-user privileges: {{{ sudo su }}} == Randomise the disk surface == By ensuring every sector of the disks is written with random data, a potential attacker will have great difficulty locating encrypted data that they might want to try to decrypt. If the surface of the disk was written with zeros (using if=/dev/zero) or some other predictable values encrypted data would be easy to identify if the disk were to fall into hostile hands. This is likely to take a long time - '''possibly 24 hours or more''' - even with running one background process for each disk: {{{ for dr in a b c d; do DEV="/dev/sd${dr}"; sh -c "dd if=/dev/urandom of=$DEV bs=512"& done }}} Check the progress by sending the '''USR1''' signal to the `dd` processes: {{{ ps -ef | sed -n 's/[a-z ]*\([0-9]*\).* dd.*/\1/p' | while read PID; do kill -USR1 $PID; done }}} == Partition the disks == Each disk is identically partitioned. The RAID-5 array will use the majority of the space. GRUB and BIOS require a ''regular'' disk layout in order to boot the system, so a small partition for /boot is first on the disks. '''Note:''' Although /boot only really requires at most 250MB (this is a development and testing PC), the size of the second mirror used for swap & hibernate dictates its large size because the two mirrors use related partitions. This requirements is driven by the following RAID-5 array requiring all its physical components to be identical in size. We repeat the same partitioning procedure for each disk, creating a 3GB boot/swap partition and put the remainder of the disk in the second partition: {{{ for dr in a b c d; do DEV="/dev/sd${dr}"; echo -e "o\nn\np\n1\n\n+3G\nn\np\n2\n\n\nt\n1\nfd\nt\n2\nfd\np\nw\n" | fdisk $DEV; done }}} '''Note:''' At this point, if fdisk reports the kernel can't re-read the partition tables: {{{ WARNING: Re-reading the partition table failed with error 16: Device or resource busy. The kernel still uses the old table. The new table will be used at the next reboot. }}} you will need to restart the system: {{{ shutdown -r now }}} and once it has restarted continue from ''Create the RAID arrays''. == Create the RAID arrays == The RAID arrays will be made up of groups of partitions. Install the Linux RAID package: {{{ apt-get install mdadm }}} First the mirror array for /boot, which will use the first partition on two disks on different IDE channels (to avoid master/slave bootlenecks): {{{ mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sda1 /dev/sdc1 mdadm: array /dev/md0 started. }}} Now create a mirror array for swap & hibernate: {{{ mdadm --create /dev/md1 --level=1 --raid-devices=2 /dev/sdb1 /dev/sdd1 mdadm: array /dev/md1 started. }}} All remaining space (in the second partition) is allocated to the RAID-5 array. Its capacity will be the number of disks in the array minus one, multiplied by the size of the partition: (N - 1) x C : (4 - 1) x 58GB = 174GB. {{{ mdadm --create /dev/md2 --level=5 --raid-devices=4 /dev/sda2 /dev/sdb2 /dev/sdc2 /dev/sdd2 mdadm: array /dev/md2 started. }}} Check the array build status: {{{ cat /proc/mdstat Personalities : [raid1] [raid6] [raid5] [raid4] md2 : active raid5 sdd2[4] sdc2[2] sdb2[1] sda2[0] 171332928 blocks level 5, 64k chunk, algorithm 2 [4/3] [UUU_] resync=DELAYED md1 : active raid1 sdd1[1] sdb1[0] 2939776 blocks [2/2] [UU] [==========>..........] resync = 54.2% (1594496/2939776) finish=0.6min speed=33434K/sec md0 : active raid1 sdc1[1] sda1[0] 2939776 blocks [2/2] [UU] [==============>......] resync = 74.1% (2179200/2939776) finish=0.3min speed=33574K/sec unused devices: }}} Wait until the arrays are finished building. Press Ctrl+C to interrupt `watch` when all the arrays have finished building: {{{ watch -n 30 cat /proc/mdstat Personalities : [raid1] [raid6] [raid5] [raid4] md2 : active raid5 sdd2[3] sdc2[2] sdb2[1] sda2[0] 171332928 blocks level 5, 64k chunk, algorithm 2 [4/4] [UUUU] md1 : active raid1 sdd1[1] sdb1[0] 2939776 blocks [2/2] [UU] md0 : active raid1 sdc1[1] sda1[0] 2939776 blocks [2/2] [UU] unused devices: }}} == Encryption == First, install the cryptography package: {{{ apt-get install cryptsetup }}} === Choosing a key-file === The key-file is kept on an external USB memory stick. There are various ways to create and store the key-file. Many guides recommend generating a random key from `/dev/random` but in my opinion anyone that managed to get access to the memory stick could easily locate the key-file because of its totally random contents, unless many 'fake' keys were also on the memory stick. My preferred method now is to use a ''real'' file. Install the [https://wiki.ubuntu.com/LiveUsbPendrivePersistent Hardy Desktop Live image on the USB memory stick] so it is bootable on any system. Because the installation has a persistent `/home/` I can copy some music and image files onto it and use one of those as the key-file. The reason I choose music and images is that most MP3 and PNG or JPG files are highly compressed and their data can have a high degree of variability, and is hard to predict. An attacker would need to know the name of the key-file to determine which of the thousands of files was the key-file. To do that, they'd need access to the encrypted system itself. For the purposes of this guide let us assume we're using a self-created music file called '''`theme-song.mp3`''' (Being self-created no-one else will have the same file - in other words, don't use a commonly shared file). Put the file on the USB memory stick and then plug it into the new system. Ensure it is mounted. In this example the USB key has two volumes on it (ubuntu & casper-rw) and the persistent files are in the casper-rw volume which usually is auto-mounted at `/media/casper-rw` (see [https://wiki.ubuntu.com/LiveUsbPendrivePersistent#head-0977f1117374135da452bd135c75f8d9fcd5bf68 Installing Ubuntu on USB pendrive using Linux]. Load the kernel module: {{{ modprobe dm-crypt }}} === Encrypt the arrays === Now encrypt the RAID-5 array: {{{ cryptsetup --hash sha512 --key-size 256 --cipher aes-cbc-essiv:sha256 \ luksFormat /dev/md2 /media/casper-rw/home/tj/Media/theme-song.mp3 WARNING! ======== This will overwrite data on /dev/md2 irrevocably. Are you sure (Type uppercase yes): YES Command successful }}} Open the encrypted device, giving it the name ''md2encrypted'': {{{ cryptsetup --key-file /media/casper-rw/home/tj/Media/theme-song.mp3 luksOpen /dev/md2 md2encrypted key slot 0 unlocked. Command successful. }}} There will now be a new device: {{{ ls -1 /dev/mapper control md2encrypted }}} == Configure Logical Volume Management (LVM) == First install the LVM package: {{{ apt-get install lvm2 }}} ==== LVM Tools Confuse Megabytes with Mebibytes ==== The LVM tool reports can be confusing because they use the wrong size suffixes (such as MB and GB). The problem is, a gigabyte (GB) is 1,000MB, a megabyte (MB) is 1,000 kilobytes, and a kilobyte (KB) is 1,000 bytes. However, LVM uses binary-based calculations, not the decimal, with KB = 1,024, MB = 1,024KB, and GB = 1,024MB. LVM ''should'' use [http://en.wikipedia.org/wiki/Mebibytes MiB] and [http://en.wikipedia.org/wiki/Gibibyte GiB] suffixes to indicate binary-based measurements. The result is, if you use fdisk to create (for example) a 3,256MB partition (3.256 x 1000 x 1000 x 1000 = 3256000000 bytes) on a physical device and then create an LVM physical volume with it, the reports from pvdisplay and vgdisplay will show the size as 3.04''GB''. (3.04 x 1024 x 1024 x 1024 = 3264175144.96). In fact, 3,256MB = 3.256GB = 3.032386303 GiB so the lvm tools have rounded-up to the next tenth. It looks as if you've miscalculated or lost disk space somewhere, but in fact LVM tools are causing confusion. ==== Determine the Number and Size of Logical Extents in Logical Volumes ==== Logical volume size is defined as the number of ''logical extents'' (LE), the size of which is the same for all logical volumes in a volume group and is the same as the ''physical extent'' (PE) size. Use `vgdisplay` to find out: {{{ vgdisplay VGraid5 --- Volume group --- VG Name VGraid5 System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 4 VG Access read/write VG Status resizable MAX LV 0 Cur LV 3 Open LV 3 Max PV 0 Cur PV 1 Act PV 1 VG Size 163.39 GB PE Size 4.00 MB Total PE 41829 Alloc PE / Size 33040 / 129.06 GB Free PE / Size 8789 / 34.33 GB VG UUID V0kRhd-oFLa-hq21-9eCs-kAnm-e1BW-xK7GPT }}} The '''PE Size''' (physical extent size) for this volume group is 4.00''MB''. The '''Total PE''' (number of extents) is 43,794. So, in fact, the PE Size is 4.00''MiB'' (4.00 x 1024 x 1024 = 4,194,304 bytes), or 4.194304''MB'' (4.194304 x 1000 x 1000 = 4,194,304 bytes). To make later calculations easier set up some definitions: {{{ export GB=1000000000 export MB=1000000 export GiB=1073741824 export MiB=1048576 }}} === Encrypted Volume === Create the physical volume and volume group: {{{ pvcreate /dev/mapper/md2encrypted Physical volume "/dev/mapper/md2encrypted" successfully created vgcreate -v VGraid5 /dev/mapper/md2encrypted Adding physical volume '/dev/mapper/md2encrypted' to volume group 'VGraid5' Creating directory "/etc/lvm/archive" Archiving volume group "VGraid5" metadata (seqno 0). Creating directory "/etc/lvm/backup" Creating volume group backup "/etc/lvm/backup/VGraid5" (seqno 1). Volume group "VGraid5" successfully created }}} Create the logical volume 'root' using 18GB in the volume group 'VGraid5': {{{ export PES_BYTES=$(echo "$(vgdisplay VGraid5 | sed -n 's/.*PE Size *\([0-9\.]*\) .*/\1/p') * $MiB" | bc) EXTENTS=$(echo "18 * $GB / $PES_BYTES" | bc); echo $EXTENTS 4291 lvcreate -l $EXTENTS -n root VGraid5 Logical volume "root" created }}} Repeat for `/var/` (10GB) and `/home/` (75% of remaining free space): {{{ EXTENTS=$(echo "10 * $GB / $PES_BYTES" | bc); echo $EXTENTS 2384 lvcreate -l $EXTENTS -n var VGraid5 Logical volume "var" created lvcreate -l 75%FREE -n home VGraid5 Logical volume "home" created }}} Using `vgdisplay` you can see this will leave ~34GB of free space. If or when one of the existing logical volumes reaches capacity simply use `lvextend` to allocate more extents from the volume group - no need to shuffle data or partitions around. {{{ vgdisplay VGraid5 | grep 'Free PE' Free PE / Size 8789 / 34.33 GB }}} Check the devices are available: {{{ ls -1 /dev/mapper control md2encrypted VGraid5-home VGraid5-root VGraid5-var }}} Now the system has all the devices ready for formatting with file-systems, so installation of the operating system can begin. == Install Ubuntu == === Fix a bug in the partition manager scripts === Ubiquity (the Ubunutu Live CD installer) depends on a set of scripts to partition the system disks. These are extremely complicated due to the vast number of permutations of storage device types the user might want to install to. As a result of ancient bugs it doesn't support installing to ''plain'' multiple-disk (md) arrays (/dev/md*), although it ''does'' support LVM devices (/dev/mapper/*) on md arrays. Because GRUB and the BIOS need to access the /boot partition (and cannot access LVM devices) /boot must be on either a plain single disk or a RAID-1 mirror. Unfortunately, Ubiquity does not ''see'' md arrays although if the alternate text-based installer CD-image is used /boot ''can'' be installed on an md array. There is a quick-and-dirty workaround to allow Ubiquity to see the md arrays. It is a patch that simply comments out a line that ''ignores'' md devices. It is possible it could cause other issues so beware, but from my experience in the scenario described in this article, it was successful. Run this command before starting the installer: {{{ sed -i 's,^\([\t ]*grep -v .^/dev/md. |\),#\1,' /lib/partman/init.d/30parted }}} Check the line has had a # prefixed to comment it out: {{{ grep '/dev/md' /lib/partman/init.d/30parted # grep -v '^/dev/md' | }}} The md devices also need to be manually formatted (''not'' partitioned though) because Ubiquity can't handle that: {{{ mkfs.ext3 -L boot /dev/md0 }}} The swap partition will be formatted after the installer has finished to avoid causing a fatal error when Ubiquity fails to mount it. === Configuring === Run the Ubuntu installer by double-clicking the '''Install''' icon on the Live CD desktop. Select the language, time-zone and keyboard layout. In the partitioner (''Prepare disk space'' dialog) choose '''Manual''', then press the '''Forward''' button. After it has scanned the disks it will present a list of devices. As well as the ''physical'' devices (/dev/sda, /dev/sdb, etc.) there will be the logical devices just created with names beginning `/dev/mapper/` and, if you applied the patch above, `/dev/md`. The choice of device names should make it straight-forward to identify the intended purpose of each. Let's begin. The ''device'' list will show amongst others: {{{ /dev/md0 /dev/md0 ext3 3010 MB 95 MB }}} Select the indented `/dev/md0 ... ext3` line. Press the '''Edit partition''' button. Change '''Use as''' to '''Ext3'''. Tick the Format check-box. Set the '''mount point''' to '''`/boot`'''. Select `/dev/mapper/VGraid5-home`. Press the '''New partition table''' button. Select the '''free space'''. Press the '''New partition''' button. Choose '''Primary''' partition, use all the space (accept the default value for partition size), location at '''Beginning'''. Use as '''Ext3'''. Set the '''mount point''' to '''`/home`'''. Select `/dev/mapper/VGraid5-root`. Press the '''New partition table''' button. Select the '''free space'''. Press the '''New partition''' button. Choose '''Primary''' partition, use all the space (accept the default value for partition size), location at '''Beginning'''. Use as '''Ext3'''. Set the '''mount point''' to '''`/`''' (root). Select `/dev/mapper/VGraid5-var`. Press the '''New partition table''' button. Select the '''free space'''. Press the '''New partition''' button. Choose '''Primary''' partition, use all the space (accept the default value for partition size), location at '''Beginning'''. Use as '''Ext3'''. Set the '''mount point''' to '''`/var`'''. Review the list to ensure it is correct, then press the '''Forward''' button to commit the changes. '''Note:''' Because of a [https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/245855 bug in the Desktop Live CD installer] it is not possible to create the swap partition during installation. Therefore the installer will show a dialog with the message: You have not selected any partitions for use as swap space. Enabling swap space is recommended so that the system can make better use of available physical memory, and so that it behaves better when physical memory is scarce. You may experience installation problems if you do not have enough physical memory. If you do not go back to the partitioning menu and assign a swap partition, the installation will continue without swap space. Press the '''Continue''' button. Continue the installation process, answering the ''Who are you?'' questions. On the ''Ready to install'' page, review the choices. Press the '''Advanced...''' button and '''disable''' (un-tick) "Install boot loader". This must be done manually after the installer has finished. Finally, when you're happy with all the choices, press the '''Install''' button and wait whilst the installer runs. When it finishes '''DO NOT''' restart the system - some additional configuration is required before the system is restarted so it will boot successfully. === Post-Installation configuration === Now a series of additional steps is required to ensure the new system can successfully boot. ==== Mount the Target System ==== Prepare the installation for access using chroot: {{{ mkdir /mnt/target mount /dev/mapper/VGraid5-root /mnt/target mount /dev/mapper/VGraid5-var /mnt/target/var mount /dev/mapper/VGraid5-home /mnt/target/home mount /dev/md0 /mnt/target/boot mount -o bind /proc /mnt/target/proc mount -o bind /dev /mnt/target/dev }}} ==== Swap ==== Format the swap partition and add it to the file-system table so it is started automatically: {{{ mkswap /dev/md1 echo "UUID=$(vol_id --uuid /dev/md1) none swap sw 0 0" >> /mnt/target/etc/fstab }}} ==== GRUB (boot loader) ==== GRUB needs to be installed to both physical disks that make up the /dev/md0 mirror. Install GRUB: {{{ chroot /mnt/target /bin/bash -c "apt-get install grub" chroot /mnt/target /bin/bash -c "mkdir -p /boot/grub" chroot /mnt/target /bin/bash -c "cp /usr/lib/grub/*/{,e2fs_}stage* /boot/grub" }}} Write the master boot record (MBR) to sector #0 of both disks. Start GRUB then issue a series of commands to locate and then install the boot loader files: {{{ grub grub> find /grub/stage2 (hd0,0) (hd2,0) grub> setup (hd0) (hd0,0) Checking if "/boot/grub/stage1" exists... no Checking if "/grub/stage1" exists... yes Checking if "/grub/stage2" exists... yes Checking if "/grub/e2fs_stage1_5" exists... yes Running "embed /grub/e2fs_stage1_5 (hd0)"... 16 sectors are embedded. succeeded Running "install /grub/stage1 (hd0) (hd0)1+16 p (hd0,0)/grub/stage2 /grub/menu.lst"... succee ded Done. grub> setup (hd2) (hd2,0) Checking if "/boot/grub/stage1" exists... no Checking if "/grub/stage1" exists... yes Checking if "/grub/stage2" exists... yes Checking if "/grub/e2fs_stage1_5" exists... yes Running "embed /grub/e2fs_stage1_5 (hd2)"... 16 sectors are embedded. succeeded Running "install /grub/stage1 (hd2) (hd2)1+16 p (hd2,0)/grub/stage2 /grub/menu.lst"... succee ded Done. grub> quit }}} Write GRUB configuration: {{{ chroot /mnt/target /bin/bash -c "update-grub" Searching for GRUB installation directory ... found: /boot/grub Searching for default file ... Generating /boot/grub/default file and setting the default boot entry to 0 Searching for GRUB installation directory ... found: /boot/grub Testing for an existing GRUB menu.lst file ... Could not find /boot/grub/menu.lst file. Would you like /boot/grub/menu.lst generated for you? (y/N) y Searching for splash image ... none found, skipping ... Found kernel: /memtest86+.bin Found kernel: /vmlinuz-2.6.24-19-generic Found kernel: /memtest86+.bin Updating /boot/grub/menu.lst ... done }}} ==== Install mdadm, cryptsetup, and lvm2 ==== The installer didn't install the packages needed for the system to understand the disk configuration, so install them manually: {{{ chroot /mnt/target /bin/bash -c "apt-get install mdadm cryptsetup lvm2" }}} ==== Multiple Disk Configuration ==== The `mdadm.conf` should have been written automatically when the mdadm package was installed into the chroot environment. Check if the existing file has entries for the three RAID devices: {{{ grep md /mnt/target/etc/mdadm/mdadm.conf # mdadm.conf # Please refer to mdadm.conf(5) for information about this file. ARRAY /dev/md0 level=raid1 num-devices=2 UUID=baaae69f:8fba6334:e368bf24:bd0fce41 ARRAY /dev/md1 level=raid1 num-devices=2 UUID=e671d608:2e4b610d:e368bf24:bd0fce41 ARRAY /dev/md2 level=raid5 num-devices=4 UUID=d47bada7:1dc7b43b:e368bf24:bd0fce41 }}} The lines above show the configuration is correct, and nothing else needs doing. Otherwise the file can be copied from the Live CD environment: {{{ cp /etc/mdadm/mdadm.conf /mnt/target/etc/mdadm/mdadm.conf }}} Alternatively, it can be written by `mdadm` itself. {{{ mdadm --detail --brief /dev/md* >> /mnt/target/etc/mdadm/mdadm.conf }}} ==== Encrypted Disk Configuration ==== Write the configuration of /dev/mapper/md2encrypted so the system knows how to open it. It will use a shell script that is executed early in the boot process from the initrd image: {{{ echo "md2encrypted /dev/disk/by-uuid/$(vol_id --uuid /dev/md2) /home/tj/Media/theme-song.mp3 luks,keyscript=/usr/local/sbin/crypto-usb-key.sh" >> /mnt/target/etc/crypttab }}} There are several suitable shell scripts already existing. I chose to modify one written by Wejn and Rodolfo Garcia published at [http://wejn.org/how-to-make-passwordless-cryptsetup.html How to setup passwordless disk encryption in Debian Etch]. It is [http://tjworld.net/raw-attachment/wiki/Linux/Ubuntu/HardyRAID5EncryptedLVM/crypto-usb-key.sh attached to this article] ready for downloading. Key improvements and functions are: * Works with Ubuntu 8.04 Hardy * Detects and uses usplash or console for messages * Works for root and other early-crypto disks * Looks for keys on already-mounted volumes (keys stored in the root-crypto disk) before trying USB * Automatic detection of USB devices * Detection and examination of *all* partitions on the device (not just partition !#1) * Automatic detection of partition type * Refactored the code * Added optional debugging code * Added comments '''Note:''' Thanks to feedback from Dave at my-iop I've since fixed a password-reading bug and added more functionality to the script. It should now deal with opening key-files from mounted disks (great if you store key-files for some volumes inside the initial encrypted volume). It simply checks for the existence of the key-file and if it finds it bypasses all the USB functionality and passes the contents of the key-file back to the crypto manager. Copy it into the new system (ensure the path and name match that specified in /mnt/target/etc/crypttab): {{{ wget http://tjworld.net/raw-attachment/wiki/Linux/Ubuntu/HardyRAID5EncryptedLVM/crypto-usb-key.sh \ -O /mnt/target/usr/local/sbin/crypto-usb-key.sh chroot /mnt/target /bin/bash -c "chmod a+x /usr/local/sbin/crypto-usb-key.sh" }}} ==== Update Initial RAM Disk (initrd) ==== {{{ chroot /mnt/target /bin/bash -c "update-initramfs -u all" update-initramfs: Generating /boot/initrd.img-2.6.24-19-generic }}} == Restart System and Test == With everything written to the disks it is time to restart: {{{ shutdown -r now }}} Remove the installation CD from the drive when prompted. When the system starts ensure the BIOS is configured to boot from one of the drives that is in the /dev/md0 RAID-1 array, and that it isn't configured to try booting from a USB device ''before'' the hard disk. Insert the stick into a USB port and let the system start. If using usplash you should see a message similar to "Success loading key-file from sde (casper-rw)" early in the boot process when `crypto-usb-key.sh` locates the key-file. If it fails you'll see "FAILED to find suitable USB key-file ..." followed by "Try to enter the LUKS password:". At this point, if the device has a pass-phrase as well as a key-file you can type it instead of using the key-file. === Debugging crypto-usb-key.sh === If that fails you'll need to enable debugging within the script by changing: {{{ DEBUG=$FALSE }}} to {{{ DEBUG=$TRUE }}} To do this the initial RAM disk image needs updating. That means the encrypted volume must be opened and mounted and the chroot environment recreated from the Live CD. Because that involves a lot of steps I put all the commands into a script. The script is attached to this article as [http://tjworld.net/attachment/wiki/Linux/Ubuntu/HardyRAID5EncryptedLVM/update-liveCD-RAIDencryptedLVM.sh update-liveCD-RAIDencryptedLVM.sh]. It will install the packages, configure and unlock the devices, recreate the chroot environment, download and install the latest version of `crypto-usb-key.sh` attached to this article, then update the initial RAM disk image. Restart the system using the Live CD and open a terminal, then: {{{ sudo su wget http://tjworld.net/raw-attachment/wiki/Linux/Ubuntu/HardyRAID5EncryptedLVM/update-liveCD-RAIDencryptedLVM.sh chmod a+x update-liveCD-RAIDencryptedLVM.sh }}} The script requires the key-file name be assigned to the environmental variable KEYFILE: {{{ KEYFILE="/home/tj/Media/theme-song.mp3" ./update-liveCD-RAIDencryptedLVM.sh }}} Now make the changes and update the initial RAM disk image: {{{ sed -i 's/^\(DEBUG=\)$FALSE/\1$TRUE/' /mnt/target/usr/local/sbin/update-usb-key.sh update-initramfs -u all }}} You might also want to remove the "splash" option from the kernel command-line in the GRUB configuration (either in the file /boot/grub/menu.lst or by pressing Escape when GRUB is starting after a reboot to edit the menu directly). This will ensure that the large number of debug messages from the script are easily readable on the console, rather than scrolling up too fast in usplash. Now the system can be restarted and hopefully the messages will help identify the cause of the problem, and suggest a solution. Once the problem is fixed don't forget to change the debug setting back to $FALSE and update the initial RAM disk image. == References == [https://wiki.ubuntu.com/LiveUsbPendrivePersistent Installing Ubuntu on USB pendrive using Linux][[BR]] [http://mazeoflies.com/articles/2008/06/09/ubuntu-hard-drive-encryption-with-external-key Ubuntu hard drive encryption with external key][[BR]] [https://help.ubuntu.com/community/EncryptedFilesystemLVMHowto Installing Ubuntu 7.04 on an Encrypted LVM Partition For Root, Swap, and Home][[BR]] [http://wejn.org/how-to-make-passwordless-cryptsetup.html How to setup passwordless disk encryption in Debian Etch][[BR]] [http://linuxgazette.net/140/pfeiffer.html Encrypted Storage with LUKS, RAID and LVM2][[BR]] [http://www.gagme.com/greg/linux/raid-lvm.php Managing RAID and LVM with Linux (v0.5)][[BR]] [http://www.howtoforge.com/linux_lvm_p6 A Beginner's Guide To LVM - LVM On RAID1][[BR]] [http://www.mythtv.org/wiki/index.php/RAID#RAID_0.2B1_.28or_1.2B0.2C_01.2C_10.29 RAID] == Updates == 4 December 2008 '''crypto-usb-key.sh''': Dropped detection of specific USB devices because Intrepid and kernel 2.6.27 no longer include "usb" in the /sys/block/.../device path. Now the script relies purely on the 'removable' flag in determining which devices to look on for the keyfile. Fixed msg() not printing to console when using Intrepid. Simplified detection of running usplash. 9 February 2009 '''crypto-usb-key.sh''': Added support for Jaunty, made script work in initrd and after root is mounted, use vol_id for FSTYPE and LABEL if possible, remove reliance on basename, minimise wait for USB device to settle by monitoring dmesg for attache event.