|
Full Disk Encryption (with ZFS root) for FreeBSD 9.xBoot from freebsd memdisk (usb stick), use „shell“. For the german of us, enter kbdcontrol -l german.iso Installation Part 1.1a: MBR, zfs boot (4K style)Clean up the disk, maybe use -F gpart destroy ada0 Create new MBR disk gpart create -s mbr ada0 Create first partition (/boot), 504 is 4K aligned gpart add -b 504 -s 20971520 -t freebsd ada0 Create next partition (root) gpart add -b 20972448 -t freebsd ada0 Set active partition gpart set -a active -i 1 ada0 Installation Part 1.1b: MBR, slices, zfs boot (4K style), used for geli crypted zfs root with seperate zfs boot as sliceClean up the disk, maybe use -F gpart destroy ada0 Create new MBR disk gpart create -s mbr ada0 Create first partition, 504 is 4K aligned gpart add -b 504 -a 4k -t freebsd ada0 Create BSD type gpart create -s bsd ada0s1 Create first partition slice (/boot) gpart add -s 10G -a 4k -t freebsd-zfs ada0s1 Create root partition slice gpart add -a 4k -t freebsd-zfs ada0s1 Set first partition as active gpart set -a active -i 1 ada0 Add bootcode to hd gpart bootcode -b /boot/boot0 ada0 Add boot1 stage dd if=/boot/zfsboot of=/dev/ada0s1 count=1 Add boot2 stage dd if=/boot/zfsboot of=/dev/ada0s1a skip=1 seek=1024 Installation Part 1.1c: GPT, protected mbr GPT stuff (4K style)Use this if your system is not gpt compatible. Clean up the disk, maybe use -F gpart destroy ada0 Create new GPT disk gpart create -s mbr ada0 Create bootcode, compatibility bootcode for mbr→gpt on mbr gpart bootcode -b /boot/pmgr ada0 Math: 40*512b = 20kb + 88*512b = 44kb = 64kb/4kb = 16 ⇒ perfect alignment for 4kb drives AND 512b gpart add -b 40 -s 88 -t freebsd-boot ada0 GPT ZFS boot code on p1 gpart add -p /boot/gptzfsboot -i 1 ada0 /boot partition, uncrypted, still correct alligned, 10485760kb/4096kb=2560 gpart add -s 10G -t freebsd-zfs ada0 And the installation partition, filling all the disk gpart add -t freebsd-zfs ada0 Now, jump to „Part2“. Installation Part1.2: GPT setup styleOnly use Part1.1 OR Part1.3! The original text is captured from https://www.dan.me.uk/blog/2012/05/06/full-disk-encryption-with-zfs-root-for-freebsd-9-x/comment-page-1/, all credits to him! I made a copy here if the page is down or the link is lost. Be careful, i think the alignment of 4K does not match here! — To follow on from my post about full disk encryption (well almost), this is how to do the same but with a ZFS filesystem. Like the other post, your /boot folder (which contains your kernel and modules) will not be encrypted, but the rest of your filesystem will be. One disadvantage of this method is that you have to enter a passphrase for EACH disk in your ZFS system each boot. Encryption inside ZFS will appear at some point – but until then this will suffice ! Boot from any FreeBSD 9 install medium (except bootonly), and choose Live CD at the install menu. For the purposes of this article, I will assume that you’re using 4 disks (da0, da1, da2, da3), a 10GB /boot (this will be mirrored on each of the 4 disks), and the remaining space as a raidz1 (roughly similar to RAID5) ZFS filesystem. The contents will be encrypted with 256-bit AES-XTS encryption with a 4 kb random data partial key and a secondary passphrase (required to type on each boot). If your CPU supports the AESNI flag, the crypto(4) framework will utilise this too. First we need to remove any existing GPT or MBR partition tables on each of the disks (ignore any ‘invalid argument’ messages): gpart destroy -F da0 gpart destroy -F da1 gpart destroy -F da2 gpart destroy -F da3 Now we need to create a new GPT partition table on each disk: gpart create -s gpt da0 gpart create -s gpt da1 gpart create -s gpt da2 gpart create -s gpt da3 We will now create a 64kb boot partition (this contains the boot loader only, so is safe and required to be unencrypted): gpart add -s 128 -t freebsd-boot da0 gpart add -s 128 -t freebsd-boot da1 gpart add -s 128 -t freebsd-boot da2 gpart add -s 128 -t freebsd-boot da3 Next, we will create the /boot partition – you can adjust the sizes here if you need, but i’d suggest not shrinking it too much or you’ll get into problems when doing OS upgrades later… Note: this is mirrored not striped across the disks for maximum resilience – so will use 10GB on each disk for 10GB total usable space. gpart add -s 10G -t freebsd-zfs da0 gpart add -s 10G -t freebsd-zfs da1 gpart add -s 10G -t freebsd-zfs da2 gpart add -s 10G -t freebsd-zfs da3 Finally, we will assign the remaining space on each disk to the root ZFS partition. This will be encrypted before we build ZFS on top of it. gpart add -t freebsd-zfs da0 gpart add -t freebsd-zfs da1 gpart add -t freebsd-zfs da2 gpart add -t freebsd-zfs da3 Now that we’ve created daXp1 (bootloader), daXp2 (/boot partition), daXp3 (root partition) – we need to write the boot loader code to each disk: gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0 gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da1 gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da2 gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da3 Jump to Part2. Installation Part2Ok, next we will build a ramdisk to mount on /boot/zfs – this helps us mount things temporarily… mdconfig -a -t malloc -s 128m -u 2 newfs -O2 /dev/md2 mount /dev/md2 /boot/zfs Now we will load the modules required for ZFS and encryption: kldload opensolaris kldload zfs kldload geom_eli Next, we build a ZFS mirror for the /boot partition and mount it temporarily (to house the encryption key) – ignore any mention of unable to mount: zpool create bootdir mirror /dev/da0p2 /dev/da1p2 /dev/da2p2 /dev/da3p2 zpool set bootfs=bootdir bootdir mkdir /boot/zfs/bootdir zfs set mountpoint=/boot/zfs/bootdir bootdir zfs mount bootdir Now we generate a random 4kb encryption key that will form (along with passphrase) the encryption key for the disk: dd if=/dev/random of=/boot/zfs/bootdir/encryption.key bs=4096 count=1 We have everything we need to start encrypting the disks now… Enter your passphrase twice for each init phase and once again for each attach phase below: geli init -b -B /boot/zfs/bootdir/da0p3.eli -e AES-XTS -K /boot/zfs/bootdir/encryption.key -l 256 -s 4096 /dev/da0p3 geli init -b -B /boot/zfs/bootdir/da1p3.eli -e AES-XTS -K /boot/zfs/bootdir/encryption.key -l 256 -s 4096 /dev/da1p3 geli init -b -B /boot/zfs/bootdir/da2p3.eli -e AES-XTS -K /boot/zfs/bootdir/encryption.key -l 256 -s 4096 /dev/da2p3 geli init -b -B /boot/zfs/bootdir/da3p3.eli -e AES-XTS -K /boot/zfs/bootdir/encryption.key -l 256 -s 4096 /dev/da3p3 geli attach -k /boot/zfs/bootdir/encryption.key /dev/da0p3 geli attach -k /boot/zfs/bootdir/encryption.key /dev/da1p3 geli attach -k /boot/zfs/bootdir/encryption.key /dev/da2p3 geli attach -k /boot/zfs/bootdir/encryption.key /dev/da3p3 Now that we have encrypted and mounted the partitions, we can build a ZFS root filesystem on top of it like so: zpool create zroot raidz1 /dev/da0p3.eli /dev/da1p3.eli /dev/da2p3.eli /dev/da3p3.eli zfs set mountpoint=/boot/zfs/zroot zroot zfs mount zroot zfs unmount bootdir mkdir /boot/zfs/zroot/bootdir zfs set mountpoint=/boot/zfs/zroot/bootdir bootdir zfs mount bootdir Note we unmounted the old boot mirror and re-mounted it within the root filesystem. This will be used later to copy the kernel and modules into. Ok, now we create all our ZFS mounts with various options as follows: zfs set checksum=fletcher4 zroot zfs create -o compression=on -o exec=on -o setuid=off zroot/tmp chmod 1777 /boot/zfs/zroot/tmp zfs create zroot/usr zfs create zroot/usr/home cd /boot/zfs/zroot; ln -s /usr/home home zfs create -o compression=lzjb -o setuid=off zroot/usr/ports zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/distfiles zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/packages zfs create zroot/var zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/crash zfs create -o exec=off -o setuid=off zroot/var/db zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/db/pkg zfs create -o exec=off -o setuid=off zroot/var/empty zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/log zfs create -o compression=gzip -o exec=off -o setuid=off zroot/var/mail zfs create -o exec=off -o setuid=off zroot/var/run zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/tmp chmod 1777 /boot/zfs/zroot/var/tmp Now we’re ready to install FreeBSD onto the new ZFS partitions. We’re going to install the base, all sources and a generic kernel – this takes some time so please be patient… cd /boot/zfs/zroot unxz -c /usr/freebsd-dist/base.txz | tar xpf - unxz -c /usr/freebsd-dist/kernel.txz | tar xpf - unxz -c /usr/freebsd-dist/src.txz | tar xpf - Now we can set /var/empty to readonly: zfs set readonly=on zroot/var/empty And now we’re ready to chroot into the installed system to setup the configuration: chroot /boot/zfs/zroot Now that the base system and kernel are installed, we can move our /boot folder to it’s final place on the ZFS unencrypted mirror and do a little housekeeping: cd / mv boot bootdir/ ln -fs bootdir/boot mv bootdir/encryption.key bootdir/boot/ mv bootdir/*.eli bootdir/boot/ We need to setup an initial /etc/rc.conf which will mount all ZFS filesystems on boot: echo ‘zfs_enable=”YES”‘ > /etc/rc.conf touch /etc/fstab And an initial /boot/loader.conf that will load ZFS, encryption and settings for encrypted disks on boot: echo ‘vfs.zfs.prefetch_disable=”1″‘ > /boot/loader.conf echo ‘vfs.root.mountfrom=”zfs:zroot”‘ >> /boot/loader.conf echo ‘zfs_load=”YES”‘ >> /boot/loader.conf echo ‘aesni_load=”YES”‘ >> /boot/loader.conf echo ‘geom_eli_load=”YES”‘ >> /boot/loader.conf echo ‘geli_da0p3_keyfile0_load=”YES”‘ >> /boot/loader.conf echo ‘geli_da0p3_keyfile0_type=”da0p3:geli_keyfile0″‘ >> /boot/loader.conf echo ‘geli_da0p3_keyfile0_name=”/boot/encryption.key”‘ >> /boot/loader.conf echo ‘geli_da1p3_keyfile0_load=”YES”‘ >> /boot/loader.conf echo ‘geli_da1p3_keyfile0_type=”da1p3:geli_keyfile0″‘ >> /boot/loader.conf echo ‘geli_da1p3_keyfile0_name=”/boot/encryption.key”‘ >> /boot/loader.conf echo ‘geli_da2p3_keyfile0_load=”YES”‘ >> /boot/loader.conf echo ‘geli_da2p3_keyfile0_type=”da2p3:geli_keyfile0″‘ >> /boot/loader.conf echo ‘geli_da2p3_keyfile0_name=”/boot/encryption.key”‘ >> /boot/loader.conf echo ‘geli_da3p3_keyfile0_load=”YES”‘ >> /boot/loader.conf echo ‘geli_da3p3_keyfile0_type=”da3p3:geli_keyfile0″‘ >> /boot/loader.conf echo ‘geli_da3p3_keyfile0_name=”/boot/encryption.key”‘ >> /boot/loader.conf The above settings tell the OS which encryption keyfile to use for each disk partition. Now you can set your root password: passwd root And configure your timezone: tzsetup And setup a dummy /etc/mail/aliases file to prevent sendmail warnings: cd /etc/mail make aliases Now you can configure any additional settings you require (such as adding new users, configuring networking or setting sshd to run on boot) – when you’re done, we need to exit the chroot: exit Now, we need to make sure the bootloader can read our ZFS pool cache (or it wont mount our ZFS disks on boot): cd /boot/zfs cp /boot/zfs/zpool.cache /boot/zfs/zroot/boot/zfs/zpool.cache Finally, we need to unmount all the ZFS filesystems and configure their final mountpoints… zfs unmount -a zfs set mountpoint=legacy zroot zfs set mountpoint=/tmp zroot/tmp zfs set mountpoint=/usr zroot/usr zfs set mountpoint=/var zroot/var zfs set mountpoint=/bootdir bootdir Now we can ‘reboot’ and remove the media while the computer reboots. Do this as soon as you can. The computer should reboot into a ZFS-based filesystem, booted from a software RAID array on fully protected disks with all but /boot partition encrypted. Note: it will ask you to enter a passphrase for each disk parition used above (4 times) – you should take care to enter the correct passwords as it will treat any passwords missed as a failed disk (you get 3 attempts at each password) Once it’s booted, you can login and run sysinstall to configure other options like networking and startup programs (like SSH!) The only point to note is that when you do an OS upgrade, during the “mergemaster” stage, it will complain that /boot is a symlink not a directory. Simply tell it to ignore/do nothing and it will install the files as normal. Enjoy! ZFS boot error: Mounting from zfs:zroot failed with error 2.You have a problem with your zpool.cache. It should be found at /boot/zfs/zpool.cache. If you want to regenerate it you should boot the freebsd live system and choose shell (i am using the memdisk from usb, works fine). Boot from your stick (e.g. ufs:/dev/da0) and do the following (or something equal to that for your system): It may be needed to set the usb stick read-write, so just remount it: mount -o rw / Then: zpool export zroot zpool import -o altroot=/mnt -o cachefile=/tmp/zpool.cache zroot cp /tmp/zpool.cache /mnt/boot/zfs/ In short: create a zpool.cache and copy it to your bootdir. If your bootdir is on a zpool, zpool import bootdir cp /tmp/zpool.cache /bootdir/boot/zfs/ CHECK THE FILE. I had copied it to /bootdir/zfs which is wrong in the setup we did here ;)… ZFS BackupsCreate a new (secure) backup drive. Attach some usb disk. If this is not a zfs backup drive yet, first destroy the old content. Be sure what you are doing! Your data will be lost! gpart destroy -F da0 Then, init the geli crypt geli init -e AES-XTS -l 256 -s 4096 /dev/da0 Attach the geli device geli attach /dev/da0 Create a new zpool… zpool create zbackup /dev/da0.eli List your old snapshots zfs list -t snapshot Destroy your old snapshot zfs destroy -r zroot@oldsnapshot Create a new recursive snapshot zfs snapshot -r zroot@newsnapshot Now, transfer your snapshot to the new disk. zfs send -R zroot@newsnapshot | zfs receive -Fdvu zbackup Export (detach) your backup pool. zpool export zbackup Detach geli. geli detach /dev/da0.eli Next time, we try to do incremental zfs snapshot transfer since last backup ;) ZFS incremental backupImport backup zpool (don't forget -R for alternate root or your root will be over'mounted) zpool import -R /zbackup zbackup Create new snapshot of your root zfs snapshot -r zroot@20130326-2026 Send the diff to the backup! :) zfs send -R -i zroot@20130208-1752 zroot@20130326-2026 | zfs receive -Fuv zbackup (-Fduv?) I love zfs. ISCSI stuff @ freebsdmanual loading as clientkldload /boot/kernel/iscsi_initiator.ko iscontrol -v -d targetaddress=10.0.0.1 initiatorname=`hostname` Edit /etc/iscsi.conf and add some devices which were discovered idisk1 { authmethod = CHAP chapIName = user1 chapSecret = secret123456 initiatorname = server.example.com TargetName = iqn.example.com:idisk1 TargetAddress = 10.0.0.1:3260,1 } starting the client manually, one per diskiscontrol -c /etc/iscsi.conf -n idisk1 iscontrol -c /etc/iscsi.conf -n idisk2 iscontrol -c /etc/iscsi.conf -n idisk3 New devices like /dev/da0, /dev/da1 should appear. stopping the iscontrolKilling the processes with term or kill doesn't seem to work. It should work with kill <pid> -HUP or killall -HUP iscontrol But please, first detach a possible geli and do zfs export. ZFS-geli-ZFSSource of this idea: http://unix.derkeiler.com/Mailing-Lists/FreeBSD/questions/2007-12/msg01469.html Be warned and think about what you are doing here! gpool destroy -F da0 zpool create -o version=28 zbck /dev/da0 zfs create -V 1024g zbck/zsvol with keyfile: geli init -K /root/p_a.key -s 4096 -l 256 /dev/zvol/zbck/zsvol geli attach -k /root/p_a.key /dev/zvol/zbck/zsvol with passphrase: geli init -s 4096 -l 256 /dev/zvol/zbck/zsvol geli attach /dev/zvol/zbck/zsvol zpool create -o version=28 zsbck /dev/zvol/zbck/zsvol.eli Done! |
|||
|
||||