Unterschiede

Hier werden die Unterschiede zwischen zwei Versionen gezeigt.

Link zu dieser Vergleichsansicht

freebsd9_full_disc_encryption_root_zfs [2014/12/29 20:00]
terminar [Installation Part 1.1b: MBR, slices, zfs boot (4K style), used for geli crypted zfs root with seperate zfs boot as slice]
freebsd9_full_disc_encryption_root_zfs [2015/03/12 10:20] (aktuell)
terminar [ZFS incremental backup]
Zeile 1: Zeile 1:
 +
 +
 +====== Full Disk Encryption (with ZFS root) for FreeBSD 9.x ======
 +
 +Boot from freebsd memdisk (usb stick), use "shell".
 +For the german of us, enter
 +  kbdcontrol -l german.iso
 +
 +===== Installation Part 1.1a: MBR, zfs boot (4K style) =====
 +
 +Clean up the disk, maybe use -F
 +  gpart destroy ada0
 +
 +Create new MBR disk
 +  gpart create -s mbr ada0
 +
 +Create first partition (/boot), 504 is 4K aligned
 +  gpart add -b 504 -s 20971520 -t freebsd ada0
 +
 +Create next partition (root)
 +  gpart add -b 20972448 -t freebsd ada0
 +
 +Set active partition
 +  gpart set -a active -i 1 ada0
 +
 +===== Installation Part 1.1b: MBR, slices, zfs boot (4K style), used for geli crypted zfs root with seperate zfs boot as slice =====
 +
 +Clean up the disk, maybe use -F
 +  gpart destroy ada0
 +
 +Create new MBR disk
 +  gpart create -s mbr ada0
 +
 +Create first partition, 504 is 4K aligned
 +  gpart add -b 504 -a 4k -t freebsd ada0
 +  
 +Create BSD type
 +  gpart create -s bsd ada0s1
 +
 +Create first partition slice (/boot)
 +  gpart add -s 10G -a 4k -t freebsd-zfs ada0s1
 +
 +Create root partition slice 
 +  gpart add -a 4k -t freebsd-zfs ada0s1
 +
 +Set first partition as active
 +  gpart set -a active -i 1 ada0
 +
 +Add bootcode to hd
 +  gpart bootcode -b /boot/boot0 ada0
 +
 +Add boot1 stage
 +  dd if=/boot/zfsboot of=/dev/ada0s1 count=1
 +
 +Add boot2 stage
 +  dd if=/boot/zfsboot of=/dev/ada0s1a skip=1 seek=1024
 +
 +
 +===== Installation Part 1.1c: GPT, protected mbr GPT stuff (4K style) =====
 +
 +Use this if your system is not gpt compatible.
 +
 +Clean up the disk, maybe use -F
 +  gpart destroy ada0
 +
 +Create new GPT disk
 +  gpart create -s mbr ada0
 +
 +Create bootcode, compatibility bootcode for mbr->gpt on mbr
 +  gpart bootcode -b /boot/pmgr ada0
 +
 +Math: 40*512b = 20kb + 88*512b = 44kb = 64kb/4kb = 16 => perfect alignment for 4kb drives AND 512b
 +  gpart add -b 40 -s 88 -t freebsd-boot ada0
 +
 +GPT ZFS boot code on p1
 +  gpart add -p /boot/gptzfsboot -i 1 ada0
 +
 +/boot partition, uncrypted, still correct alligned, 10485760kb/4096kb=2560
 +  gpart add -s 10G -t freebsd-zfs ada0
 +
 +And the installation partition, filling all the disk
 +  gpart add -t freebsd-zfs ada0
 +
 +
 +Now, jump to "Part2".
 +
 +===== Installation Part1.2: GPT setup style =====
 +
 +Only use Part1.1 OR Part1.3!
 +
 +The original text is captured from [[https://www.dan.me.uk/blog/2012/05/06/full-disk-encryption-with-zfs-root-for-freebsd-9-x/comment-page-1/]], all credits to him! I made a copy here if the page is down or the link is lost.
 +
 +
 +Be careful, i think the alignment of 4K does not match here!
 +
 +---
 +
 +To follow on from my post about full disk encryption (well almost), this is how to do the same but with a ZFS filesystem.  Like the other post, your /boot folder (which contains your kernel and modules) will not be encrypted, but the rest of your filesystem will be.
 +
 +One disadvantage of this method is that you have to enter a passphrase for EACH disk in your ZFS system each boot.  Encryption inside ZFS will appear at some point – but until then this will suffice !
 +
 +Boot from any FreeBSD 9 install medium (except bootonly), and choose Live CD at the install menu.
 +
 +For the purposes of this article, I will assume that you’re using 4 disks (da0, da1, da2, da3), a 10GB /boot (this will be mirrored on each of the 4 disks), and the remaining space as a raidz1 (roughly similar to RAID5) ZFS filesystem.  The contents will be encrypted with 256-bit AES-XTS encryption with a 4 kb random data partial key and a secondary passphrase (required to type on each boot).
 +
 +If your CPU supports the AESNI flag, the crypto(4) framework will utilise this too.
 +
 +First we need to remove any existing GPT or MBR partition tables on each of the disks (ignore any ‘invalid argument’ messages):
 +
 +    gpart destroy -F da0
 +    gpart destroy -F da1
 +    gpart destroy -F da2
 +    gpart destroy -F da3
 +
 +Now we need to create a new GPT partition table on each disk:
 +
 +    gpart create -s gpt da0
 +    gpart create -s gpt da1
 +    gpart create -s gpt da2
 +    gpart create -s gpt da3
 +
 +We will now create a 64kb boot partition (this contains the boot loader only, so is safe and required to be unencrypted):
 +
 +    gpart add -s 128 -t freebsd-boot da0
 +    gpart add -s 128 -t freebsd-boot da1
 +    gpart add -s 128 -t freebsd-boot da2
 +    gpart add -s 128 -t freebsd-boot da3
 +
 +Next, we will create the /boot partition – you can adjust the sizes here if you need, but i’d suggest not shrinking it too much or you’ll get into problems when doing OS upgrades later… Note: this is mirrored not striped across the disks for maximum resilience – so will use 10GB on each disk for 10GB total usable space.
 +
 +    gpart add -s 10G -t freebsd-zfs da0
 +    gpart add -s 10G -t freebsd-zfs da1
 +    gpart add -s 10G -t freebsd-zfs da2
 +    gpart add -s 10G -t freebsd-zfs da3
 +
 +Finally, we will assign the remaining space on each disk to the root ZFS partition.  This will be encrypted before we build ZFS on top of it.
 +
 +    gpart add -t freebsd-zfs da0
 +    gpart add -t freebsd-zfs da1
 +    gpart add -t freebsd-zfs da2
 +    gpart add -t freebsd-zfs da3
 +
 +Now that we’ve created daXp1 (bootloader), daXp2 (/boot partition), daXp3 (root partition) – we need to write the boot loader code to each disk:
 +
 +    gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0
 +    gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da1
 +    gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da2
 +    gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da3
 +
 +Jump to Part2.
 +
 +===== Installation Part2 =====
 +
 +
 +Ok, next we will build a ramdisk to mount on /boot/zfs – this helps us mount things temporarily…
 +
 +    mdconfig -a -t malloc -s 128m -u 2
 +    newfs -O2 /dev/md2
 +    mount /dev/md2 /boot/zfs
 +
 +Now we will load the modules required for ZFS and encryption:
 +
 +    kldload opensolaris
 +    kldload zfs
 +    kldload geom_eli
 +
 +Next, we build a ZFS mirror for the /boot partition and mount it temporarily (to house the encryption key) – ignore any mention of unable to mount:
 +
 +    zpool create bootdir mirror /dev/da0p2 /dev/da1p2 /dev/da2p2 /dev/da3p2
 +    zpool set bootfs=bootdir bootdir
 +    mkdir /boot/zfs/bootdir
 +    zfs set mountpoint=/boot/zfs/bootdir bootdir
 +    zfs mount bootdir
 +
 +Now we generate a random 4kb encryption key that will form (along with passphrase) the encryption key for the disk:
 +
 +    dd if=/dev/random of=/boot/zfs/bootdir/encryption.key bs=4096 count=1
 +
 +We have everything we need to start encrypting the disks now… Enter your passphrase twice for each init phase and once again for each attach phase below:
 +
 +    geli init -b -B /boot/zfs/bootdir/da0p3.eli -e AES-XTS -K /boot/zfs/bootdir/encryption.key -l 256 -s 4096 /dev/da0p3
 +    geli init -b -B /boot/zfs/bootdir/da1p3.eli -e AES-XTS -K /boot/zfs/bootdir/encryption.key -l 256 -s 4096 /dev/da1p3
 +    geli init -b -B /boot/zfs/bootdir/da2p3.eli -e AES-XTS -K /boot/zfs/bootdir/encryption.key -l 256 -s 4096 /dev/da2p3
 +    geli init -b -B /boot/zfs/bootdir/da3p3.eli -e AES-XTS -K /boot/zfs/bootdir/encryption.key -l 256 -s 4096 /dev/da3p3
 +    geli attach -k /boot/zfs/bootdir/encryption.key /dev/da0p3
 +    geli attach -k /boot/zfs/bootdir/encryption.key /dev/da1p3
 +    geli attach -k /boot/zfs/bootdir/encryption.key /dev/da2p3
 +    geli attach -k /boot/zfs/bootdir/encryption.key /dev/da3p3
 +
 +Now that we have encrypted and mounted the partitions, we can build a ZFS root filesystem on top of it like so:
 +
 +    zpool create zroot raidz1 /dev/da0p3.eli /dev/da1p3.eli /dev/da2p3.eli /dev/da3p3.eli
 +    zfs set mountpoint=/boot/zfs/zroot zroot
 +    zfs mount zroot
 +    zfs unmount bootdir
 +    mkdir /boot/zfs/zroot/bootdir
 +    zfs set mountpoint=/boot/zfs/zroot/bootdir bootdir
 +    zfs mount bootdir
 +
 +Note we unmounted the old boot mirror and re-mounted it within the root filesystem.  This will be used later to copy the kernel and modules into.
 +
 +Ok, now we create all our ZFS mounts with various options as follows:
 +
 +    zfs set checksum=fletcher4 zroot
 +    zfs create -o compression=on -o exec=on -o setuid=off zroot/tmp
 +    chmod 1777 /boot/zfs/zroot/tmp
 +    zfs create zroot/usr
 +    zfs create zroot/usr/home
 +    cd /boot/zfs/zroot; ln -s /usr/home home
 +    zfs create -o compression=lzjb -o setuid=off zroot/usr/ports
 +    zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/distfiles
 +    zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/packages
 +    zfs create zroot/var
 +    zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/crash
 +    zfs create -o exec=off -o setuid=off zroot/var/db
 +    zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/db/pkg
 +    zfs create -o exec=off -o setuid=off zroot/var/empty
 +    zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/log
 +    zfs create -o compression=gzip -o exec=off -o setuid=off zroot/var/mail
 +    zfs create -o exec=off -o setuid=off zroot/var/run
 +    zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/tmp
 +    chmod 1777 /boot/zfs/zroot/var/tmp
 +
 +Now we’re ready to install FreeBSD onto the new ZFS partitions.  We’re going to install the base, all sources and a generic kernel – this takes some time so please be patient…
 +
 +    cd /boot/zfs/zroot
 +    unxz -c /usr/freebsd-dist/base.txz | tar xpf -
 +    unxz -c /usr/freebsd-dist/kernel.txz | tar xpf -
 +    unxz -c /usr/freebsd-dist/src.txz | tar xpf -
 +
 +Now we can set /var/empty to readonly:
 +
 +    zfs set readonly=on zroot/var/empty
 +
 +And now we’re ready to chroot into the installed system to setup the configuration:
 +
 +    chroot /boot/zfs/zroot
 +
 +Now that the base system and kernel are installed, we can move our /boot folder to it’s final place on the ZFS unencrypted mirror and do a little housekeeping:
 +
 +    cd /
 +    mv boot bootdir/
 +    ln -fs bootdir/boot
 +    mv bootdir/encryption.key bootdir/boot/
 +    mv bootdir/*.eli bootdir/boot/
 +
 +We need to setup an initial /etc/rc.conf which will mount all ZFS filesystems on boot:
 +
 +    echo ‘zfs_enable=”YES”‘ > /etc/rc.conf
 +    touch /etc/fstab
 +
 +And an initial /boot/loader.conf that will load ZFS, encryption and settings for encrypted disks on boot:
 +
 +    echo ‘vfs.zfs.prefetch_disable=”1″‘ > /boot/loader.conf
 +    echo ‘vfs.root.mountfrom=”zfs:zroot”‘ >> /boot/loader.conf
 +    echo ‘zfs_load=”YES”‘ >> /boot/loader.conf
 +    echo ‘aesni_load=”YES”‘ >> /boot/loader.conf
 +    echo ‘geom_eli_load=”YES”‘ >> /boot/loader.conf
 +    echo ‘geli_da0p3_keyfile0_load=”YES”‘ >> /boot/loader.conf
 +    echo ‘geli_da0p3_keyfile0_type=”da0p3:geli_keyfile0″‘ >> /boot/loader.conf
 +    echo ‘geli_da0p3_keyfile0_name=”/boot/encryption.key”‘ >> /boot/loader.conf
 +    echo ‘geli_da1p3_keyfile0_load=”YES”‘ >> /boot/loader.conf
 +    echo ‘geli_da1p3_keyfile0_type=”da1p3:geli_keyfile0″‘ >> /boot/loader.conf
 +    echo ‘geli_da1p3_keyfile0_name=”/boot/encryption.key”‘ >> /boot/loader.conf
 +    echo ‘geli_da2p3_keyfile0_load=”YES”‘ >> /boot/loader.conf
 +    echo ‘geli_da2p3_keyfile0_type=”da2p3:geli_keyfile0″‘ >> /boot/loader.conf
 +    echo ‘geli_da2p3_keyfile0_name=”/boot/encryption.key”‘ >> /boot/loader.conf
 +    echo ‘geli_da3p3_keyfile0_load=”YES”‘ >> /boot/loader.conf
 +    echo ‘geli_da3p3_keyfile0_type=”da3p3:geli_keyfile0″‘ >> /boot/loader.conf
 +    echo ‘geli_da3p3_keyfile0_name=”/boot/encryption.key”‘ >> /boot/loader.conf
 +
 +The above settings tell the OS which encryption keyfile to use for each disk partition.
 +
 +Now you can set your root password:
 +
 +    passwd root
 +
 +And configure your timezone:
 +
 +    tzsetup
 +
 +And setup a dummy /etc/mail/aliases file to prevent sendmail warnings:
 +
 +    cd /etc/mail
 +    make aliases
 +
 +Now you can configure any additional settings you require (such as adding new users, configuring networking or setting sshd to run on boot) – when you’re done, we need to exit the chroot:
 +
 +    exit
 +
 +Now, we need to make sure the bootloader can read our ZFS pool cache (or it wont mount our ZFS disks on boot):
 +
 +    cd /boot/zfs
 +    cp /boot/zfs/zpool.cache /boot/zfs/zroot/boot/zfs/zpool.cache
 +
 +Finally, we need to unmount all the ZFS filesystems and configure their final mountpoints…
 +
 +    zfs unmount -a
 +    zfs set mountpoint=legacy zroot
 +    zfs set mountpoint=/tmp zroot/tmp
 +    zfs set mountpoint=/usr zroot/usr
 +    zfs set mountpoint=/var zroot/var
 +    zfs set mountpoint=/bootdir bootdir
 +
 +Now we can ‘reboot’ and remove the media while the computer reboots.  Do this as soon as you can.
 +
 +The computer should reboot into a ZFS-based filesystem, booted from a software RAID array on fully protected disks with all but /boot partition encrypted.  Note: it will ask you to enter a passphrase for each disk parition used above (4 times) – you should take care to enter the correct passwords as it will treat any passwords missed as a failed disk (you get 3 attempts at each password)
 +
 +Once it’s booted, you can login and run sysinstall to configure other options like networking and startup programs (like SSH!)
 +
 +The only point to note is that when you do an OS upgrade, during the “mergemaster” stage, it will complain that /boot is a symlink not a directory.  Simply tell it to ignore/do nothing and it will install the files as normal.
 +
 +Enjoy!
 +
 +
 +===== ZFS boot error: Mounting from zfs:zroot failed with error 2. =====
 +
 +You have a problem with your zpool.cache. It should be found at /boot/zfs/zpool.cache.
 +If you want to regenerate it you should boot the freebsd live system and choose shell (i am using the memdisk from usb, works fine).
 +Boot from your stick (e.g. ufs:/dev/da0) and do the following (or something equal to that for your system):
 +
 +It may be needed to set the usb stick read-write, so just remount it:
 +  mount -o rw /
 +
 +Then:
 +  zpool export zroot
 +  zpool import -o altroot=/mnt -o cachefile=/tmp/zpool.cache zroot
 +  cp /tmp/zpool.cache /mnt/boot/zfs/
 +
 +In short: create a zpool.cache and copy it to your bootdir.
 +If your bootdir is on a zpool, 
 +
 +  zpool import bootdir
 +  cp /tmp/zpool.cache /bootdir/boot/zfs/
 +
 +CHECK THE FILE. I had copied it to /bootdir/zfs which is wrong in the setup we did here ;)...
 +
 +===== ZFS Backups =====
 +
 +Create a new (secure) backup drive. Attach some usb disk.
 +If this is not a zfs backup drive yet, first destroy the old content. Be sure what you are doing! Your data will be lost!
 +
 +  gpart destroy -F da0
 +  
 +Then, init the geli crypt
 +
 +  geli init -e AES-XTS -l 256 -s 4096 /dev/da0
 +
 +Attach the geli device
 +  geli attach /dev/da0
 +
 +Create a new zpool...
 +  zpool create zbackup /dev/da0.eli
 +  
 +List your old snapshots
 +  zfs list -t snapshot
 +
 +Destroy your old snapshot
 +  zfs destroy -r zroot@oldsnapshot
 +
 +Create a new recursive snapshot
 +  zfs snapshot -r zroot@newsnapshot
 +
 +Now, transfer your snapshot to the new disk.
 +  zfs send -R zroot@newsnapshot | zfs receive -Fdvu zbackup
 +
 +Export (detach) your backup pool.
 +  zpool export zbackup
 +  
 +Detach geli.
 +  geli detach /dev/da0.eli
 +
 +Next time, we try to do incremental zfs snapshot transfer since last backup ;)
 +
 +===== ZFS incremental backup =====
 +
 +Import backup zpool (don't forget -R for alternate root or your root will be over'mounted)
 +  zpool import -R /zbackup zbackup
 +
 +Create new snapshot of your root
 +  zfs snapshot -r zroot@20130326-2026
 +
 +Send the diff to the backup! :)
 +  zfs send -R -i zroot@20130208-1752 zroot@20130326-2026 | zfs receive -Fuv zbackup
 +  (-Fduv?)
 +  
 +I love zfs.
 +
 +
 +===== ISCSI stuff @ freebsd =====
 +
 +Source: http://people.freebsd.org/~rse/iscsi/iscsi.txt
 +
 +==== manual loading as client ====
 +
 +  kldload /boot/kernel/iscsi_initiator.ko
 +  iscontrol -v -d targetaddress=10.0.0.1 initiatorname=`hostname`
 +
 +Edit /etc/iscsi.conf and add some devices which were discovered
 +  idisk1 {
 +    authmethod      = CHAP
 +    chapIName       = user1
 +    chapSecret      = secret123456
 +    initiatorname   = server.example.com
 +    TargetName      = iqn.example.com:idisk1
 +    TargetAddress   = 10.0.0.1:3260,1
 +  }
 +  
 +==== starting the client manually, one per disk ====
 +
 +  iscontrol -c /etc/iscsi.conf -n idisk1
 +  iscontrol -c /etc/iscsi.conf -n idisk2
 +  iscontrol -c /etc/iscsi.conf -n idisk3
 +
 +New devices like /dev/da0, /dev/da1 should appear.
 +
 +==== stopping the iscontrol ====
 +Killing the processes with term or kill doesn't seem to work. It should work with
 +  kill <pid> -HUP or
 +  killall -HUP iscontrol
 +
 +But please, first detach a possible geli and do zfs export.
 +
 +
 +=== ZFS-geli-ZFS ===
 +
 +Source of this idea: http://unix.derkeiler.com/Mailing-Lists/FreeBSD/questions/2007-12/msg01469.html
 +
 +Be warned and think about what you are doing here!
 +  gpool destroy -F da0
 +
 +  zpool create -o version=28 zbck /dev/da0
 +  zfs create -V 1024g zbck/zsvol
 +  
 +with keyfile:
 +  geli init -K /root/p_a.key -s 4096 -l 256 /dev/zvol/zbck/zsvol
 +  geli attach -k /root/p_a.key /dev/zvol/zbck/zsvol
 +
 +with passphrase:
 +  geli init -s 4096 -l 256 /dev/zvol/zbck/zsvol
 +  geli attach /dev/zvol/zbck/zsvol
 +
 +  zpool create -o version=28 zsbck /dev/zvol/zbck/zsvol.eli
 +
 +Done!
  
 
 
         
         
   
(29978 views) freebsd9_full_disc_encryption_root_zfs.txt · Zuletzt geändert: 2015/03/12 10:20 von terminar