Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Next revisionBoth sides next revision | ||
| docs:guide-user:additional-software:extroot_configuration [2021/02/20 09:17] – [1. Preparation] vgaetera | docs:guide-user:additional-software:extroot_configuration [2024/04/23 04:42] – Set up /rwm mount point before copying, so it's available from new extroot also - previous order means it's only mounted if extroot fails projectgus | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ====== Extroot configuration ====== | ====== Extroot configuration ====== | ||
| - | {{section> | + | {{section> |
| ===== Introduction ===== | ===== Introduction ===== | ||
| - | This guide describes how to configure OpenWrt to use a storage device (usb or sata or sdcard | + | This guide describes how to configure OpenWrt to use a storage device (USB or SATA or SD card or whatever) to expand your root filesystem, to install freely all the packages you need. |
| - | In most supported devices OpenWrt splits the internal storage into '' | + | In most supported devices OpenWrt splits the internal storage into '' |
| ^ Partition ^ Mount point ^ Compression ^ Writable | | ^ Partition ^ Mount point ^ Compression ^ Writable | | ||
| | '' | | '' | ||
| - | | '' | + | | '' |
| | '' | | '' | ||
| - | This way OpenWrt fits even in tiny amounts of internal storage (as low as 4 MiB), but still allows to write settings and install some packages in the writable partition without changing all linux programs used. | + | This way OpenWrt fits even in tiny amounts of internal storage (as low as 4 MiB), but still allows to write settings and install some packages in the writable partition without changing all Linux programs used. |
| Extroot works by setting another overlay partition in the external storage device, and during boot this new overlay partition will be mounted over the internal storage' | Extroot works by setting another overlay partition in the external storage device, and during boot this new overlay partition will be mounted over the internal storage' | ||
| - | This approach allows easy fallback in case the external storage device is removed, as your device will still have its own overlay partition and thus will load all configuration from there. | + | This approach |
| Which means that it will behave exactly the same as just before you set up extroot. | Which means that it will behave exactly the same as just before you set up extroot. | ||
| + | |||
| + | Note that OpenWrt is known to [[flyspray> | ||
| + | You can work around the issue by using ''/'' | ||
| ===== Instructions ===== | ===== Instructions ===== | ||
| + | The following instructions assume that you already have access to a shell on your OpenWRT device. | ||
| + | Most if not all of these commands can be done via the web interface, however that is emphatically not recommended. | ||
| + | Usually the shell is accessed via [[: | ||
| + | |||
| ==== 1. Preparation ==== | ==== 1. Preparation ==== | ||
| - | Devices | + | Devices |
| - | Remove all packages you have installed to add functionality, | + | Remove all packages you have installed to add secondary |
| - | After you make the extroot you will have all space you need. | + | (If you do not have a record of what these are, try removing ' |
| - | This installs | + | Leave only those needed to access the internet and needed to access the extroot filesystem. |
| + | After you make the extroot you will have all the space you need to install secondary | ||
| - | <code bash> | + | You may not need to make a custom image: try the OEM image first (OpenWRT GL.inet for a GL.inet mango). |
| - | opkg update | + | |
| - | opkg install block-mount kmod-fs-ext4 kmod-usb-storage kmod-usb-ohci kmod-usb-uhci e2fsprogs fdisk | + | |
| - | </ | + | |
| - | Risk-adverse users may wish to create | + | The extroot can be anything that '' |
| + | Currently '' | ||
| + | It must a [[commit>? | ||
| + | For most, this filesystem will be a on USB storage device. | ||
| + | However, it could also be on an SD-Card or a SATA drive connected via e-sata or even a network block device | ||
| + | If you're using a USB connected device follow | ||
| - | ==== 2. Configuring rootfs_data ==== | + | The following assumes that you will be creating your extroot as an EXT4 filesystem on your OpenWRT device with a connected USB flash drive. |
| - | Connect with ssh to the device. | + | The process is similar for other kinds of devices. |
| - | Configure ''/ | + | |
| + | Installing these packages requires a sensible amount of extra filespace. If you completely fill the filesystem by installing these, | ||
| + | you will probably have to re-flash | ||
| + | remove some installed packages first. Good candidates for removal are ntfs3 and ntfs3-utils: you can re-install them later after | ||
| + | you have extroot installed. | ||
| + | |||
| + | Install the required packages. | ||
| <code bash> | <code bash> | ||
| - | DEVICE=" | + | opkg update |
| - | uci -q delete fstab.rwm | + | opkg install block-mount kmod-fs-ext4 e2fsprogs parted kmod-usb-storage |
| - | uci set fstab.rwm=" | + | |
| - | uci set fstab.rwm.device=" | + | |
| - | uci set fstab.rwm.target="/ | + | |
| - | uci commit fstab | + | |
| </ | </ | ||
| - | Or you can identify | + | Identify |
| <code bash> | <code bash> | ||
| - | grep -e rootfs_data | + | ls -l /sys/block |
| </ | </ | ||
| - | The directory ''/ | + | ==== 2. Partitioning and formatting |
| - | Later you can edit ''/ | + | Partition and format |
| - | + | ||
| - | ==== 3. Configuring extroot | + | |
| - | See what partitions you have using the following command: | + | |
| - | + | ||
| - | <code bash> | + | |
| - | + | ||
| - | You will see similar output: | + | |
| <code bash> | <code bash> | ||
| - | / | + | DISK="/ |
| - | /dev/mtdblock3: MOUNT="/ | + | parted |
| - | /dev/sda1: UUID=" | + | DEVICE="${DISK}1" |
| + | mkfs.ext4 -L extroot ${DEVICE} | ||
| </ | </ | ||
| - | Here we see '' | + | This will wipe all data on the disk, so do not run these commands blindly. |
| - | Format | + | ==== 3. Configuring extroot ==== |
| + | Configure | ||
| <code bash> | <code bash> | ||
| - | mkfs.ext4 | + | eval $(block info ${DEVICE} | grep -o -e ' |
| + | eval $(block info | grep -o -e ' | ||
| + | uci -q delete fstab.extroot | ||
| + | uci set fstab.extroot=" | ||
| + | uci set fstab.extroot.uuid=" | ||
| + | uci set fstab.extroot.target=" | ||
| + | uci commit fstab | ||
| </ | </ | ||
| - | Now we configure '' | + | ==== 4. Configuring rootfs_data |
| + | Configure a mount entry for the the original | ||
| <code bash> | <code bash> | ||
| - | DEVICE="/ | + | ORIG=" |
| - | eval $(block info " | + | uci -q delete fstab.rwm |
| - | uci -q delete fstab.overlay | + | uci set fstab.rwm=" |
| - | uci set fstab.overlay=" | + | uci set fstab.rwm.device="${ORIG}" |
| - | uci set fstab.overlay.uuid="${UUID}" | + | uci set fstab.rwm.target="/ |
| - | uci set fstab.overlay.target="/ | + | |
| uci commit fstab | uci commit fstab | ||
| </ | </ | ||
| - | If you have a swap partition | + | This will allow you to access the '' |
| - | ==== 4. Transferring data ==== | + | ==== 5. Transferring data ==== |
| - | We now transfer | + | Transfer |
| <code bash> | <code bash> | ||
| - | mount / | + | mount ${DEVICE} |
| - | cp -f -a /overlay/. /mnt | + | tar -C ${MOUNT} -cvf - . | tar -C /mnt -xf - |
| - | umount /mnt | + | |
| </ | </ | ||
| - | Reboot the device: | + | ==== 6. Apply changes ==== |
| + | Reboot the device | ||
| <code bash> | <code bash> | ||
| Line 104: | Line 117: | ||
| ===== Testing ===== | ===== Testing ===== | ||
| - | ==== Web interface ==== | + | ==== Web interface |
| - | - **[[http:// | + | - **LuCI -> System -> Mount Points** should show USB partition mounted as '' |
| - | - **[[http:// | + | - **LuCI -> System -> Software** should show free space of overlay partition. |
| - | ==== Command-line | + | ==== Command-line |
| The USB partition should be mounted to ''/ | The USB partition should be mounted to ''/ | ||
| Free space for ''/'' | Free space for ''/'' | ||
| Line 127: | Line 140: | ||
| <code bash> | <code bash> | ||
| - | logread | sed -n -e "/- preinit -/,/- init -/p" | + | block info; uci show fstab; |
| </ | </ | ||
| Line 139: | Line 152: | ||
| * Do not use vfat (FAT/ | * Do not use vfat (FAT/ | ||
| - | * If the partition containing your extroot isn't mounted during boot, but you can mount it without problems from a shell, you should try to increase '' | + | * If the partition containing your extroot isn't mounted during boot, but you can mount it without problems from a shell, you should try to increase '' |
| <code bash> | <code bash> | ||
| Line 173: | Line 186: | ||
| This makes package lists survive reboot and saves some RAM. | This makes package lists survive reboot and saves some RAM. | ||
| - | === Web interface === | + | === Web interface |
| - | - **[[http:// | + | - Navigate to **LuCI -> System -> Software -> Configuration** to change ''/ |
| - | - **[[http:// | + | - Navigate to **LuCI -> System -> Software -> Actions -> Update lists** to do an initial build of the package list onto extroot. |
| - | === Command-line | + | === Command-line |
| <code bash> | <code bash> | ||
| sed -i -e "/ | sed -i -e "/ | ||
| Line 188: | Line 201: | ||
| <code bash> | <code bash> | ||
| # Create swap file | # Create swap file | ||
| - | dd if=/ | + | DIR=" |
| - | mkswap | + | dd if=/ |
| + | mkswap | ||
| # Enable swap file | # Enable swap file | ||
| uci -q delete fstab.swap | uci -q delete fstab.swap | ||
| uci set fstab.swap=" | uci set fstab.swap=" | ||
| - | uci set fstab.swap.device=" | + | uci set fstab.swap.device=" |
| uci commit fstab | uci commit fstab | ||
| - | / | + | service |
| # Verify swap status | # Verify swap status | ||
| Line 265: | Line 279: | ||
| ==== Remote file system ==== | ==== Remote file system ==== | ||
| Follow: [[https:// | Follow: [[https:// | ||
| + | |||
| + | ==== LUKS encrypted extroot ==== | ||
| + | You may wish to have your extroot filesystem in a LUKS encrypted container. | ||
| + | As of OpenWrt 22.03.2, this [[https:// | ||
| + | OpenWrt does not have an official way to open encrypted LUKS volumes before the extroot check happens during the normal boot path. | ||
| + | So at the time of extroot check time, the extroot filesystem will not be visible and the boot process will continue as if there is not extroot. | ||
| + | Below are two different methods for getting the system to run on an encrypted extroot. | ||
| + | The first method is preferable because there are less side-effects and is a cleaner approach. | ||
| + | |||
| + | Before doing any of the below, you'll need to create the LUKS container in which to put your extroot filesystem. | ||
| + | Follow the [[docs: | ||
| + | You will need enough space on your '' | ||
| + | Once, you have your LUKS container follow the [[# | ||
| + | |||
| + | === PREINIT === | ||
| + | In the PREINIT phase of boot, '' | ||
| + | Using a stock OpenWRT firmware, there will be no extroot config on the ROM device. | ||
| + | To check the '' | ||
| + | If found, it will search in order ''/ | ||
| + | This is why the '' | ||
| + | |||
| + | Because '' | ||
| + | This is done by creating a script to replace the '' | ||
| + | The script below does this while relying on the '' | ||
| + | So run '' | ||
| + | To install the block script, move the binary at ''/ | ||
| + | |||
| + | The '' | ||
| + | So if this file does not exist, the extroot will not be configured. | ||
| + | This allows one to easily disable setting up the crypto disk early on, thus effectively disabling extroot. | ||
| + | Once on the extroot, to access this file you'll need to mount the '' | ||
| + | |||
| + | Once the '' | ||
| + | |||
| + | <code bash block> | ||
| + | #!/bin/sh | ||
| + | |||
| + | # Prereqs: | ||
| + | # * packages: | ||
| + | # * block-mount | ||
| + | # * cryptsetup | ||
| + | # * move /sbin/block to / | ||
| + | # * install decrypt script to / | ||
| + | # | ||
| + | # This script should be placed at / | ||
| + | # or /sbin/block if already on the overlayfs and be set with execute | ||
| + | # permission. | ||
| + | # It is expected that the extroot is on a device that the kernel names as | ||
| + | # sd* or mmcblk*, otherwise modify appropriately. | ||
| + | |||
| + | # Set to 1 to enable debug logs | ||
| + | export DEBUG= | ||
| + | |||
| + | SDIR=${0%/ | ||
| + | BLOCK=" | ||
| + | LD_LIBRARY_PATH=${LD_LIBRARY_PATH: | ||
| + | LD_LIBRARY_PATH=" | ||
| + | PATH=$PATH: | ||
| + | |||
| + | block() { | ||
| + | ( exec -a ${0} ${BLOCK} " | ||
| + | } | ||
| + | |||
| + | if [ " | ||
| + | exec block " | ||
| + | fi | ||
| + | |||
| + | get_jiffies() { | ||
| + | head -n3 / | ||
| + | } | ||
| + | |||
| + | if [ -z " | ||
| + | TIME=$(get_jiffies) | ||
| + | export BLOCK_LOG="/ | ||
| + | exec 2>" | ||
| + | set -x | ||
| + | fi | ||
| + | |||
| + | if [ ! -x " | ||
| + | echo " | ||
| + | return 1 | ||
| + | fi | ||
| + | |||
| + | if [ " | ||
| + | # We are being called to setup the extroot, so make sure crypto block | ||
| + | # devices are all setup. | ||
| + | |||
| + | # Hotplug runs too late, create device nodes for /dev/sd*, if there are any | ||
| + | for SYSDEVPATH in / | ||
| + | [ ! -f " | ||
| + | [ -e "/ | ||
| + | MAJMIN=$(cat " | ||
| + | mknod / | ||
| + | done | ||
| + | |||
| + | # Load modules needed for cryptsetup | ||
| + | KVER=$(uname -r) | ||
| + | insmod ${SDIR}/ | ||
| + | insmod ${SDIR}/ | ||
| + | insmod ${SDIR}/ | ||
| + | insmod ${SDIR}/ | ||
| + | |||
| + | # FIXME: Why does block info only show ubi devices? | ||
| + | # block info | cut -d: -f1 | | ||
| + | # Do this hack instead, only check scsi and mmc devices | ||
| + | find /dev -type b | grep -E "/ | ||
| + | while read DEVPATH; do | ||
| + | cryptsetup --disable-locks isLuks $DEVPATH || continue | ||
| + | export ACTION=add DEVNAME=" | ||
| + | # Assume this script is located in $OVERLAY/ | ||
| + | ALTROOT=" | ||
| + | done | ||
| + | fi | ||
| + | |||
| + | block " | ||
| + | </ | ||
| + | |||
| + | === / | ||
| + | There is another way to work around the current limitations. | ||
| + | **However, it should only be used if the above method does not work for your setup, it is more prone to breaking or having strange side effects**. | ||
| + | The basic idea is that extroot will be setup as in the [[# | ||
| + | This will be expected. | ||
| + | Modifications to ''/ | ||
| + | |||
| + | So at this point your uci fstab configuration should have a mount section with target ''/ | ||
| + | I use the '' | ||
| + | Here's a relevant snippet of script that illustrates what needs to be put into ''/ | ||
| + | Currently this script will not work for LUKS volumes being opened with a password. | ||
| + | The volume must be opened with a keyfile (stdin is not properly setup in ''/ | ||
| + | In the script below, the key is stored at ''/ | ||
| + | Check your threat model to see if this works for you. | ||
| + | |||
| + | <code bash> | ||
| + | # Only setup the encrypted extroot if / | ||
| + | # This makes it easier disable the encrypted extroot from failsafe mode. | ||
| + | mkdir -p /mnt/tmp | ||
| + | if [ -e / | ||
| + | # Setup crypt device which contains the extroot | ||
| + | cryptsetup open -d / | ||
| + | umount /overlay | ||
| + | |||
| + | # /tmp will get overridden by another tmpfs by mount_root, but we need the | ||
| + | # initial one because it contains the ubus named socket. | ||
| + | mount --bind /tmp /mnt/tmp | ||
| + | |||
| + | # Re-run mount_root now that we have a block device that it will recognize | ||
| + | # as an extroot. This sleep is needed, otherwise procd seems to freak out | ||
| + | # and the watchdog timer doesn' | ||
| + | sleep 5 | ||
| + | PREINIT=1 mount_root | ||
| + | |||
| + | # Free the new tmpfs just created by mount_root. Since it will never be used, | ||
| + | # its just wasting memory. | ||
| + | umount -l /tmp | ||
| + | |||
| + | # Put the original tmpfs back to where it was in the VFS, primarily so that | ||
| + | # programs can find the ubus socket. | ||
| + | mount --bind / | ||
| + | |||
| + | # Need to re-run this too for some reason, otherwise some other mounts are not | ||
| + | # mounted after mount_root, eg. /rwm. | ||
| + | block mount | ||
| + | |||
| + | # Reload rpcd to register rpc objects on the extroot | ||
| + | service rpcd reload | ||
| + | fi | ||
| + | </ | ||
| + | |||
| + | **NOTE:** Since this method is essentially redoing some of the boot process, it does take longer. | ||
| + | On my device, its about 20-30 seconds longer for the web interface to be available. | ||
| + | Logging in via SSH is not delayed though. | ||
| ==== System upgrade ==== | ==== System upgrade ==== | ||
| Line 275: | Line 460: | ||
| * Finally, if you upgrade all packages but the kernel and the kernel modules, some packages like '' | * Finally, if you upgrade all packages but the kernel and the kernel modules, some packages like '' | ||
| - | ==== Small flash ==== | + | ==== Custom image ==== |
| - | This method is useful for devices | + | This method is useful for devices |
| In the default OpenWrt firmware images there are no tools to make extroot, as the build system currently makes only barebone images. | In the default OpenWrt firmware images there are no tools to make extroot, as the build system currently makes only barebone images. | ||
| The only way to go for these devices is to rebuild a firmware image with the right packages using the Image Builder. | The only way to go for these devices is to rebuild a firmware image with the right packages using the Image Builder. | ||
| Line 308: | Line 493: | ||
| This will build a firmware image that is able to read a partition formatted with ext4 filesystem. | This will build a firmware image that is able to read a partition formatted with ext4 filesystem. | ||
| - | |||
| - | <WRAP center round todo 90%> | ||
| Sadly the package **e2fsprogs** with the tools for ext4 filesystem is too large to fit in 4 MiB devices. | Sadly the package **e2fsprogs** with the tools for ext4 filesystem is too large to fit in 4 MiB devices. | ||
| - | If someone can split the mke2fs and the filesystem checker tool from it, we can adjust this tutorial to use ext4. | ||
| - | </ | ||
| Afterwards, open the folder **bin** inside the Image Builder folder, then open the **target** folder, then the folder you find in it (it has a device-type-specific name), and then inside a folder called **generic** and you should reach the flashable images. | Afterwards, open the folder **bin** inside the Image Builder folder, then open the **target** folder, then the folder you find in it (it has a device-type-specific name), and then inside a folder called **generic** and you should reach the flashable images. | ||
| Line 318: | Line 499: | ||
| Then you will have to format the USB drive with ext4 filesystem, and to do that you will need to use a Linux LiveCD or [[https:// | Then you will have to format the USB drive with ext4 filesystem, and to do that you will need to use a Linux LiveCD or [[https:// | ||
| - | Sadly this is inconvenient but as said above we cannot fit formatting tools in devices with 4MB of flash | + | Sadly this is inconvenient but as said above we cannot fit formatting tools in devices with 4MB of flash. |
| ==== Automated setup ==== | ==== Automated setup ==== | ||
| - | You can use an [[docs: | + | You can use the [[docs: |
| + | |||
| + | ==== Automated upgrade ==== | ||
| + | Set up [[docs: | ||
| + | Packages required by Extroot | ||
| + | |||
| + | <code bash> | ||
| + | cat << " | ||
| + | if uci -q get fstab.extroot > /dev/null \ | ||
| + | && [ ! -e / | ||
| + | && [ -e / | ||
| + | && lock -n / | ||
| + | then | ||
| + | UUID=" | ||
| + | DIR=" | ||
| + | DEV=" | ||
| + | if touch / | ||
| + | && grep -q -e " | ||
| + | && mount " | ||
| + | then | ||
| + | BAK=" | ||
| + | mv -f /mnt/etc /mnt/upper " | ||
| + | cp -f -a " | ||
| + | umount " | ||
| + | fi | ||
| + | lock -u / | ||
| + | reboot | ||
| + | fi | ||
| + | exit 1 | ||
| + | EOF | ||
| + | cat << " | ||
| + | / | ||
| + | EOF | ||
| + | </ | ||