Sysupgrade – Technical Reference

In contrast to opkg, mtd and others, sysupgrade is merely a shell script: /sbin/sysupgrade intended to facilitate easy updates.

This page lists all sysupgrade command-line options. For the overall upgrade procedure and typical usage, please read OpenWrt OS upgrade procedure (sysupgrade or LuCI) instead.

sysupgrade supports the following options (r34975):

Usage: /sbin/sysupgrade [<upgrade-option>...] <image file or URL>
       /sbin/sysupgrade [-q] [-i] <backup-command> <file>

upgrade-option:
        -d <delay>   add a delay before rebooting
        -f <config>  restore configuration from .tar.gz (file or url)
        -i           interactive mode
        -c           attempt to preserve all changed files in /etc/
        -n           do not save configuration over reflash
        -p           do not attempt to restore the partition table after flash.
        -T | --test
                     Verify image and config .tar.gz but do not actually flash.
        -F | --force
                     Flash image even if image checks fail, this is dangerous!
        -q           less verbose
        -v           more verbose
        -h | --help  display this help

backup-command:
        -b | --create-backup <file>
                     create .tar.gz of files specified in sysupgrade.conf
                     then exit. Does not flash an image. If file is '-',
                     i.e. stdout, verbosity is set to 0 (i.e. quiet).
        -r | --restore-backup <file>
                     restore a .tar.gz created with sysupgrade -b
                     then exit. Does not flash an image. If file is '-',
                     the archive is read from stdin.
        -l | --list-backup
                     list the files that would be backed up when calling
                     sysupgrade -b. Does not create a backup file.

WARNING: Preserving files across sysupgrades can be fatal (see 'NOTE: ...') on systems with weak cpu and exceptionally large rootfs_data partitions.

Files to be preserved depend on the following:

• /etc/sysupgrade.conf - customizable backup configuration.
• /lib/upgrade/keep.d/* - system configurations provided by specific packages preserved by default.
• opkg list-changed-conffiles - list of files derived by package manager.

SAVE_OVERLAY saves the complete /overlay directory. There's no rule what needs to be saved... sometimes I install additional packages but then include them in the later firmware builds, so I don't have to preserve those. Sometimes files are modified and the new firmware already contains the modifications so those are also not needed. But config files are needed or router will boot with default settings. But those are preserved by default (unless using -n).

-- Does this mean, I make an archive.tar.gz of /etc and /root for example and sysupgrade -f archive.tar.gz will flash the router and afterwards restores the configs from this archive?

That's what is says: 'restore configuration from .tar.gz (file or url)'. Anything archived in the tgz will be written to /overlay after the flash. This way you can hand-pick the files that will be the system after new firmware boot.

The sysupgrade process works roughly like this, starting from the execution of /sbin/sysupgrade:

1. Locate the file to upgrade from on the filesystem, or if the second option to sysupgrade starts with http://, download the firmware file using wget.
2. Do some minor validation of various things, and grab whatever config files the end user wants to be restored and packs them up into a tarball.
3. Send a message, via ubus, to procd, to initiate the upgrade.
4. procd does some stuff which I haven't finished completely understanding just yet, but it looks like firmware verification to make sure we don't upgrade to a bad firmware file.
5. Notably, procd does not terminate any services here.
6. procd replaces itself (via execvp) with the program /sbin/upgraded. Service management is no longer possible.
7. /sbin/upgraded now acts as PID1. It executes the shell script /lib/upgrade/stage2 with parameters. The remaining sequence runs from this shell script.
8. Terminate (SIGKILL) all telnet, ash, and dropbear processes.
9. Loop over all remaining processes, sending them the TERM signal.
10. Loop over all remaining processes, sending them the KILL signal.
11. Create a new RAM filesystem, mount it, and copy over a very small set of binaries into it.
12. Change root into the new RAM filesystem.
13. Remount /overlay read-only, and lazy-unmount it.
14. Write the upgraded firmware to disk. (This can include platform/device-specific upgrade steps in platform_do_upgrade, handling mtd or partitioning details.)
15. Write the saved configuration (if any) to disk. (This is a platform-specific step, via platform_copy_config.)
16. Unmount any remaining filesystems.
17. Reboot.

There are plenty of potential deficiencies in this process, among them:

• Hardcodes a list of “potentially-interfering” / interactive processes (ash, telnet, dropbear) to kill first; this is not exhaustive or up-to-date (e.g., telnet is no longer in the base install; openssh is not handled).
• Does not give processes much time in between TERM and KILL signals.
• Does not utilise procd to tear down services.
• Susceptible to fork bombs.
• Easily broken by open files on storage devices (e.g., for swap, or explicitly opened), as these can prevent unmounting /overlay.
• Does not handle errors (e.g., I/O) in writing the disk image, potentially leaving a partially-upgraded system upon reboot.

Many of these deficiencies are historical artifacts, remaining simply because no one has fixed them.

Thanks to Michael Jones for writing most of this down on the mailing list [OpenWrt-Devel] Sysupgrade and Failed to kill all processes.