owut: OpenWrt Upgrade Tool

For a more detailed description of the overall process, see the Attended Sysupgrade page.

owut is a command line upgrade tool that creates and installs custom builds of OpenWrt containing all of your currently installed packages and retaining your configuration.

Its subcommands and options allow you to

• check latest build status
• list all the OpenWrt versions available on the upgrade server
• download an image without installing it
• verify and install an image you downloaded
• generate lists of installed package for use with Firmware Selector or source builds
• when generating an image you can
  • add packages to or remove packages from the build list
  • include a custom uci-defaults script
  • specify a non-standard root filesystem size
  • change the file system type at upgrade time
• and more.

You can specify these options on the command line, or you can use the standard OpenWrt config system to store these values and avoid having to remember and retype them on every upgrade.

owut is a standard, optional OpenWrt package, available on all platforms supported by SNAPSHOT or release builds from 24.x and later.

opkg update  &&  opkg install owut

If you find that you need a feature that is not in your current version of owut, you can upgrade as follows.

opkg update  &&  opkg upgrade owut

There is usually no need to specifically upgrade owut, as once installed owut will be upgraded along with everything else whenever you do a full firmware upgrade.

If you have questions about installation or configuration, post on the support forum thread.

If your goal is simply to upgrade your router's current firmware, staying on the same version (e.g., 23.05 or SNAPSHOT), then just use the upgrade command. If there are any problems uncovered by the various pre-build checks, or if an error is detected during the build, then the upgrade will abort with a message indicating the issue.

If owut finds that package downgrades, or no changes were made since your last upgrade, it will tell you this and stop. You can re-run the command with the --force option, which will proceed with the build and install, keeping all configuration.

Note that owut is currently available only on SNAPSHOT or versions 24.x and later.

$ opkg update && opkg install owut
$ owut upgrade
Board-name     generic
Target         x86/64
Root-FS-type   squashfs
Sys-type       combined-efi
Package-arch   x86_64
Version-from   SNAPSHOT r26670-5f9fb964c3 (kernel 6.6.33)
Version-to     SNAPSHOT r26713-68f7ca23fb (kernel 6.6.33)
Build-FS-type  squashfs
Build-at       2024-06-19T17:36:09Z
Image-prefix   openwrt-x86-64-generic
Image-file     openwrt-x86-64-generic-squashfs-combined-efi.img.gz
Image-URL      https://downloads.openwrt.org/snapshots/targets/x86/64/openwrt-x86-64-generic-squashfs-combined-efi.img.gz
Installed      272 packages
Top-level       82 packages
Default         46 packages
User-installed  51 packages (top-level only)

Package version changes:
  base-files                   1603~5f9fb964c3                            1605~68f7ca23fb
1 packages are out-of-date.

Default package analysis:
  Default            Provided-by
  dnsmasq            not installed, possibly replaced by another package
  kmod-dwmac-intel   not installed, possibly replaced by another package
  nftables           not installed, possibly replaced by another package

There are currently package build failures for SNAPSHOT x86_64:
  Feed: packages
    basicstation               Wed Jun 19 08:07:53 2024 - not installed
    xtables-addons             Wed Jun 19 09:39:10 2024 - not installed
  Feed: telephony
    freeswitch                 Wed Jun 19 11:30:36 2024 - not installed
Failures don't affect this device, details at
  https://downloads.openwrt.org/snapshots/faillogs/x86_64/

Build: Requesting version r26713-68f7ca23fb (kernel 6.6.33)

Requesting build ----------------------
Hash:   00b63d9130a1df0a3af34deb5d9a17ff
Status: 202
Detail: queued - 0 ahead of you

Progress   1 (  11s) -----------------------------
Status: 202
Detail: started - calculate_packages_hash

Progress   2 (  22s) -----------------------------
Status: 202
Detail: started - calculate_packages_hash

Progress   3 (  33s) -----------------------------
Status: 202
Detail: started - building_image

...

Progress  14 ( 154s) -----------------------------
Status: 202
Detail: started - building_image

Progress  15 ( 165s) -----------------------------
Status: 200
Detail: done

Build completed in 167 seconds.
  rootfs_size_mb = default
  init-script    = no-init-script

Image source: https://sysupgrade.openwrt.org/store/00b63d9130a1df0a3af34deb5d9a17ff/openwrt-5c211e880ca6-x86-64-generic-squashfs-combined-efi.img.gz
Image saved : /tmp/firmware.bin
Manifest    : /tmp/firmware-manifest.json
Verifying   : /tmp/firmware.bin (24812785 bytes) against /tmp/firmware.sha256sums
  Saved sha256 matches
  Wed Jun 19 18:28:38 PDT 2024 upgrade: Image metadata not present
  Wed Jun 19 18:28:38 PDT 2024 upgrade: Reading partition table from bootdisk...
  Wed Jun 19 18:28:39 PDT 2024 upgrade: Extract boot sector from the image
  Wed Jun 19 18:28:39 PDT 2024 upgrade: Reading partition table from image...
  Wed Jun 19 18:28:39 PDT 2024 upgrade: Partition layout has changed. Full image will be written.
Checks complete, image is valid.
Installing /tmp/firmware.bin and rebooting...

... system reboots ...

$ owut -h
owut - OpenWrt Upgrade Tool %%VERSION%% (/root/bin/owut)
 
owut is an upgrade tool for OpenWrt.
 
Usage: owut COMMAND [-V VERSION_TO] [-v] [-k] [--force] [-a ADD] [-r REMOVE] [-I INIT_SCRIPT] [-F FSTYPE] [-S ROOTFS_SIZE] [-i IMAGE] [-f FORMAT] [-p PRE_INSTALL]
  -h/--help            - Show this message and quit.
  --version            - Show the program version and terminate.
 
  COMMAND - Sub-command to execute, must be one of:
    check    - Collect all resources and report stats.
    list     - Show all the packages installed by user.
    blob     - Display the json blob for the ASU build request.
    download - Build, download and verify an image.
    verify   - Verify the downloaded image.
    install  - Install the specified local image.
    upgrade  - Build, download, verify and install an image.
    versions - Show available versions.
    dump     - Collect all resources and dump internal data structures.
 
  -V/--version-to VERSION_TO - Specify the target version, defaults to installed version.
  -v/--verbose         - Print various diagnostics.  Repeat for even more output.
  -k/--keep            - Save all downloaded working files.
  --force              - Force a build even when there are downgrades or no changes.
  -a/--add ADD         - Comma-separated list of new packages to add to build list.
  -r/--remove REMOVE   - Comma-separated list of installed packages to remove from build list.
  -I/--init-script INIT_SCRIPT - Path to uci-defaults script to run on first boot ('-' use stdin).
  -F/--fstype FSTYPE   - Desired root file system type (squashfs, ext4, ubifs, jffs2).
  -S/--rootfs-size ROOTFS_SIZE - Root file system size in MB (1-1024).
  -i/--image IMAGE     - Image name for download, verify, install and upgrade.
  -f/--format FORMAT   - Format for 'list' output (fs-user, fs-all, config).
  -p/--pre-install PRE_INSTALL - Script to exec just prior to launching final sysupgrade.
 
Full documentation
  https://openwrt.org/docs/guide-user/installation/sysupgrade.owut
 
Questions and discussion
  https://forum.openwrt.org/t/owut-openwrt-upgrade-tool/200035
 
Issues and bug reports
  https://github.com/efahl/owut/issues

When you do not explicitly specify a --version-to value, owut looks for the newest version of the installed branch and sets that as the target version.

For all examples, assume that latest on the 21.02 branch is the 21.02.7 release, and on 23.05 it's the 23.05.4 release.

The exception to this is if the installed version is a SNAPSHOT, either release or main, in which case, the version-to target remains at the installed version.

When you do specify --version-to, it must name a valid version or branch. If you provide an invalid value, owut will show you all the available versions (or you can do this manually with owut versions).

When you specify a full version, then the input is checked against the available versions and left untouched:

If you specify only a branch (i.e., a version number without the final “dot value”), then the version-to target is set to the latest release on that branch.

Note that character case is not important in naming the version, owut converts internally to what the upgrade server requires. You can say snapshot, SnapShot or SNAPSHOT or rc1 or rC1 and owut knows what to do.

owut's normal behavior is to avoid doing unneeded work by stopping a build request when no changes are found. The --force option is used to override this and do a re-build and re-install of the current system. This might be useful if you have inadvertently deleted packages or something similar, and can't easily figure out how to recover.

When owut detects downgrades in packages, it will indicate this by coloring the new version number red in the Package version changes: list and report the number of downgrades at the bottom of the list. This also causes owut to stop processing any download or upgrade in progress, unless you specify --force option.

The --add and --remove options allow you to add packages to or remove packages from the build list submitted by owut to the ASU build server.

For example, if you wanted to upgrade and simultaneously switch to the full versions of dnsmasq and tc, you'd say this.

$ owut upgrade -r dnsmasq,tc-tiny -a dnsmasq-full,tc-full
...

1. If you add a package that is already installed, owut silently ignores this and carries on.
2. If you add a package that doesn't exist, hasn't been ported to your device or is currently unavailable, then owut reports an error and stops.

1. If you attempt to remove a package this is not installed, owut reports the error and stops.
2. If you remove one of the default packages for your device, owut will produce a warning, the package will be removed and owut will continue with your request. Note that this may break things, either during the build or after installing on your device, and you are responsible for ensuring that removal of the package is appropriate.

   $ owut check -r kmod-igb
   WARNING: package 'kmod-igb' is a default package, removal may have unknown side effects
   ...

3. If you attempt to remove a package that has dependents (that is a “non-top-level package” -- something that was installed as a requirement for another package), owut will warn you about this, the package will be removed from the build list and owut will proceed. Note that this typically has no effect, as the package will be pulled back in by the same top-level package that installed it in the first place.

   $ owut check -r kernel
   WARNING: package 'kernel' has dependents and removal will have no effect on the build
   ...

If you are not familiar with uci-defaults (aka “first boot scripts”), you can read up here: UCI defaults. The underlying mechanism that implements owut (and auc and LuCI Attended Sysupgrade and Firmware Selector) builds is the Image Builder, so its description may also be useful: Image builder - Custom files.

The --init-script option allows you to specify the name of a file containing a uci-defaults script, which is to be executed at first boot. The ASU server takes your init-script source and places it in the image in /etc/uci-defaults/99-asu-defaults (there is no means to change this name). On immutable file systems (say squashfs), this also results in the file being stored in /rom/etc/uci-defaults/99-asu-defaults, which comes into play with LuCI Attended Sysupgrade.

Here's a comparison of owut with how other upgrade tools implement this functionality.

• auc does not implement this ability (but the patch exists).
• Firmware Selector behaves identically to owut with its Script to run on first boot (uci-defaults) input field.
• LuCI Attended Sysupgrade implements this by looking for /rom/etc/uci-defaults/99-asu-defaults and then relays the contents of that file implicitly. LuCI ASU's shortcoming is that it doesn't allow you to delete or change what's already there. owut makes this explicit, if you want the script included in your new image, then you must specify it when you request a build.

The terms used in the following two scenarios (comprising seven use cases) are:

• “mutable system” - A device using a read-write file system for its main storage. Typically in OpenWrt, this is an ext4 file system.

  $ mount | grep '(ro'
  ... no output ...

• “immutable system” - A device which has a read-only partition mounted on /rom, where the original system image is stored. Usually, but not always, this is a squashfs file system.

  $ mount | grep '(ro'
  /dev/root on /rom type squashfs (ro,relatime,errors=continue)

• “asu-defaults” - The file /rom/etc/uci-defaults/99-asu-defaults, which only exists on devices with an immutable file system and whose images have been built by the ASU build server.

On an immutable system, there are five cases to consider:

1. An asu-defaults file does not exist and you don't need or want one:

   owut upgrade

2. An asu-defaults file exists and you want to keep it unchanged:

   owut upgrade --init-script /rom/etc/uci-defaults/99-asu-defaults

3. An asu-defaults file exists and you want to modify it:

   cp /rom/etc/uci-defaults/99-asu-defaults my-modified-init-script.sh
   vi my-modified-init-script.sh  # Change it.
   owut upgrade --init-script my-modified-init-script.sh

4. An asu-defaults file does not exist but you want to add one:

   vi my-new-init-script.sh
   owut upgrade --init-script my-new-init-script.sh

5. An asu-defaults file exists and you want to delete it from build:

   owut upgrade  # Just ignore the warning.

On a mutable system, since there is no /rom/etc/uci-defaults, you only have two choices:

1. You don't want to create an asu-defaults file in your build:

   owut upgrade

2. You do want to create one:

   owut upgrade --init-script my-init-script.sh

Note that for all of the above cases, if you do use asu-defaults, then you are responsible for keeping a backup. For best practices, see the persistent defaults example.

On rare occasion, it might be desirable to change the file system type of an installation. Usually this is done on devices with expandable file systems, x86 and ARM SBCs, where the storage device is not fixed size FLASH memory (in fact, if you try to change the file system type on an all-in-one, FLASH-based device, the build will almost always fail).

But, say for example, you have an x86 with an SSD and want to switch from the current squashfs to use ext4. Simply upgrade with the desired file system, and upon reboot you'll be running the targeted file system.

$ owut upgrade --fstype ext4
Target         x86/64
Profile        generic
Root-FS-type   squashfs                   <<<  Installed
Sys-type       combined-efi
Package-arch   x86_64
Version-from   SNAPSHOT r26504-d4acd05218 (kernel 6.6.32)
Version-to     SNAPSHOT r26733-2b6772c82c (kernel 6.6.34)
Build-FS-type  ext4                       <<<  Requested
...

The --rootfs-size option allows those devices with expandable file systems (again, typically x86 and ARM SBCs) to specify the size of the root file system. The default value varies depending on target, but is often 104 MB and the --rootfs-size option allow you to increase that up to 1024 MB, thus allowing more or bigger packages to be installed.

You may place this value in the config and avoid typing it on the command line every time you upgrade.

$ uci set attendedsysupgrade.owut.rootfs_size=256
$ uci commit
$ uci show attendedsysupgrade.owut
attendedsysupgrade.owut=owut
attendedsysupgrade.owut.rootfs_size='256'

The owut list command uses the --format option that takes one of the following values.

• fs-user - (the default) produces a package list for use by the Firmware Selector that contains only the top-level, user-installed package modifications. You'd copy and paste this after the default list in the FS 'Installed Packages' field.
• fs-all - produces a package list for FS containing all top-level packages, which you'd paste over the values in the FS 'Installed Packages' field.
• config - produces a build .config snippet of user-installed, top-level packages that you can use when doing source builds. Each output line looks like CONFIG_PACKAGE_collectd-mod-thermal=y.

The fs-* options generate lists in the Image Builder syntax, where mentioning a package name adds it to the list, and prefixing a package name with a dash removes it.

For example, if you have installed dnsmasq-full, then the default dnsmasq basic package must be removed. That would look like this (trimmed down for clarity):

$ owut list
... dnsmasq-full ... -dnsmasq ...

There are several packages provided by the defaults that are named using a generic package name, and actually provided by something with a different name, nftables is a prominent one (it is provided by either nftables-json or nftables-nojson, there is no nftables package). owut appears to be removing it, but it really is just saying, “use the default, whatever that is”. In the following example, we see only the -nftables removal, but not default package nftables-json as it will be added implicitly.

$ opkg whatprovides nftables
What provides nftables
    nftables-json
 
$ owut list
... -nftables ...

This can happen for other files that appear to be “deleted from nowhere” due to dependencies. As an example of this, if you are using luci-ssl-openssl, then the list output will contain -libustream-mbedtls which would otherwise be added by defaults resulting in an “impossible package selection” error.

You'll often see other evidence of these mappings when using check in the default package analysis results:

$ owut check
...
Default package analysis:
  Default                        Provided-by
  dnsmasq                        dnsmasq-full
  kmod-dwmac-intel               not installed
  nftables                       nftables-json
...

owut has a hook between image verification and image install, allowing you to do automatic backups, archiving of build artifacts or any other final modifications to the system prior to the installation. Use the --pre-install /path/to/script option from the command line, or add option pre_install '/path/to/script' to the owut section of the config file.

The standard installation of owut provides several examples in /etc/owut.d/pre-install.sh.

Note that the directory /etc/owut.d/ is part of the standard sysupgrade backup, so any files you store there will become part of system backups and persist across upgrades (assuming you “keep config”).

Any of the command line options may be stored in the owut section of /etc/config/attendedsysupgrade. For example, if you are using --rootfs-size 256 on the command line on every upgrade, you could edit the config and add that option as follows:

config server 'server'
        option url 'https://sysupgrade.openwrt.org'

config owut 'owut'
       option rootfs_size 256

Note that the dashes in the command line “long” option names are turned into underscores in the config option name, but beyond that the syntax is pretty much identical. To see the effect of your changes to the config, you can use the owut dump command and examine the options section of the output.

There is a single exception to the naming convention, command line --verbose maps to verbosity in the config file. The config file value is an integer, setting the default verbosity level, and every mention on the command line simply increments it: option verbosity 1, then owut check -v -v -v results in an output verbosity of 4 (i.e., more output than you ever thought possible).

$ owut dump | head -20
{
"version": "owut/2024.06.18-r1",
"options": {
  "command": "dump",
  "version_to": null,
  "verbosity": 0,
  "keep": false,
  "force": false,
  "add": null,
  "remove": null,
  "init_script": null,
  "fstype": null,
  "rootfs_size": 256,
  "image": "/tmp/firmware.bin",
  "format": null
},
...

You can force the default for the --version-to by setting this option in the config. Note that this is redundant with owut's default behavior (see the details of Selecting a version, above).

config owut 'owut'
        option version_to '23.05'

Before you set this one, please read about expanding the root file system, above, as this may not apply to your device.

For devices with expandable storage, typically x86 and ARM SBCs, you may find it useful to expand the root file system size thus allowing you to easily install a larger number of packages or store more persistent data.

config owut 'owut'
        option rootfs_size 512

If you've used custome uci-defaults, you are probably aware that the scripts are deleted from /etc/uci-defaults/ on first boot. For an immutable installation, typically using a squashfs file system, you can view and recover your custom script from /rom/etc/uci-defaults/, but on an ext4 file system, there is no rom partition, and the files are gone forever.

owut addresses this issue by allowing you to place your script on a persistent location, and then setting up the config file so that it becomes part of the installed image*.

1. Create your init-script in a persistent location.

   $ mkdir /etc/owut.d/
   $ echo "# My first boot script." > /etc/owut.d/custom-init.sh

2. Make sure it is carried over in all sysupgrade backups.

   $ echo "/etc/owut.d/" >> /etc/sysupgrade.conf
   $ sysupgrade -l | grep custom
   /etc/owut.d/custom-init.sh

3. Configure owut to include the script into your builds.

   $ uci set attendedsysupgrade.owut.init_script=/etc/owut.d/custom-init.sh
   $ uci commit
   $ uci show attendedsysupgrade.owut
   attendedsysupgrade.owut=owut
   attendedsysupgrade.owut.init_script='/etc/owut.d/custom-init.sh'

4. Verify that everything is set up properly. Your script should appear in the json as the defaults entry.

   $ owut blob
   {
     "client": "owut/%%VERSION%%",
     "target": "x86/64",
     "profile": "generic",
     "version": "SNAPSHOT",
     "version_code": "r26773-85d9fd6f0e",
     "filesystem": "ext4",
     "diff_packages": true,
     "packages": [
       "base-files",
       "btop",
       "busybox",
       ... a bunch more packages ...
       "ucode-mod-uloop",
       "urandom-seed",
       "urngd"
     ],
     "defaults": "# My first boot script.\n"
   }

* - As of June 2024, you can also accomplish this latter part by creating an image with the OpenWrt Firmware Selector, manually pasting the script into its Script to run on first boot (uci-defaults) field.

Q: I have a big x86/ARM/whatever router. Why can't I create an image with --rootfs-size 120000 and use all of my 128 GB drive?

A: The upper bound is dictated to be 1024 MB by the ASU server for practical reasons. The way the partitions are created on some devices requires that the full-sized image be created and then compressed, which takes a lot of RAM, disk and time.

As a test, I did a few imagebuilder runs with rootfs partitions ranging from the default (for x86) of 104 MB up to 20000 MB to see how long they would take.

Those last two rows should make fairly clear why increasing the upper limit is infeasible, until such time as the build process is reworked to reduce the time required to create larger images.

Note: the above tests were all run on an AMD R9 7950x 5.8GHz CPU with 64GB CL6000 RAM and PCIe 4 SSD, which is generally 3x faster than the ASU server hardware, so these numbers are well below what you'd see if using ASU server.

• ASU server - github source, API documentation
• auc - wiki page, see final commit removing auc from the package feed
• LuCI Attended Sysupgrade - wiki page, github source
• Firmware Selector - github source, build site
• Downloads site - OpenWrt build artifacts and release archive
• Dashboards - OpenWrt buildbot dashboards
• ucode - Reference Manual
• ucode-mod-uclient - github source