owut: OpenWrt Upgrade Tool

For a more detailed description of the overall process, see the Attended Sysupgrade page.

owut is a command line upgrade tool that creates and installs custom builds of OpenWrt containing all of your currently installed packages and retaining your configuration.

Its subcommands and options allow you to

  • check latest build status
  • list all the OpenWrt versions available on the upgrade server
  • download an image without installing it
  • verify and install an image you downloaded
  • generate lists of installed package for use with Firmware Selector or source builds
  • when generating an image you can
    • add packages to or remove packages from the build list
    • include a custom uci-defaults script
    • specify a non-standard root filesystem size
    • change the file system type at upgrade time
  • and more.

You can specify these options on the command line, or you can use the standard OpenWrt config system to store these values and avoid having to remember and retype them on every upgrade.

If your goal is simply to upgrade your router's current firmware, staying on the same version (e.g., 23.05 or SNAPSHOT), then just use the upgrade command. If there are any problems uncovered by the various pre-build checks, or if an error is detected during the build, then the upgrade will abort with a message indicating the issue.

Your first step is always...

Make a backup!

  • From LuCI, go to System → Backup/Flash firmware. Click Generate archive.
  • From CLI use sysupgrade --create-backup /tmp/backup.tar.gz and use scp or some other tool to copy the file to a safe location (usually another host).

Just do it. Every time...

If owut finds that package downgrades, or no changes were made since your last upgrade, it will tell you this and stop. You can re-run the command with the --force option, which will proceed with the build and install, keeping all configuration.

Note that owut is currently available only on SNAPSHOT or versions 24.x and later.

$ opkg update && opkg install owut
$ owut upgrade
Board-name     generic
Target         x86/64
Root-FS-type   squashfs
Sys-type       combined-efi
Package-arch   x86_64
Version-from   SNAPSHOT r26670-5f9fb964c3 (kernel 6.6.33)
Version-to     SNAPSHOT r26713-68f7ca23fb (kernel 6.6.33)
Build-FS-type  squashfs
Build-at       2024-06-19T17:36:09Z
Image-prefix   openwrt-x86-64-generic
Image-file     openwrt-x86-64-generic-squashfs-combined-efi.img.gz
Image-URL      https://downloads.openwrt.org/snapshots/targets/x86/64/openwrt-x86-64-generic-squashfs-combined-efi.img.gz
Installed      272 packages
Top-level       82 packages
Default         46 packages
User-installed  51 packages (top-level only)

Package version changes:
  base-files                   1603~5f9fb964c3                            1605~68f7ca23fb
1 packages are out-of-date.

Default package analysis:
  Default            Provided-by
  dnsmasq            not installed, possibly replaced by another package
  kmod-dwmac-intel   not installed, possibly replaced by another package
  nftables           not installed, possibly replaced by another package

There are currently package build failures for SNAPSHOT x86_64:
  Feed: packages
    basicstation               Wed Jun 19 08:07:53 2024 - not installed
    xtables-addons             Wed Jun 19 09:39:10 2024 - not installed
  Feed: telephony
    freeswitch                 Wed Jun 19 11:30:36 2024 - not installed
Failures don't affect this device, details at
  https://downloads.openwrt.org/snapshots/faillogs/x86_64/

Build: Requesting version r26713-68f7ca23fb (kernel 6.6.33)

Requesting build ----------------------
Hash:   00b63d9130a1df0a3af34deb5d9a17ff
Status: 202
Detail: queued - 0 ahead of you

Progress   1 (  11s) -----------------------------
Status: 202
Detail: started - calculate_packages_hash

Progress   2 (  22s) -----------------------------
Status: 202
Detail: started - calculate_packages_hash

Progress   3 (  33s) -----------------------------
Status: 202
Detail: started - building_image

...

Progress  14 ( 154s) -----------------------------
Status: 202
Detail: started - building_image

Progress  15 ( 165s) -----------------------------
Status: 200
Detail: done

Build completed in 167 seconds.
  rootfs_size_mb = default
  init-script    = no-init-script

Image source: https://sysupgrade.openwrt.org/store/00b63d9130a1df0a3af34deb5d9a17ff/openwrt-5c211e880ca6-x86-64-generic-squashfs-combined-efi.img.gz
Image saved : /tmp/firmware.bin
Manifest    : /tmp/firmware-manifest.json
Verifying   : /tmp/firmware.bin (24812785 bytes) against /tmp/firmware.sha256sums
  Saved sha256 matches
  Wed Jun 19 18:28:38 PDT 2024 upgrade: Image metadata not present
  Wed Jun 19 18:28:38 PDT 2024 upgrade: Reading partition table from bootdisk...
  Wed Jun 19 18:28:39 PDT 2024 upgrade: Extract boot sector from the image
  Wed Jun 19 18:28:39 PDT 2024 upgrade: Reading partition table from image...
  Wed Jun 19 18:28:39 PDT 2024 upgrade: Partition layout has changed. Full image will be written.
Checks complete, image is valid.
Installing /tmp/firmware.bin and rebooting...

... system reboots ...
$ owut -h
owut - OpenWrt Upgrade Tool 2024.07.18~23c20667-r1 (/usr/bin/owut)
 
owut is an upgrade tool for OpenWrt.
 
Usage: owut COMMAND [-V VERSION_TO] [-v] [-k] [--force] [-a ADD] [-r REMOVE] [-I INIT_SCRIPT] [-F FSTYPE] [-S ROOTFS_SIZE] [-i IMAGE] [-f FORMAT]
  -h/--help            - Show this message and quit.
  --version            - Show the program version and terminate.
 
  COMMAND - Sub-command to execute, must be one of:
    check    - Collect all resources and report stats.
    list     - Show all the packages installed by user.
    blob     - Display the json blob for the ASU build request.
    download - Build, download and verify an image.
    verify   - Verify the downloaded image.
    install  - Install the specified local image.
    upgrade  - Build, download, verify and install an image.
    versions - Show available versions.
    dump     - Collect all resources and dump internal data structures.
 
  -V/--version-to VERSION_TO - Specify the target version, defaults to installed version.
  -v/--verbose         - Print various diagnostics.  Repeat for even more output.
  -k/--keep            - Save all downloaded working files.
  --force              - Force a build even when there are downgrades or no changes.
  -a/--add ADD         - Comma-separated list of new packages to add to build list.
  -r/--remove REMOVE   - Comma-separated list of installed packages to remove from build list.
  -I/--init-script INIT_SCRIPT - Path to uci-defaults script to run on first boot ('-' use stdin).
  -F/--fstype FSTYPE   - Desired root file system type (squashfs, ext4, ubifs, jffs2).
  -S/--rootfs-size ROOTFS_SIZE - Root file system size in MB (1-1024).
  -i/--image IMAGE     - Image name for download, verify, install and upgrade.
  -f/--format FORMAT   - Format for 'list' output (fs-user, fs-all, config).
 
Full documentation
  https://openwrt.org/docs/guide-user/installation/sysupgrade.owut
 
Questions and discussion
  https://forum.openwrt.org/t/owut-openwrt-upgrade-tool/200035
 
Issues and bug reports
  https://github.com/efahl/owut/issues
Sub-Command Description
check Downloads all resource files, collects the metadata from the device and the resources, and displays a report on everything found. This includes available version upgrades on all packages, availability of installed packages, listing of all package build breakages, and so on. At the end of the report, you'll see an indication as to whether it is possible to upgrade or not.
list This sub-command allows you to generate the list of packages installed on your device. This list is tailored for use with either the OpenWrt Firmware Selector or for use with source builds. For more details, see the --format option description, below.
blob Display the json blob for the ASU build request. Mostly useful for debugging and satisfying your curiosity.
download Build, download and verify an image. Used to create an image that you may then archive off-system, before subsequently owut installing it.
verify Verify the downloaded image. After you have downloaded an image, verify can be used to make sure it corresponds to the downloaded checksums and is correct according to sysupgrade.
install Install the specified image. Does another verify, then runs sysupgrade to install the image, results in a reboot.
upgrade Build, download, verify and install an image. Short hand way to run all steps involved in upgrading the system, basically a download and install in one command.
versions Show available versions according to what the ASU server knows. This may not be a complete list with respect to what is available on the build servers (aka downloads.openwrt.org), as the ASU server does not deal with archives and discourages use of out-of-date releases.
dump Collect all resources and dump internal data structures. Again, like the blob sub-command, this is for debugging and curious users.
Option Default Description
-V/--version-to VERSION newest version on current branch Search for updates for this version or branch.
-v/--verbose - Print various diagnostics and operational messages; repeat for even more output.
-k/--keep false Save all downloaded working files, primarily for diagnostics and debugging. Look in /tmp/ after using this option; turn on -v to watch as they are saved.
--force false Force download or upgrade when there are package downgrades, or when there are no changes detected.
-a/--add ADD none Comma-separated list of new packages to add to build list.
-r/--remove REMOVE none Comma-separated list of installed packages to remove from build list.
-I/--init-script INIT_SCRIPT none Path to uci-defaults script to run on first boot ('-' use stdin).
-F/--fstype FSTYPE current Desired root file system type. May be one of squashfs, ext4, ubifs or jffs2 depending on platform constraints.
-S/--rootfs-size ROOTFS_SIZE current Root file system size in MB (1-1024).
-i/--image IMAGE /tmp/firmware.bin The image name used for the download, verify, install and upgrade sub-commands.
-f/--format FORMAT fs-user Format for list output. Valid formats are fs-user, fs-all and config.

When you don't specify a version

When you do not explicitly specify a --version-to value, owut looks for the newest version of the installed branch and sets that as the target version.

For all examples, assume that latest on the 21.02 branch is the 21.02.7 release, and on 23.05 it's the 23.05.4 release.

Installed Version Target
21.02.2 21.02.7
23.05.0-rc1 23.05.4
23.05.3 23.05.4
23.05.4 23.05.4

The exception to this is if the installed version is a SNAPSHOT, either release or main, in which case, the version-to target remains at the installed version.

Installed Version Target
22.03-SNAPSHOT 22.03-SNAPSHOT
23.05-SNAPSHOT 23.05-SNAPSHOT
SNAPSHOT SNAPSHOT

When you do specify a version

When you do specify --version-to, it must name a valid version or branch. If you provide an invalid value, owut will show you all the available versions (or you can do this manually with owut versions).

When you specify a full version, then the input is checked against the available versions and left untouched:

User specifies Target
--version-to 23.05.3 23.05.3
--version-to 23.05.0-rc2 23.05.0-rc2
--version-to 23.05-snapshot 23.05-SNAPSHOT
--version-to snapshot SNAPSHOT

If you specify only a branch (i.e., a version number without the final “dot value”), then the version-to target is set to the latest release on that branch.

User specifies Target
--version-to 21.02 21.02.7
--version-to 23.05 23.05.4
--version-to SNAPSHOT SNAPSHOT

Note that character case is not important in naming the version, owut converts internally to what the upgrade server requires. You can say snapshot, SnapShot or SNAPSHOT or rc1 or rC1 and owut knows what to do.

owut's normal behavior is to avoid doing unneeded work by stopping a build request when no changes are found. The --force option is used to override this and do a re-build and re-install of the current system. This might be useful if you have inadvertently deleted packages or something similar, and can't easily figure out how to recover.

When owut detects downgrades in packages, it will indicate this by coloring the new version number red in the Package version changes: list and report the number of downgrades at the bottom of the list. This also causes owut to stop processing any download or upgrade in progress, unless you specify --force option.

The --add and --remove options allow you to add packages to or remove packages from the build list submitted by owut to the ASU build server.

For example, if you wanted to upgrade and simultaneously switch to the full versions of dnsmasq and tc, you'd say this.

$ owut upgrade -r dnsmasq,tc-tiny -a dnsmasq-full,tc-full
...

About adding packages

  1. If you add a package that is already installed, owut silently ignores this and carries on.
  2. If you add a package that doesn't exist, hasn't been ported to your device or is currently unavailable, then owut reports an error and stops.

About removing packages

  1. If you attempt to remove a package this is not installed, owut reports the error and stops.
  2. If you remove one of the default packages for your device, owut will produce a warning, the package will be removed and owut will continue with your request. Note that this may break things, either during the build or after installing on your device, and you are responsible for ensuring that removal of the package is appropriate.
    $ owut check -r kmod-igb
    WARNING: package 'kmod-igb' is a default package, removal may have unknown side effects
    ...
  3. If you attempt to remove a package that has dependents (that is a “non-top-level package” -- something that was installed as a requirement for another package), owut will warn you about this, the package will be removed from the build list and owut will proceed. Note that this typically has no effect, as the package will be pulled back in by the same top-level package that installed it in the first place.
    $ owut check -r kernel
    WARNING: package 'kernel' has dependents and removal will have no effect on the build
    ...

Do not put sensitive information in the init-script file.

The text you provide in the init-script file is sent to the build server as part of the build configuration. This exposes it to potential disclosure as the build request traverses the internet. Once the build has completed, the file's contents are stored on the build server in the generated image, where anyone with knowledge of the build hash may download and access it.

If you are not familiar with uci-defaults (aka “first boot scripts”), you can read up here: UCI defaults. The underlying mechanism that implements owut (and auc and LuCI Attended Sysupgrade and Firmware Selector) builds is the Image Builder, so its description may also be useful: Image builder - Custom files.

The --init-script option allows you to specify the name of a file containing a uci-defaults script, which is to be executed at first boot. The ASU server takes your init-script source and places it in the image in /etc/uci-defaults/99-asu-defaults (there is no means to change this name). On immutable file systems (say squashfs), this also results in the file being stored in /rom/etc/uci-defaults/99-asu-defaults, which comes into play with LuCI Attended Sysupgrade.

Here's a comparison of owut with how other upgrade tools implement this functionality.

  • auc does not implement this ability (but the patch exists).
  • Firmware Selector behaves identically to owut with its Script to run on first boot (uci-defaults) input field.
  • LuCI Attended Sysupgrade implements this by looking for /rom/etc/uci-defaults/99-asu-defaults and then relays the contents of that file implicitly. LuCI ASU's shortcoming is that it doesn't allow you to delete or change what's already there. owut makes this explicit, if you want the script included in your new image, then you must specify it when you request a build.

The terms used in the following two scenarios (comprising seven use cases) are:

  • “mutable system” - A device using a read-write file system for its main storage. Typically in OpenWrt, this is an ext4 file system.
    $ mount | grep '(ro'
    ... no output ...
  • “immutable system” - A device which has a read-only partition mounted on /rom, where the original system image is stored. Usually, but not always, this is a squashfs file system.
    $ mount | grep '(ro'
    /dev/root on /rom type squashfs (ro,relatime,errors=continue)
  • “asu-defaults” - The file /rom/etc/uci-defaults/99-asu-defaults, which only exists on devices with an immutable file system and whose images have been built by the ASU build server.

On an immutable system, there are five cases to consider:

  1. An asu-defaults file does not exist and you don't need or want one:
    owut upgrade
  2. An asu-defaults file exists and you want to keep it unchanged:
    owut upgrade --init-script /rom/etc/uci-defaults/99-asu-defaults
  3. An asu-defaults file exists and you want to modify it:
    cp /rom/etc/uci-defaults/99-asu-defaults my-modified-init-script.sh
    vi my-modified-init-script.sh  # Change it.
    owut upgrade --init-script my-modified-init-script.sh
  4. An asu-defaults file does not exist but you want to add one:
    vi my-new-init-script.sh
    owut upgrade --init-script my-new-init-script.sh
  5. An asu-defaults file exists and you want to delete it from build:
    owut upgrade  # Just ignore the warning.

On a mutable system, since there is no /rom/etc/uci-defaults, you only have two choices:

  1. You don't want to create an asu-defaults file in your build:
    owut upgrade
  2. You do want to create one:
    owut upgrade --init-script my-init-script.sh

Note that for all of the above cases, if you do use asu-defaults, then you are responsible for keeping a backup. For best practices, see the persistent defaults example.

On rare occasion, it might be desirable to change the file system type of an installation. Usually this is done on devices with expandable file systems, x86 and ARM SBCs, where the storage device is not fixed size FLASH memory (in fact, if you try to change the file system type on an all-in-one, FLASH-based device, the build will almost always fail).

But, say for example, you have an x86 with an SSD and want to switch from the current squashfs to use ext4. Simply upgrade with the desired file system, and upon reboot you'll be running the targeted file system.

$ owut upgrade --fstype ext4
Target         x86/64
Profile        generic
Root-FS-type   squashfs                   <<<  Installed
Sys-type       combined-efi
Package-arch   x86_64
Version-from   SNAPSHOT r26504-d4acd05218 (kernel 6.6.32)
Version-to     SNAPSHOT r26733-2b6772c82c (kernel 6.6.34)
Build-FS-type  ext4                       <<<  Requested
...

Changing your root file system size often causes owut's final installation performed by sysupgrade to lose your configuration, so be prepared to recover with a backup. Note that if you lose /etc/config/network, the LAN IP of the device may change, so think about how you'll attach to the device before you proceed.

The --rootfs-size option allows those devices with expandable file systems (again, typically x86 and ARM SBCs) to specify the size of the root file system. The default value varies depending on target, but is often 104 MB and the --rootfs-size option allow you to increase that up to 1024 MB, thus allowing more or bigger packages to be installed.

You may place this value in the config and avoid typing it on the command line every time you upgrade.

$ uci set attendedsysupgrade.owut.rootfs_size=256
$ uci commit
$ uci show attendedsysupgrade.owut
attendedsysupgrade.owut=owut
attendedsysupgrade.owut.rootfs_size='256'

The owut list command uses the --format option that takes one of the following values.

  • fs-user - (the default) produces a package list for use by the Firmware Selector that contains only the top-level, user-installed package modifications. You'd copy and paste this after the default list in the FS 'Installed Packages' field.
  • fs-all - produces a package list for FS containing all top-level packages, which you'd paste over the values in the FS 'Installed Packages' field.
  • config - produces a build .config snippet of user-installed, top-level packages that you can use when doing source builds. Each output line looks like CONFIG_PACKAGE_collectd-mod-thermal=y.

Note that although you can store any of the command line options in the config file, doing so with certain options may be confusing. Use your discretion when setting defaults: as an example, setting option force true is probably not a good idea.

Any of the command line options may be stored in the owut section of /etc/config/attendedsysupgrade. For example, if you are using --rootfs-size 256 on the command line on every upgrade, you could edit the config and add that option as follows:

config server 'server'
        option url 'https://sysupgrade.openwrt.org'

config owut 'owut'
       option rootfs_size 256

Note that the dashes in the command line “long” option names are turned into underscores in the config option name, but beyond that the syntax is pretty much identical. To see the effect of your changes to the config, you can use the owut dump command and examine the options section of the output.

There is a single exception to the naming convention, command line --verbose maps to verbosity in the config file. The config file value is an integer, setting the default verbosity level, and every mention on the command line simply increments it: option verbosity 1, then owut check -v -v -v results in an output verbosity of 4 (i.e., more output than you ever thought possible).

$ owut dump | head -20
{
"version": "owut/2024.06.18-r1",
"options": {
  "command": "dump",
  "version_to": null,
  "verbosity": 0,
  "keep": false,
  "force": false,
  "add": null,
  "remove": null,
  "init_script": null,
  "fstype": null,
  "rootfs_size": 256,
  "image": "/tmp/firmware.bin",
  "format": null
},
...

Stay on a given release

You can force the default for the --version-to by setting this option in the config. Note that this is redundant with owut's default behavior (see the details of Selecting a version, above).

config owut 'owut'
        option version_to '23.05'

Setting root file system size

Before you set this one, please read about expanding the root file system, above, as this may not apply to your device.

For devices with expandable storage, typically x86 and ARM SBCs, you may find it useful to expand the root file system size thus allowing you to easily install a larger number of packages or store more persistent data.

config owut 'owut'
        option rootfs_size 512

Persistent uci-defaults

If you've used custome uci-defaults, you are probably aware that the scripts are deleted from /etc/uci-defaults/ on first boot. For an immutable installation, typically using a squashfs file system, you can view and recover your custom script from /rom/etc/uci-defaults/, but on an ext4 file system, there is no rom partition, and the files are gone forever.

owut addresses this issue by allowing you to place your script on a persistent location, and then setting up the config file so that it becomes part of the installed image*.

  1. Create your init-script in a persistent location.
    $ mkdir /etc/owut.d/
    $ echo "# My first boot script." > /etc/owut.d/custom-init.sh
  2. Make sure it is carried over in all sysupgrade backups.
    $ echo "/etc/owut.d/" >> /etc/sysupgrade.conf
    $ sysupgrade -l | grep custom
    /etc/owut.d/custom-init.sh
  3. Configure owut to include the script into your builds.
    $ uci set attendedsysupgrade.owut.init_script=/etc/owut.d/custom-init.sh
    $ uci commit
    $ uci show attendedsysupgrade.owut
    attendedsysupgrade.owut=owut
    attendedsysupgrade.owut.init_script='/etc/owut.d/custom-init.sh'
  4. Verify that everything is set up properly. Your script should appear in the json as the defaults entry.
    $ owut blob
    {
      "client": "owut/%%VERSION%%",
      "target": "x86/64",
      "profile": "generic",
      "version": "SNAPSHOT",
      "version_code": "r26773-85d9fd6f0e",
      "filesystem": "ext4",
      "diff_packages": true,
      "packages": [
        "base-files",
        "btop",
        "busybox",
        ... a bunch more packages ...
        "ucode-mod-uloop",
        "urandom-seed",
        "urngd"
      ],
      "defaults": "# My first boot script.\n"
    }

* - As of June 2024, you can also accomplish this latter part by creating an image with the OpenWrt Firmware Selector, manually pasting the script into its Script to run on first boot (uci-defaults) field.

Q: I have a big x86/ARM/whatever router. Why can't I create an image with --rootfs-size 120000 and use all of my 128 GB drive?

A: The upper bound is dictated to be 1024 MB by the ASU server for practical reasons. The way the partitions are created on some devices requires that the full-sized image be created and then compressed, which takes a lot of RAM, disk and time.

As a test, I did a few imagebuilder runs with rootfs partitions ranging from the default (for x86) of 104 MB up to 20000 MB to see how long they would take.

ROOTFS_PARTSIZE= real user img size
104 26s 18s 12M
512 48s 25s 13M
1024 74s 33s 13M
10000 11m47s 4m36s 32M
20000 28m15s 13m9s 32M

Those last two rows should make fairly clear why increasing the upper limit is infeasible, until such time as the build process is reworked to reduce the time required to create larger images.

Note: the above tests were all run on an AMD R9 7950x 5.8GHz CPU with 64GB CL6000 RAM and PCIe 4 SSD, which is generally 3x faster than the ASU server hardware, so these numbers are well below what you'd see if using ASU server.

This website uses cookies. By using the website, you agree with storing cookies on your computer. Also you acknowledge that you have read and understand our Privacy Policy. If you do not agree leave the website.More information about cookies
  • Last modified: 2024/07/18 19:10
  • by efahlgren