| Both sides previous revision Previous revision Next revision | Previous revision |
| docs:guild-user:installation:sysupgrade.owut [2024/06/21 17:37] – [Quick Start] fix missing message efahlgren | docs:guild-user:installation:sysupgrade.owut [2024/06/23 22:44] (current) – delete contents - in wrong location efahlgren |
|---|
| ===== Overview ===== | |
| |
| ''owut'' is an command line upgrade tool that installs custom builds of OpenWrt containing all of the currently installed packages and retaining your configuration. It allows you to add or remove packages, include a custom ''uci-defaults'' script, specify a non-standard root filesystem size and more. You can specify these options on the command line, or you can use the standard OpenWrt ''config'' system to store these values and avoid having to remember and retype them on every upgrade. | |
| |
| ===== Quick Start ===== | |
| |
| If your goal is simply to upgrade your router's current firmware, staying on the same version (e.g., 23.05 or SNAPSHOT), then just use the ''upgrade'' command. If there are any problems uncovered by the various pre-build checks, or if an error is detected during the build, then the upgrade will abort with a message indicating the issue. | |
| |
| If ''owut'' finds that no changes were made since you last upgrade, it will tell you this and stop. You can override this behavior with the ''--force'' option, which will proceed to build and re-install your current version, keeping all configuration. | |
| |
| <code> | |
| $ owut upgrade | |
| Board-name generic | |
| Target x86/64 | |
| Root-FS-type squashfs | |
| Sys-type combined-efi | |
| Package-arch x86_64 | |
| Version-from SNAPSHOT r26670-5f9fb964c3 (kernel 6.6.33) | |
| Version-to SNAPSHOT r26713-68f7ca23fb (kernel 6.6.33) | |
| Build-FS-type squashfs | |
| Build-at 2024-06-19T17:36:09Z | |
| Image-prefix openwrt-x86-64-generic | |
| Image-file openwrt-x86-64-generic-squashfs-combined-efi.img.gz | |
| Image-URL https://downloads.openwrt.org/snapshots/targets/x86/64/openwrt-x86-64-generic-squashfs-combined-efi.img.gz | |
| Installed 272 packages | |
| Top-level 82 packages | |
| Default 46 packages | |
| User-installed 51 packages (top-level only) | |
| |
| Package version changes: | |
| base-files 1603~5f9fb964c3 1605~68f7ca23fb | |
| 1 packages are out-of-date. | |
| |
| Default package analysis: | |
| Default Provided-by | |
| dnsmasq not installed, possibly replaced by another package | |
| kmod-dwmac-intel not installed, possibly replaced by another package | |
| nftables not installed, possibly replaced by another package | |
| |
| There are currently package build failures for SNAPSHOT x86_64: | |
| Feed: packages | |
| basicstation Wed Jun 19 08:07:53 2024 - Package not installed locally | |
| xtables-addons Wed Jun 19 09:39:10 2024 - Package not installed locally | |
| Feed: telephony | |
| freeswitch Wed Jun 19 11:30:36 2024 - Package not installed locally | |
| Failures don't affect this device, details at | |
| https://downloads.openwrt.org/snapshots/faillogs/x86_64/ | |
| |
| Build: Requesting version r26713-68f7ca23fb (kernel 6.6.33) | |
| |
| Requesting build ---------------------- | |
| Hash: 00b63d9130a1df0a3af34deb5d9a17ff | |
| Status: 202 | |
| Detail: queued - 0 ahead of you | |
| |
| Progress 1 ( 11s) ----------------------------- | |
| Status: 202 | |
| Detail: started - calculate_packages_hash | |
| |
| Progress 2 ( 22s) ----------------------------- | |
| Status: 202 | |
| Detail: started - calculate_packages_hash | |
| |
| Progress 3 ( 33s) ----------------------------- | |
| Status: 202 | |
| Detail: started - building_image | |
| |
| ... | |
| |
| Progress 14 ( 154s) ----------------------------- | |
| Status: 202 | |
| Detail: started - building_image | |
| |
| Progress 15 ( 165s) ----------------------------- | |
| Status: 200 | |
| Detail: done | |
| |
| Build completed in 167 seconds. | |
| rootfs_size_mb = default | |
| init-script = no-init-script | |
| |
| Image source: https://sysupgrade.openwrt.org/store/00b63d9130a1df0a3af34deb5d9a17ff/openwrt-5c211e880ca6-x86-64-generic-squashfs-combined-efi.img.gz | |
| Image saved : /tmp/firmware.bin | |
| Manifest : /tmp/firmware-manifest.json | |
| Verifying : /tmp/firmware.bin (24812785 bytes) against /tmp/firmware.sha256sums | |
| Saved sha256 matches | |
| Wed Jun 19 18:28:38 PDT 2024 upgrade: Image metadata not present | |
| Wed Jun 19 18:28:38 PDT 2024 upgrade: Reading partition table from bootdisk... | |
| Wed Jun 19 18:28:39 PDT 2024 upgrade: Extract boot sector from the image | |
| Wed Jun 19 18:28:39 PDT 2024 upgrade: Reading partition table from image... | |
| Wed Jun 19 18:28:39 PDT 2024 upgrade: Partition layout has changed. Full image will be written. | |
| Checks complete, image is valid. | |
| Installing /tmp/firmware.bin and rebooting... | |
| |
| ... system reboots ... | |
| </code> | |
| |
| ===== Usage ===== | |
| |
| For simple | |
| |
| <code bash> | |
| owut - OpenWrt Upgrade Tool version 2024.06.18-r1 (/root/bin/owut) | |
| |
| owut is an upgrade tool for OpenWrt. | |
| |
| Usage: owut COMMAND [-V VERSION] [-v] [-k] [--force] [-a ADD] [-r REMOVE] [-I INIT_SCRIPT] [-F FSTYPE] [-S ROOTFS_SIZE] [-i IMAGE] [-f FORMAT] | |
| -h/--help - Show this message and quit. | |
| --version - Show the program version and terminate. | |
| |
| COMMAND - Sub-command to execute, must be one of: | |
| check - Collect all resources and report stats. | |
| list - Show all the packages installed by user. | |
| blob - Display the json blob for the ASU build request. | |
| download - Build, download and verify an image. | |
| verify - Verify the downloaded image. | |
| install - Install the specified local image. | |
| upgrade - Build, download, verify and install an image. | |
| versions - Show available versions. | |
| dump - Collect all resources and dump internal data structures. | |
| |
| -V/--version-to VERSION - Specify the target version, defaults to installed version. | |
| -v/--verbose - Print various diagnostics. Repeat for even more output. | |
| -k/--keep - Save all downloaded working files. | |
| --force - Force download when there are no changes detected. | |
| -a/--add ADD - Comma-separated list of new packages to add to build list. | |
| -r/--remove REMOVE - Comma-separated list of installed packages to remove from build list. | |
| -I/--init-script INIT_SCRIPT - Path to uci-defaults script to run on first boot ('-' use stdin). | |
| -F/--fstype FSTYPE - Desired root file system type (squashfs, ext4, ubifs, jffs2). | |
| -S/--rootfs-size ROOTFS_SIZE - Root file system size in MB (1-1024). | |
| -i/--image IMAGE - Image name for download, verify, install and upgrade. | |
| -f/--format FORMAT - Format for 'list' output (fs-user, fs-all, config). | |
| </code> | |
| |
| ===== Sub-Commands ===== | |
| |
| ^ Sub-Command ^ Description ^ | |
| | ''check'' | Downloads all resource files, collects the metadata from the device and the resources, and displays a report on everything found. This includes available version upgrades on all packages, availability of installed packages, listing of all package build breakages, and so on. At the end of the report, you'll see an indication as to whether it is possible to upgrade or not. | | |
| | ''list'' | This sub-command allows you to generate the list of packages installed on your device. This list is tailored for use with either the [[https://firmware-selector.openwrt.org/|OpenWrt Firmware Selector]] or for use with source builds. For more details, see the ''--format'' option description, below. | | |
| | ''blob'' | Display the json blob for the ASU build request. Mostly useful for debugging and satisfying your curiosity. | | |
| | ''download'' | Build, download and ''verify'' an image. Used to create an image that you may then archive off-system, before subsequently ''owut install''ing it. | | |
| | ''verify'' | Verify the downloaded image. After you have ''download''ed an image, ''verify'' can be used to make sure it corresponds to the downloaded checksums and is correct according to ''sysupgrade''. | | |
| | ''install'' | Install the specified image. Does another ''verify'', then runs ''sysupgrade'' to install the image, results in a reboot. | | |
| | ''upgrade'' | Build, download, verify and install an image. Short hand way to run all steps involved in upgrading the system, basically a ''download'' and ''install'' in one command. | | |
| | ''versions'' | Show available versions according to what the ASU server knows. This may not be a complete list with respect to what is available on the build servers (aka ''downloads.openwrt.org''), as the ASU server does not deal with archives and discourages use of out-of-date releases. | | |
| | ''dump'' | Collect all resources and dump internal data structures. Again, like the ''blob'' sub-command, this is for debugging and curious users. | | |
| |
| |
| ===== Command Line Options ===== | |
| |
| ^ Option ^ Default ^ Description ^ | |
| | ''-V/--version-to VERSION'' | newest version on current branch | Search for updates for this version or branch. | | |
| | ''-v/--verbose'' | - | Print various diagnostics and operational messages; repeat for even more output. | | |
| | ''-k/--keep'' | false | Save all downloaded working files, primarily for diagnostics and debugging. Look in ''/tmp/'' after using this option; turn on ''-v'' to watch as they are saved. | | |
| | ''--force'' | false | Force ''download'' or ''upgrade'' when there are no changes detected. | | |
| | ''-a/--add ADD'' | none | Comma-separated list of new packages to add to build list. | | |
| | ''-r/--remove REMOVE'' | none | Comma-separated list of installed packages to remove from build list. | | |
| | ''-I/--init-script INIT_SCRIPT'' | none | Path to uci-defaults script to run on first boot ('-' use stdin). | | |
| | ''-F/--fstype FSTYPE'' | current | Desired root file system type. May be one of ''squashfs'', ''ext4'', ''ubifs'' or ''jffs2'' depending on platform constraints. | | |
| | ''-S/--rootfs-size ROOTFS_SIZE'' | current | Root file system size in MB (1-1024). | | |
| | ''-i/--image IMAGE'' | ''/tmp/firmware.bin'' | The image name used for the ''download'', ''verify'', ''install'' and ''upgrade'' sub-commands. | | |
| | ''-f/--format FORMAT'' | ''fs-user'' | Format for ''list'' output. Valid formats are ''fs-user'', ''fs-all'' and ''config''. | | |
| |
| === Selecting a version === | |
| |
| When you do not explicitly specify a version-to value, ''owut'' looks for the newest version of the installed branch and sets that as the target version. | |
| |
| For all examples, assume that latest on the ''21.02'' branch is the ''21.02.7'' release and on ''23.05'' it's the ''23.05.4'' release. | |
| |
| ^ Installed Version ^ Target ^ | |
| | ''21.02.2'' | ''21.02.7'' | | |
| | ''23.05.0-rc1'' | ''23.05.4'' | | |
| | ''23.05.3'' | ''23.05.4'' | | |
| | ''23.05.4'' | ''23.05.4'' | | |
| |
| The exception to this is if the installed version is a ''SNAPSHOT'', either release or main, in which case, version-to is set to the installed version. | |
| |
| ^ Installed Version ^ Target ^ | |
| | ''22.03-SNAPSHOT'' | ''22.03-SNAPSHOT'' | | |
| | ''23.05-SNAPSHOT'' | ''23.05-SNAPSHOT'' | | |
| | ''SNAPSHOT'' | ''SNAPSHOT'' | | |
| |
| When you do specify a value, it must be a valid version or branch. | |
| |
| If the user specifies a full version, then the input is checked against the available versions and left untouched: | |
| |
| ^ User specifies ^ Target ^ | |
| | ''--version-to 23.05.3'' | ''23.05.3'' | | |
| | ''--version-to 23.05.0-rc2'' | ''23.05.0-rc2'' | | |
| | ''--version-to 23.05-snapshot'' | ''23.05-SNAPSHOT'' | | |
| | ''--version-to snapshot'' | ''SNAPSHOT'' | | |
| |
| If you specify only a branch (i.e., a version number without the final "dot value"), then the version-to is set to the latest release on that branch. | |
| |
| ^ User specifies ^ Target ^ | |
| | ''--version-to 21.02'' | ''21.02.7'' | | |
| | ''--version-to 23.05'' | ''23.05.4'' | | |
| | ''--version-to SNAPSHOT'' | ''SNAPSHOT'' | | |
| |
| Note that character case is not important on the command line input, ''owut'' converts internally to what the upgrade server requires. You can say ''snapshot'', ''SnapShot'' or ''SNAPSHOT'' or ''rc1'' or ''rC1'' and ''owut'' knows what to do. | |
| |
| === Forcing a build === | |
| |
| ''owut'''s normal behavior is to avoid doing unneeded work by stopping a build request when no changes are found. The ''--force'' option is used to override this and do a re-build and re-install of the current system. This might be useful if you have inadvertantly deleted packages or something similar, and can't easily figure out how to recover. | |
| |
| === Adding and removing packages === | |
| The ''--add'' and ''--remove'' options allow you to add packages to or remove packages from the build list submitted by ''owut'' to the ASU build server. ''owut'' verifies when you add a package that it is available on your platform; if already installed, the package is silently ignored. On the other hand, when removing a package, it must be installed or an error is reported. | |
| |
| For example, if you wanted to upgrade and simultaneously switch to the full versions of ''dnsmasq'' and ''tc'', you'd say this. | |
| <code> | |
| $ owut upgrade -r dnsmasq,tc-tiny -a dnsmasq-full,tc-full | |
| ... | |
| </code> | |
| |
| === Using a uci-defaults script === | |
| |
| If you are not familiar with ''uci-defaults'' (aka "first boot scripts"), you can read up here: [[docs:guide-developer:uci-defaults|UCI defaults]]. The underlying mechanism that implements ''owut'' (and ''auc'' and LuCI Attended Sysupgrade and Firmware Selector) builds is the Image Builder, so its description may also be useful: [[docs:guide-user:additional-software:imagebuilder#custom_files|Image builder - Custom files]]. | |
| |
| The ''--init-script'' option allows you to specify a ''uci-defaults'' script, which is executed at first boot. The ASU server takes your init-script source and places it in the image in ''/etc/uci-defaults/99-asu-defaults'' and there is no method to change this name. On immutable file systems (say ''squashfs''), this also results in the file being stored in ''/rom/etc/uci-defaults/99-asu-defaults'', which comes into play with LuCI Attended Sysupgrade. | |
| |
| Here's a comparison of ''owut'' with how other upgrade tools implement this functionality. | |
| * ''auc'' does not implement this ability. | |
| * Firmware Selector behaves identically to ''owut'' with its ''Script to run on first boot (uci-defaults)'' input field. | |
| * LuCI Attended Sysupgrade implements this by looking for ''/rom/etc/uci-defaults/99-asu-defaults'' and then relays the contents of that file implicitly. LuCI ASU's shortcoming is that it doesn't allow you to delete or change what's already there. ''owut'' makes this explicit, if you want the script included in your new image, then you must specify it when you request a build. | |
| |
| For a detailed example of use, see [[#persistent_uci-defaults|persistent uci-defaults]], below. | |
| |
| === Changing file system type === | |
| |
| On rare occasion, it might be desireable to change the file system type of an installation. Usually this is done on devices with expandable file systems, x86 and ARM SBCs, where the storage device is not fixed size FLASH memory (in fact, if you try to change the file system type on an all-in-one, FLASH-based device, the build will almost always fail). | |
| |
| But, say for example, you have an x86 with an SSD and want to switch from the current ''squashfs'' to use ''ext4''. Simply upgrade with the desired file system, and upon reboot you'll be running the targetted file system. | |
| |
| <code> | |
| $ owut upgrade --fstype ext4 | |
| Board-name generic | |
| Target x86/64 | |
| Root-FS-type squashfs <<< Installed | |
| Sys-type combined-efi | |
| Package-arch x86_64 | |
| Version-from SNAPSHOT r26504-d4acd05218 (kernel 6.6.32) | |
| Version-to SNAPSHOT r26733-2b6772c82c (kernel 6.6.34) | |
| Build-FS-type ext4 <<< Requested | |
| ... | |
| </code> | |
| |
| === Expanding root file system === | |
| |
| TODO The --rootfs-size option allows those of you with expandable file systems (I'm lookin' at you, x86 and armsr users) to specify the size of the root file system. On my x86 test box, I have built the last dozen ext4 images using 128, 256 and 512 MB root file systems with no issues. (The default varies depending on target, but is often 104 MB.) | |
| |
| |
| === List formatting === | |
| |
| TODO | |
| The owut list command uses the ''--format'' option that takes one of | |
| |
| fs-user - (the default) produces a package list for use by the Firmware Selector that contains only the top-level, user-installed package modifications. You'd copy'n'paste this after the default list in the FS 'Installed Packages' field. | |
| fs-all - produces a package list for FS containing all top-level packages, which you'd past over the values in the FS 'Installed Packages' field. | |
| config - produces a build .config snippet of user-installed, top-level packages that you can use when doing source builds. Each output line looks like CONFIG_PACKAGE_collectd-mod-thermal=y. | |
| |
| |
| ===== Configuration ===== | |
| |
| <WRAP round warning 100%> | |
| Note that although you can store any of the command line options in the config file, doing so with certain options may be confusing. Use your discretion when setting defaults: as an example, setting ''option force true'' is probably not a good idea. | |
| </WRAP> | |
| |
| Any of the command line options may be stored in the ''owut'' section of ''/etc/config/attendedsysupgrade''. For example, if you are using ''--rootfs-size 256'' on the command line on every upgrade, you could edit the config and add that option as follows: | |
| |
| <code> | |
| config server 'server' | |
| option url 'https://sysupgrade.openwrt.org' | |
| |
| config owut 'owut' | |
| option rootfs_size 256 | |
| </code> | |
| |
| Note that the dashes in the command line "long" option names are turned into underscores in the config option name, but beyond that the syntax is pretty much identical. To see the effect of your changes to the config, you can use the ''owut dump'' command and examine the ''options'' section of the output. | |
| |
| There is a single exception to the naming convention, command line ''--verbose'' maps to ''verbosity'' in the config file. The config file value is an integer, setting the default verbosity level, and every mention on the command line simply increments it: ''option verbosity 1'', then ''owut check -v -v -v'' results in an output verbosity of 4 (i.e., more output than you ever thought possible). | |
| |
| <code json> | |
| $ owut dump | head -20 | |
| { | |
| "version": "owut/2024.06.18-r1", | |
| "options": { | |
| "command": "dump", | |
| "version": null, | |
| "verbosity": 0, | |
| "keep": false, | |
| "force": false, | |
| "add": null, | |
| "remove": null, | |
| "init_script": null, | |
| "fstype": null, | |
| "rootfs_size": 256, | |
| "image": "/tmp/firmware.bin", | |
| "format": null, | |
| "_device": null | |
| }, | |
| ... | |
| </code> | |
| |
| ==== Configuration Examples ==== | |
| |
| === Stay on a given release === | |
| |
| You can force the default for the ''--version-to'' by setting this option in the config. Note that this is redundant with ''owut'''s default behavior (see the details of [[#selecting_a_version|Selecting a version]], above). | |
| |
| <code> | |
| config owut 'owut' | |
| option version_to '23.05' | |
| </code> | |
| |
| === Setting root file system size === | |
| |
| Before you set this one, please read about [[#expanding_root_file_system|expanding the root file system]], above, as this may not apply to your device. | |
| |
| For devices with expandable storage, typically x86 and ARM SBCs, you may find it useful to expand the root file system size thus allowing you to easily install a larger number of packages or store more persistent data. | |
| |
| <code> | |
| config owut 'owut' | |
| option rootfs_size 512 | |
| </code> | |
| |
| === Persistent uci-defaults === | |
| |
| |
| ===== References ===== | |
| |
| ASU server [[https://github.com/openwrt/asu|ASU server on github]] | |
| ''auc'' | |
| LuCI Attended Sysupgrade | |
| Firmware Selector | |