Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Next revisionBoth sides next revision
docs:guide-user:installation:sysupgrade.owut [2024/07/02 16:32] – [Overview] link to attended sysupgrade overview efahlgrendocs:guide-user:installation:sysupgrade.owut [2024/08/18 20:34] – [List formatting] efahlgren
Line 22: Line 22:
 You can specify these options on the command line, or you can use the standard OpenWrt ''config'' system to store these values and avoid having to remember and retype them on every upgrade. You can specify these options on the command line, or you can use the standard OpenWrt ''config'' system to store these values and avoid having to remember and retype them on every upgrade.
  
 +===== Installation and Upgrading =====
 +
 +''owut'' is a standard, optional OpenWrt package, available on all platforms supported by SNAPSHOT or release builds from 24.x and later.
 +<code>
 +opkg update  &&  opkg install owut
 +</code>
 +
 +If you find that you need a feature that is not in your current version of ''owut'', you can upgrade as follows.
 +
 +<code>
 +opkg update  &&  opkg upgrade owut
 +</code>
 +
 +There is usually no need to specifically upgrade ''owut'', as once installed ''owut'' will be upgraded along with everything else whenever you do a full firmware upgrade.
 +
 +If you have questions about installation or configuration, [[https://forum.openwrt.org/t/owut-openwrt-upgrade-tool/200035|post on the support forum thread]].
 ===== Quick Start ===== ===== Quick Start =====
  
 If your goal is simply to upgrade your router's current firmware, staying on the same version (e.g., 23.05 or SNAPSHOT), then just use the ''upgrade'' command.  If there are any problems uncovered by the various pre-build checks, or if an error is detected during the build, then the upgrade will abort with a message indicating the issue. If your goal is simply to upgrade your router's current firmware, staying on the same version (e.g., 23.05 or SNAPSHOT), then just use the ''upgrade'' command.  If there are any problems uncovered by the various pre-build checks, or if an error is detected during the build, then the upgrade will abort with a message indicating the issue.
  
-<WRAP round warning 100%>+<WRAP round important 100%>
 Your first step is always... Your first step is always...
  
Line 35: Line 51:
 </WRAP> </WRAP>
  
-If ''owut'' finds that no changes were made since you last upgrade, it will tell you this and stop.  You can override this behavior with the ''--force'' option, which will proceed to build and re-install your current version, keeping all configuration.+If ''owut'' finds that package downgrades, or no changes were made since your last upgrade, it will tell you this and stop.  You can re-run the command with the ''--force'' option, which will proceed with the build and install, keeping all configuration.
  
-Note that ''owut'' is currently available only on SNAPSHOT.+Note that ''owut'' is currently available only on SNAPSHOT or versions 24.x and later.
  
 <code> <code>
Line 130: Line 146:
  
 <code bash> <code bash>
-$ owut --help +$ owut -h 
-owut - OpenWrt Upgrade Tool version 2024.06.18-r1 (/root/bin/owut)+owut - OpenWrt Upgrade Tool %%VERSION%% (/root/bin/owut)
  
 owut is an upgrade tool for OpenWrt. owut is an upgrade tool for OpenWrt.
  
-Usage: owut COMMAND [-V VERSION] [-v] [-k] [--force] [-a ADD] [-r REMOVE] [-I INIT_SCRIPT] [-F FSTYPE] [-S ROOTFS_SIZE] [-i IMAGE] [-f FORMAT]+Usage: owut COMMAND [-V VERSION_TO] [-v] [-k] [--force] [-a ADD] [-r REMOVE] [-I INIT_SCRIPT] [-F FSTYPE] [-S ROOTFS_SIZE] [-i IMAGE] [-f FORMAT] [-p PRE_INSTALL]
   -h/--help            - Show this message and quit.   -h/--help            - Show this message and quit.
   --version            - Show the program version and terminate.   --version            - Show the program version and terminate.
Line 150: Line 166:
     dump     - Collect all resources and dump internal data structures.     dump     - Collect all resources and dump internal data structures.
  
-  -V/--version-to VERSION - Specify the target version, defaults to installed version.+  -V/--version-to VERSION_TO - Specify the target version, defaults to installed version.
   -v/--verbose         - Print various diagnostics.  Repeat for even more output.   -v/--verbose         - Print various diagnostics.  Repeat for even more output.
   -k/--keep            - Save all downloaded working files.   -k/--keep            - Save all downloaded working files.
-  --force              - Force download when there are no changes detected.+  --force              - Force a build even when there are downgrades or no changes.
   -a/--add ADD         - Comma-separated list of new packages to add to build list.   -a/--add ADD         - Comma-separated list of new packages to add to build list.
   -r/--remove REMOVE   - Comma-separated list of installed packages to remove from build list.   -r/--remove REMOVE   - Comma-separated list of installed packages to remove from build list.
Line 161: Line 177:
   -i/--image IMAGE     - Image name for download, verify, install and upgrade.   -i/--image IMAGE     - Image name for download, verify, install and upgrade.
   -f/--format FORMAT   - Format for 'list' output (fs-user, fs-all, config).   -f/--format FORMAT   - Format for 'list' output (fs-user, fs-all, config).
 +  -p/--pre-install PRE_INSTALL - Script to exec just prior to launching final sysupgrade.
 +
 +Full documentation
 +  https://openwrt.org/docs/guide-user/installation/sysupgrade.owut
 +
 +Questions and discussion
 +  https://forum.openwrt.org/t/owut-openwrt-upgrade-tool/200035
 +
 +Issues and bug reports
 +  https://github.com/efahl/owut/issues
 </code> </code>
  
Line 167: Line 193:
 ^ Sub-Command ^ Description ^ ^ Sub-Command ^ Description ^
 | ''check''    | Downloads all resource files, collects the metadata from the device and the resources, and displays a report on everything found.  This includes available version upgrades on all packages, availability of installed packages, listing of all package build breakages, and so on.  At the end of the report, you'll see an indication as to whether it is possible to upgrade or not. | | ''check''    | Downloads all resource files, collects the metadata from the device and the resources, and displays a report on everything found.  This includes available version upgrades on all packages, availability of installed packages, listing of all package build breakages, and so on.  At the end of the report, you'll see an indication as to whether it is possible to upgrade or not. |
-| ''list''     | This sub-command allows you to generate the list of packages installed on your device.  This list is tailored for use with either the [[https://firmware-selector.openwrt.org/|OpenWrt Firmware Selector]] or for use with source builds.  For more details, see the ''--format'' option description, below. |+| ''list''     | This sub-command allows you to generate the list of packages installed on your device.  This list is tailored for use with either the [[https://firmware-selector.openwrt.org/|OpenWrt Firmware Selector]] or for use with source builds.  For more details, see the ''--format'' [[#list_formatting|option description]], below. |
 | ''blob''     | Display the json blob for the ASU build request.  Mostly useful for debugging and satisfying your curiosity. | | ''blob''     | Display the json blob for the ASU build request.  Mostly useful for debugging and satisfying your curiosity. |
 | ''download'' | Build, download and ''verify'' an image.  Used to create an image that you may then archive off-system, before subsequently ''owut install''ing it. | | ''download'' | Build, download and ''verify'' an image.  Used to create an image that you may then archive off-system, before subsequently ''owut install''ing it. |
Line 180: Line 206:
  
 ^ Option ^ Default ^ Description ^ ^ Option ^ Default ^ Description ^
-| ''-V/--version-to VERSION''    | newest version on current branch | Search for updates for this version or branch. |+| ''-V/--version-to VERSION''    | newest version on current branch | Search for updates for this version or branch.  [[#list_formatting|Detailed description.]]|
 | ''-v/--verbose''         | - | Print various diagnostics and operational messages; repeat for even more output. | | ''-v/--verbose''         | - | Print various diagnostics and operational messages; repeat for even more output. |
 | ''-k/--keep''            | false | Save all downloaded working files, primarily for diagnostics and debugging.  Look in ''/tmp/'' after using this option; turn on ''-v'' to watch as they are saved. | | ''-k/--keep''            | false | Save all downloaded working files, primarily for diagnostics and debugging.  Look in ''/tmp/'' after using this option; turn on ''-v'' to watch as they are saved. |
-| ''--force''              | false | Force ''download'' or ''upgrade'' when there are no changes detected. | +| ''--force''              | false | Force ''download'' or ''upgrade'' when there are package downgrades, or when there are no changes detected. [[#forcing_a_build|Detailed description.]] 
-| ''-a/--add ADD''         | none | Comma-separated list of new packages to add to build list. | +| ''-a/--add ADD''         | none | Comma-separated list of new packages to add to build list.  [[#about_adding_packages|Detailed description.]] 
-| ''-r/--remove REMOVE''   | none | Comma-separated list of installed packages to remove from build list. | +| ''-r/--remove REMOVE''   | none | Comma-separated list of installed packages to remove from build list. [[#about_removing_packages|Detailed description.]] 
-| ''-I/--init-script INIT_SCRIPT'' | none | Path to uci-defaults script to run on first boot ('-' use stdin). | +| ''-I/--init-script INIT_SCRIPT'' | none | Path to uci-defaults script to run on first boot ('-' use stdin).  [[#using_a_uci-defaults_script|Detailed description.]] 
-| ''-F/--fstype FSTYPE''   | current | Desired root file system type.  May be one of ''squashfs'', ''ext4'', ''ubifs'' or ''jffs2'' depending on platform constraints. | +| ''-F/--fstype FSTYPE''   | current | Desired root file system type.  May be one of ''squashfs'', ''ext4'', ''ubifs'' or ''jffs2'' depending on platform constraints.  [[#changing_file_system_type|Detailed description.]] 
-| ''-S/--rootfs-size ROOTFS_SIZE'' | current | Root file system size in MB (1-1024). |+| ''-S/--rootfs-size ROOTFS_SIZE'' | current | Root file system size in MB (1-1024). [[#expanding_root_file_system|Detailed description.]] |
 | ''-i/--image IMAGE''     | ''/tmp/firmware.bin'' | The image name used for the ''download'', ''verify'', ''install'' and ''upgrade'' sub-commands. | | ''-i/--image IMAGE''     | ''/tmp/firmware.bin'' | The image name used for the ''download'', ''verify'', ''install'' and ''upgrade'' sub-commands. |
-| ''-f/--format FORMAT''   | ''fs-user'' | Format for ''list'' output.  Valid formats are ''fs-user'', ''fs-all'' and ''config''. |+| ''-f/--format FORMAT''   | ''fs-user'' | Format for ''list'' output.  Valid formats are ''fs-user'', ''fs-all'' and ''config'' [[#list_formatting|Detailed description.]] | 
 +| ''-p/--pre-install PRE_INSTALL'' | none | Path to a user-defined script that will be run after image verification and before the actual installation of the image.  [[#pre-install_script|Detailed description.]] |
  
 ==== Selecting a version ==== ==== Selecting a version ====
  
 +=== When you don't specify a version ===
 When you do not explicitly specify a ''--version-to'' value, ''owut'' looks for the newest version of the installed branch and sets that as the target version. When you do not explicitly specify a ''--version-to'' value, ''owut'' looks for the newest version of the installed branch and sets that as the target version.
  
Line 204: Line 232:
 | ''23.05.4''     | ''23.05.4'' | | ''23.05.4''     | ''23.05.4'' |
  
-The exception to this is if the installed version is a ''SNAPSHOT'', either release or main, in which case, version-to is set to the installed version.+The exception to this is if the installed version is a ''SNAPSHOT'', either release or main, in which case, the version-to target remains at the installed version.
  
 ^ Installed Version ^ Target ^ ^ Installed Version ^ Target ^
Line 210: Line 238:
 | ''23.05-SNAPSHOT'' | ''23.05-SNAPSHOT'' | | ''23.05-SNAPSHOT'' | ''23.05-SNAPSHOT'' |
 | ''SNAPSHOT''       | ''SNAPSHOT''       | | ''SNAPSHOT''       | ''SNAPSHOT''       |
 +
 +=== When you do specify a version ===
  
 When you do specify ''--version-to'', it must name a valid version or branch.  If you provide an invalid value, ''owut'' will show you all the available versions (or you can do this manually with ''owut versions''). When you do specify ''--version-to'', it must name a valid version or branch.  If you provide an invalid value, ''owut'' will show you all the available versions (or you can do this manually with ''owut versions'').
Line 221: Line 251:
 | ''--version-to snapshot''       | ''SNAPSHOT''       | | ''--version-to snapshot''       | ''SNAPSHOT''       |
  
-If you specify only a branch (i.e., a version number without the final "dot value"), then the version-to is set to the latest release on that branch.+If you specify only a branch (i.e., a version number without the final "dot value"), then the version-to target is set to the latest release on that branch.
  
 ^ User specifies ^ Target ^ ^ User specifies ^ Target ^
Line 228: Line 258:
 | ''--version-to SNAPSHOT'' | ''SNAPSHOT'' | | ''--version-to SNAPSHOT'' | ''SNAPSHOT'' |
  
-Note that character case is not important on the command line input, ''owut'' converts internally to what the upgrade server requires.  You can say ''snapshot'', ''SnapShot'' or ''SNAPSHOT'' or ''rc1'' or ''rC1'' and ''owut'' knows what to do.+Note that character case is not important in naming the version, ''owut'' converts internally to what the upgrade server requires.  You can say ''snapshot'', ''SnapShot'' or ''SNAPSHOT'' or ''rc1'' or ''rC1'' and ''owut'' knows what to do.
  
 ==== Forcing a build ==== ==== Forcing a build ====
  
 ''owut'''s normal behavior is to avoid doing unneeded work by stopping a build request when no changes are found.  The ''--force'' option is used to override this and do a re-build and re-install of the current system.  This might be useful if you have inadvertently deleted packages or something similar, and can't easily figure out how to recover. ''owut'''s normal behavior is to avoid doing unneeded work by stopping a build request when no changes are found.  The ''--force'' option is used to override this and do a re-build and re-install of the current system.  This might be useful if you have inadvertently deleted packages or something similar, and can't easily figure out how to recover.
 +
 +When ''owut'' detects downgrades in packages, it will indicate this by coloring the new version number red in the ''Package version changes:'' list and report the number of downgrades at the bottom of the list.  This also causes ''owut'' to stop processing any ''download'' or ''upgrade'' in progress, unless you specify ''--force'' option.
  
 ==== Adding and removing packages ==== ==== Adding and removing packages ====
Line 265: Line 297:
 ==== Using a uci-defaults script ==== ==== Using a uci-defaults script ====
  
-<WRAP round warning 100%>+<WRAP round important 100%>
 **Do not put sensitive information in the ''init-script'' file.** **Do not put sensitive information in the ''init-script'' file.**
  
Line 331: Line 363:
 <code> <code>
 $ owut upgrade --fstype ext4 $ owut upgrade --fstype ext4
-Board-name     generic 
 Target         x86/64 Target         x86/64
 +Profile        generic
 Root-FS-type   squashfs                   <<<  Installed Root-FS-type   squashfs                   <<<  Installed
 Sys-type       combined-efi Sys-type       combined-efi
Line 343: Line 375:
  
 ==== Expanding root file system ==== ==== Expanding root file system ====
 +
 +<WRAP round alert 100%>
 +Changing your root file system size often causes ''owut'''s final installation performed by ''sysupgrade'' to lose your configuration, so be prepared to recover with a backup.  Note that if you lose ''/etc/config/network'', the LAN IP of the device may change, so think about how you'll attach to the device before you proceed.
 +</WRAP>
  
 The ''--rootfs-size'' option allows those devices with expandable file systems (again, typically x86 and ARM SBCs) to specify the size of the root file system.  The default value varies depending on target, but is often 104 MB and the ''--rootfs-size'' option allow you to increase that up to 1024 MB, thus allowing more or bigger packages to be installed. The ''--rootfs-size'' option allows those devices with expandable file systems (again, typically x86 and ARM SBCs) to specify the size of the root file system.  The default value varies depending on target, but is often 104 MB and the ''--rootfs-size'' option allow you to increase that up to 1024 MB, thus allowing more or bigger packages to be installed.
Line 362: Line 398:
   * ''fs-all'' - produces a package list for FS containing all top-level packages, which you'd paste //over// the values in the FS 'Installed Packages' field.   * ''fs-all'' - produces a package list for FS containing all top-level packages, which you'd paste //over// the values in the FS 'Installed Packages' field.
   * ''config'' - produces a build ''.config'' snippet of user-installed, top-level packages that you can use when doing source builds. Each output line looks like ''CONFIG_PACKAGE_collectd-mod-thermal=y''.   * ''config'' - produces a build ''.config'' snippet of user-installed, top-level packages that you can use when doing source builds. Each output line looks like ''CONFIG_PACKAGE_collectd-mod-thermal=y''.
 +
 +The ''fs-*'' options generate lists in the Image Builder syntax, where mentioning a package name adds it to the list, and prefixing a package name with a dash removes it.
 +
 +For example, if you have installed ''dnsmasq-full'', then the default ''dnsmasq'' basic package must be removed.  That would look like this (trimmed down for clarity):
 +
 +<code bash>
 +$ owut list
 +... dnsmasq-full ... -dnsmasq ...
 +</code>
 +
 +There are several packages provided by the defaults that are named using a generic package name, and actually provided by something with a different name, ''nftables'' is a prominent one (it is provided by either ''nftables-json'' or ''nftables-nojson'', there is no ''nftables'' package).  ''owut'' appears to be removing it, but it really is just saying, "use the default, whatever that is" In the following example, we see only the ''-nftables'' removal, but not default package ''nftables-json'' as it will be added implicitly.
 +
 +<code bash>
 +$ opkg whatprovides nftables
 +What provides nftables
 +    nftables-json
 +
 +$ owut list
 +... -nftables ...
 +</code>
 +
 +This can happen for other files that appear to be "deleted from nowhere" due to dependencies.  As an example of this, if you are using ''luci-ssl-openssl'', then the list output will contain ''-libustream-mbedtls'' which would otherwise be added by defaults resulting in an "impossible package selection" error.
 +
 +You'll often see other evidence of these mappings when using ''check'' in the default package analysis results:
 +<code>
 +$ owut check
 +...
 +Default package analysis:
 +  Default                        Provided-by
 +  dnsmasq                        dnsmasq-full
 +  kmod-dwmac-intel               not installed
 +  nftables                       nftables-json
 +...
 +</code>
 +
 +==== Pre-install Script ====
 +
 +''owut'' has a hook between image verification and image install, allowing you to do automatic backups, archiving of build artifacts or any other final modifications to the system prior to the installation.  Use the ''--pre-install /path/to/script'' option from the command line, or add ''option pre_install '/path/to/script''' to the ''owut'' section of the config file.
 +
 +The standard installation of ''owut'' provides several examples in ''/etc/owut.d/pre-install.sh''.
 +
 +Note that the directory ''/etc/owut.d/'' is part of the standard sysupgrade backup, so any files you store there will become part of system backups and persist across upgrades (assuming you "keep config").
 +
  
 ===== Configuration ===== ===== Configuration =====
  
-<WRAP round warning 100%>+<WRAP round important 100%>
 Note that although you can store any of the command line options in the config file, doing so with certain options may be confusing.  Use your discretion when setting defaults: as an example, setting ''option force true'' is probably not a good idea. Note that although you can store any of the command line options in the config file, doing so with certain options may be confusing.  Use your discretion when setting defaults: as an example, setting ''option force true'' is probably not a good idea.
 </WRAP> </WRAP>
Line 399: Line 478:
   "rootfs_size": 256,   "rootfs_size": 256,
   "image": "/tmp/firmware.bin",   "image": "/tmp/firmware.bin",
-  "format": null, +  "format": null
-  "_device": null+
 }, },
 ... ...
Line 435: Line 513:
   - Create your init-script in a persistent location. <code>   - Create your init-script in a persistent location. <code>
 $ mkdir /etc/owut.d/ $ mkdir /etc/owut.d/
-$ echo "# My first boot script." > /etc/owut.d/00_custom-init.sh+$ echo "# My first boot script." > /etc/owut.d/custom-init.sh
 </code> </code>
   - Make sure it is carried over in all ''sysupgrade'' backups. <code>   - Make sure it is carried over in all ''sysupgrade'' backups. <code>
 $ echo "/etc/owut.d/" >> /etc/sysupgrade.conf $ echo "/etc/owut.d/" >> /etc/sysupgrade.conf
 $ sysupgrade -l | grep custom $ sysupgrade -l | grep custom
-/etc/owut.d/00_custom-init.sh+/etc/owut.d/custom-init.sh
 </code> </code>
   - Configure ''owut'' to include the script into your builds. <code>   - Configure ''owut'' to include the script into your builds. <code>
-$ uci set attendedsysupgrade.owut.init_script=/etc/owut.d/00_custom-init.sh+$ uci set attendedsysupgrade.owut.init_script=/etc/owut.d/custom-init.sh
 $ uci commit $ uci commit
 $ uci show attendedsysupgrade.owut $ uci show attendedsysupgrade.owut
 attendedsysupgrade.owut=owut attendedsysupgrade.owut=owut
-attendedsysupgrade.owut.init_script='/etc/owut.d/00_custom-init.sh'+attendedsysupgrade.owut.init_script='/etc/owut.d/custom-init.sh'
 </code> </code>
   - Verify that everything is set up properly.  Your script should appear in the json as the ''defaults'' entry. <code>   - Verify that everything is set up properly.  Your script should appear in the json as the ''defaults'' entry. <code>
Line 464: Line 542:
     "busybox",     "busybox",
     ... a bunch more packages ...     ... a bunch more packages ...
-        "ucode-mod-uloop",+    "ucode-mod-uloop",
     "urandom-seed",     "urandom-seed",
     "urngd"     "urngd"
  • Last modified: 2024/11/29 17:07
  • by efahlgren