Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Next revisionBoth sides next revision | ||
| docs:guide-user:additional-software:imagebuilder [2021/07/28 06:49] – [Usage] formatting someothertime | docs:guide-user:additional-software:imagebuilder [2024/04/01 16:19] – [Debian 12+ / Ubuntu / Mint] sixx | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ====== Using the Image Builder ====== | ====== Using the Image Builder ====== | ||
| + | See also: | ||
| + | [[docs: | ||
| + | [[docs: | ||
| + | [[docs: | ||
| - | The Image Builder (previously called the Image Generator) is a pre-compiled environment suitable for creating custom images without the need for compiling them from source.\\ | + | The Image Builder (previously called the Image Generator) is a pre-compiled environment suitable for creating custom images without the need for compiling them from source. |
| - | It downloads pre-compiled packages and integrates them in a single flashable image.\\ | + | It downloads pre-compiled packages and integrates them in a single flashable image. |
| - | Doing so is useful if: | + | Doing so is useful if: |
| * you want to fit more packages in a small flash size | * you want to fit more packages in a small flash size | ||
| * you want to follow development snapshots | * you want to follow development snapshots | ||
| Line 10: | Line 14: | ||
| * you want to mass-flash dozens of devices and you need a specific firmware setup | * you want to mass-flash dozens of devices and you need a specific firmware setup | ||
| - | Alternative guides to achieving the same goal: | + | <WRAP important> |
| - | [[docs: | + | The Image Builder |
| - | [[docs: | + | When recent/ |
| - | + | ||
| - | Consider also removing packages if you have a device with very little firmware space: | + | |
| - | [[docs: | + | |
| - | + | ||
| - | < | + | |
| - | Imagebuilder | + | |
| </ | </ | ||
| - | ===== Frontends based on imagebuilder ===== | ||
| - | |||
| - | There are several tools that provide a frontend interface to the imagebuilder (either web-interface, | ||
| - | |||
| - | See [[docs: | ||
| ===== Prerequisites ===== | ===== Prerequisites ===== | ||
| - | :!: The Image Builder runs only in 64bit linux. You can however run a 64bit linux in VM (i.e. virtualbox) | + | <WRAP important> |
| - | :!: The Image Builder has similar prerequisites as [[docs: | + | * The Image Builder runs only in 64-bit Linux. You can however run a 64-bit Linux in PC or VM, e.g. VirtualBox, |
| + | | ||
| + | </ | ||
| Example dependencies in the most common distros: | Example dependencies in the most common distros: | ||
| - | ==== Arch / Manjaro | + | ==== Arch ==== |
| <code bash> | <code bash> | ||
| - | pacman -S --needed base-devel ncurses zlib gawk git gettext \ | + | sudo pacman -S --needed base-devel ncurses zlib gawk git gettext \ |
| openssl libxslt wget unzip python | openssl libxslt wget unzip python | ||
| </ | </ | ||
| - | ==== CentOS / Fedora ==== | + | ==== Fedora ==== |
| <code bash> | <code bash> | ||
| - | dnf install git gawk gettext ncurses-devel zlib-devel \ | + | sudo dnf install git gawk gettext ncurses-devel zlib-devel \ |
| openssl-devel libxslt wget which @c-development @development-tools \ | openssl-devel libxslt wget which @c-development @development-tools \ | ||
| - | @development-libs zlib-static which python3 | + | @development-libs zlib-static which python3 |
| </ | </ | ||
| - | ==== Debian / Ubuntu ==== | + | ==== Debian / Ubuntu |
| <code bash> | <code bash> | ||
| - | apt install build-essential | + | sudo apt install build-essential |
| - | zlib1g-dev gawk git gettext libssl-dev xsltproc rsync wget unzip python | + | gettext libssl-dev xsltproc rsync wget unzip python3 python3-distutils |
| </ | </ | ||
| + | ==== WSL ==== | ||
| + | This method is NOT OFFICIALLY supported. | ||
| + | But it works. | ||
| + | |||
| + | [[docs: | ||
| ===== Obtaining the Image Builder ===== | ===== Obtaining the Image Builder ===== | ||
| - | You can download an archive that contains the **Image Builder**, it is usually located in the same download page where you find the firmware image for your device.\\ | + | You can download an archive that contains the **Image Builder**, it is usually located in the same download page where you find the firmware image for your device. |
| - | For example, this is the page where you can download all firmware images for **ath79/ | + | |
| - | https:// | + | For example, this is the page where you can download all firmware images for **ath79/ |
| + | [[https:// | ||
| and you will find a **openwrt-imagebuilder-ath79-generic.Linux-x86_64.tar.xz** archive with the image builder in it. | and you will find a **openwrt-imagebuilder-ath79-generic.Linux-x86_64.tar.xz** archive with the image builder in it. | ||
| - | Also, it is always created by the build system because it is needed to create the image file. If the option " | + | Also, it is always created by the build system because it is needed to create the image file. |
| + | If the option " | ||
| - | ---- | + | ===== Usage ===== |
| + | <WRAP important> | ||
| + | All operations should be performed with a non-root user account. | ||
| + | </ | ||
| - | ===== Add Package Repositories (optional) ===== | + | Unpack |
| - | The **Image Generator** you download from the OpenWrt pages is already configured to download any non-default packages from official repositories.\\ | + | |
| - | The package sources are configured in the '' | + | |
| - | An example of the contents of the '' | + | <code bash> |
| + | tar -J -x -f openwrt-imagebuilder-*.tar.xz | ||
| + | cd openwrt-imagebuilder-*/ | ||
| + | </ | ||
| - | < | + | The image building can be customized with the following variables: |
| - | # src/gz %n http:// | + | |
| - | # src custom file:/// | + | |
| - | ## Remote package repositories | + | ^ Variable ^ Description ^ |
| - | src/gz openwrt_core http:// | + | | '' |
| - | src/gz openwrt_base http:// | + | | '' |
| - | src/gz openwrt_luci http:// | + | | '' |
| - | src/gz openwrt_packages http:// | + | | '' |
| - | src/gz openwrt_routing http:// | + | | '' |
| - | src/gz openwrt_telephony http://downloads.openwrt.org/ | + | | '' |
| + | |||
| + | Run '' | ||
| - | ## This is the local package repository, do not remove! | + | ==== Selecting profile ==== |
| - | src imagebuilder file: | + | The '' |
| + | |||
| + | <code bash> | ||
| + | PROFILE=" | ||
| </ | </ | ||
| - | The '' | + | Run '' |
| - | If you want to add a custom local repository, copy the < | + | ==== Selecting packages ==== |
| + | The '' | ||
| + | By default (empty PACKAGES variable) the Image Builder will create a minimal image with device-specific kernel and drivers, uci, ssh, switch, firewall, ppp and ipv6 support. | ||
| - | If you have custom repositories online, copy and modify the < | + | < |
| + | PACKAGES=" | ||
| + | </ | ||
| + | The example above will include pkg1, pkg2, pkg3, and exclude pkg4, pkg5, pkg6, note the " | ||
| - | NOTE: if you want to override | + | You don't need to list all dependencies of the packages |
| - | ===== Usage ===== | + | |
| - | All operations should | + | The list of currently installed packages on your device can be obtained with the following command: |
| - | **Unpacking: | + | < |
| - | < | + | echo $(opkg list-installed | sed -e "s/\s.*$//") |
| - | tar xJf openwrt*.tar.xz #changedir: cd openwrt... | + | |
| </ | </ | ||
| + | <WRAP important> | ||
| + | Many devices are limited in storage capacity and there is no guarantee that the build system will detect when you have added too many packages to fit into the device storage space, which may render the device unbootable if installed. | ||
| + | If in doubt, do not go overboard. | ||
| + | Use what you had installed on the device last as a guide or create a minimal image first, install it to the device and test what you would like to add first. | ||
| + | Consider removing unnecessary packages to [[docs: | ||
| + | </ | ||
| - | ==== make image ==== | + | In addition ABI versioned packages such as '' |
| - | **make | + | You may get compile errors when these are provided as packages. |
| + | To avoid issues you should omit them from image builder and let the correct versions be installed via package dependencies. | ||
| + | The '' | ||
| - | To change this not-so-useful default behavior | + | ==== Custom packages ==== |
| + | If there is a custom package or ipk you would prefer to use create a '' | ||
| - | * //'' | + | ==== Custom files ==== |
| - | * //'' | + | The '' |
| - | * //'' | + | This is especially useful if you need to change |
| - | * //'' | + | |
| - | * //'' | + | |
| - | * //'' | + | |
| - | (see also the makefile used, [[https:// | + | <code bash> |
| + | FILES=" | ||
| + | </code> | ||
| - | Example syntax: | + | The '' |
| - | < | + | |
| - | See the sections below for a more in-depth explanation. After the make command is finished, the generated | + | It is strongly recommended to use [[docs: |
| + | This helps minimize conflicts with auto-generated | ||
| - | ==== make help ==== | + | see: [[:docs:guide-user:additional-software:imagebuilder# |
| - | < | + | |
| - | help: This help text | + | |
| - | info: Show a list of available target profiles | + | |
| - | clean: | + | |
| - | image: Build an image (see below for more information). | + | |
| - | Building | + | ==== Building |
| - | By default | + | After you select the appropriate profile, packages and custom files, pass it to the ''make image'' command. |
| - | target profile and package set. You can use the following parameters | + | |
| - | to change that: | + | |
| - | | + | <code bash> |
| - | make image PACKAGES=" | + | make image \ |
| - | make image FILES=" | + | PROFILE=" |
| - | make image BIN_DIR="< | + | PACKAGES=" |
| - | make image EXTRA_IMAGE_NAME="< | + | FILES=" |
| - | make image DISABLED_SERVICES=" | + | DISABLED_SERVICES=" |
| + | </code> | ||
| - | Print manifest: | + | After the make command is finished, |
| - | List " | + | |
| - | You can use the following parameters: | + | |
| - | make manifest PROFILE=" | + | The built image will be found under the subdirectory '' |
| - | make manifest PACKAGES="<pkg1> [< | + | |
| + | ==== Cleaning up ==== | ||
| + | To clean up temporary build files and generated images, use the '' | ||
| + | |||
| + | ==== Examples ==== | ||
| + | The following example shows: | ||
| + | * Creating the directory for the configuration files. | ||
| + | * Using '' | ||
| + | * Generating an image for WL500GP with custom packages and '' | ||
| + | |||
| + | <code bash> | ||
| + | mkdir -p files/ | ||
| + | scp root@192.168.1.1:/ | ||
| + | scp root@192.168.1.1:/ | ||
| + | scp root@192.168.1.1:/ | ||
| + | make image \ | ||
| + | PROFILE=" | ||
| + | PACKAGES=" | ||
| + | FILES=" | ||
| + | DISABLED_SERVICES=" | ||
| </ | </ | ||
| + | ===== Troubleshooting ===== | ||
| + | - Did you run everything as a non-root user? | ||
| + | - Check the logged output, are there package issues (conflicts, improper names)? | ||
| + | - Check the logged output, did you exceed maximum space? | ||
| + | - Check the logged output, are there other obvious errors? | ||
| + | - Wait a few hours/ | ||
| + | - Verify you have a supported OS, prerequisites, | ||
| - | The built image will be found under the subdirectory '' | + | ===== Extras |
| - | ==== PROFILE Variable | + | The topics below go beyond simple usage and aimed at developers and advanced users. |
| - | Syntax:< | + | |
| - | === Pre-defined Profiles | + | ==== Detailed help ==== |
| - | Run '' | + | See also: [[https:// |
| - | Example output from '' | + | |
| + | Getting detailed help: | ||
| - | == ar71xx-generic Profiles == | ||
| < | < | ||
| + | # make help | ||
| + | |||
| + | Available Commands: | ||
| + | help: This help text | ||
| + | info: Show a list of available target profiles | ||
| + | clean: | ||
| + | image: | ||
| + | |||
| + | Building images: | ||
| + | By default 'make image' will create an image with the default | ||
| + | target profile and package set. You can use the following parameters | ||
| + | to change that: | ||
| + | |||
| + | make image PROFILE="< | ||
| + | make image PACKAGES="< | ||
| + | make image FILES="< | ||
| + | make image BIN_DIR="< | ||
| + | make image EXTRA_IMAGE_NAME="< | ||
| + | make image DISABLED_SERVICES="< | ||
| + | make image ADD_LOCAL_KEY=1 # store locally generated signing key in built images | ||
| + | |||
| + | Print manifest: | ||
| + | List " | ||
| + | You can use the following parameters: | ||
| + | |||
| + | make manifest PROFILE="< | ||
| + | make manifest PACKAGES="< | ||
| + | make manifest STRIP_ABI=1 # remove ABI version from printed package names | ||
| + | </ | ||
| + | |||
| + | ==== Available profiles ==== | ||
| + | Listing available profiles: | ||
| + | |||
| + | < | ||
| + | # make info | ||
| + | |||
| Available Profiles: | Available Profiles: | ||
| Line 167: | Line 240: | ||
| rp-n53: | rp-n53: | ||
| Asus RP-N53 | Asus RP-N53 | ||
| - | Packages: | + | Packages: |
| rt-n14u: | rt-n14u: | ||
| Asus RT-N14u | Asus RT-N14u | ||
| - | Packages: | + | Packages: |
| whr-1166d: | whr-1166d: | ||
| Buffalo WHR-1166D | Buffalo WHR-1166D | ||
| - | Packages: | + | Packages: |
| whr-300hp2: | whr-300hp2: | ||
| Buffalo WHR-300HP2 | Buffalo WHR-300HP2 | ||
| Packages: | Packages: | ||
| - | -and many many more- | + | ... |
| </ | </ | ||
| - | After you find the appropriate profile pass it to the '' | + | ==== Building |
| + | It is possible to use a buildroot to create your own Image Builder and integrate in it all packages so it will be able to generate images without downloading packages. | ||
| - | For example, if we wanted | + | In the graphical configuration, select " |
| + | You can [[docs: | ||
| - | < | + | Don't call '' |
| - | $ make image PROFILE=rt-n14u | + | |
| - | </code> | + | |
| - | ==== PACKAGES Variable | + | ==== Adding package repositories |
| - | The '' | + | The Image Builder you download from the OpenWrt pages is already configured to download any non-default packages from official repositories. |
| + | The package sources are configured in the '' | ||
| + | Sources are specified in //opkg// native config format. | ||
| + | This can be either | ||
| - | Syntax:< | + | An example of the contents of the '' |
| - | The example above will include pkg1, pkg2, pkg3, and exclude pkg4, pkg5, pkg6, note the "-" before each excluded package. | + | |
| - | You don't need to list all dependencies of the packages you need in this list, the Image Generator uses '' | + | <code bash> |
| + | ## Place your custom repositories here, they must match the architecture | ||
| + | # src/gz %n http:// | ||
| + | # src custom file:/// | ||
| - | **Tip:** The list of currently installed | + | ## Remote package repositories |
| - | < | + | src/gz openwrt_core http:// |
| + | src/gz openwrt_base http:// | ||
| + | src/gz openwrt_luci http:// | ||
| + | src/gz openwrt_packages http:// | ||
| + | src/gz openwrt_routing http:// | ||
| + | src/gz openwrt_telephony http:// | ||
| - | <WRAP center round important 60%> | + | ## This is the local package repository, do not remove! |
| - | Many devices are limited in storage capacity and there is no guarantee that the build system will detect when you have added too many packages to fit into the device storage space, which may render the device unbootable if installed. If in doubt, do not go overboard. Use what you had installed on the device last as a guide or create a minimal image first, install it to the device and test what you would like to add first. | + | src imagebuilder file: |
| - | </WRAP> | + | </code> |
| - | ==== FILES Variable ==== | + | The '' |
| - | The '' | + | |
| - | Syntax: | + | If you want to add a custom local repository, copy the '' |
| - | < | + | If you have problems with using you local repository because the " |
| - | **Note:** The '' | + | If you have custom repositories online, copy and modify the '' |
| + | NOTE: if you want to override packages coming from an existing feed, you must write your custom feed ABOVE the line of the package feed containing the packages you want to override, as shown in the examples above. | ||
| - | ==== Examples | + | ==== Restricting root access |
| - | The following example shows: | + | Create a non-privileged admin user and lock root password. |
| + | Configure privilege elevation with sudo. | ||
| + | Set up key-based authentication and disable password authentication for Dropbear. | ||
| - | -Creating the directory for the configuration files | + | <code bash> |
| - | -Using '' | + | mkdir -p files/ |
| - | -Generating an image for WL500GP with custom packages and '' | + | cat << " |
| - | + | USER_NAME=" | |
| - | <code>mkdir -p files/etc/config | + | USER_SSHPUB=" |
| - | scp root@192.168.1.1: | + | USER_SHELL=" |
| - | scp root@192.168.1.1: | + | SUDO_USER=" |
| - | scp root@192.168.1.1:/etc/config/ | + | SUDO_GROUP=" |
| - | make image PROFILE=wl500gp | + | groupadd -r " |
| + | useradd -m -G " | ||
| + | passwd -l " | ||
| + | cat << EOI > /etc/sudoers.d/00-custom | ||
| + | %${SUDO_GROUP} ALL=(ALL) ALL | ||
| + | EOI | ||
| + | USER_HOME=" | ||
| + | mkdir -p " | ||
| + | cat << EOI > " | ||
| + | ${USER_SSHPUB} | ||
| + | EOI | ||
| + | uci set dropbear.@dropbear[0].PasswordAuth=" | ||
| + | uci set dropbear.@dropbear[0].RootPasswordAuth=" | ||
| + | uci commit dropbear | ||
| + | /etc/init.d/dropbear restart | ||
| + | EOF | ||
| + | make image \ | ||
| + | FILES=" | ||
| + | PACKAGES=" | ||
| </ | </ | ||
| - | ===== Cleanup | + | ==== Adding/ |
| - | To clean up temporary build files and generated images, use the **make clean** command. | + | |
| - | ====== Advanced Topics ====== | + | <WRAP important> |
| - | The topics | + | Examples |
| + | </ | ||
| - | ===== Adding/ | + | The image building |
| - | The image generation | + | If you add a new profile without also adding an appropriate macro to the image-generation Makefile, no suitable firmware file will get generated when using the custom profile. |
| - | :!: Make sure to remove | + | Remove |
| The location of the profiles for the pre-compiled package for // | The location of the profiles for the pre-compiled package for // | ||
| - | Remarkably, all that needs to be done to add a new profile, is to add a new file to the // | + | Remarkably, all that needs to be done to add a new profile, is to add a new file to the // |
| + | //While this may have been the case in earlier releases, for 17.01, it appears that manual editing of '' | ||
| - | Here is what the // | + | Here is what the // |
| - | < | + | < |
| - | < | + | |
| define Profile/ | define Profile/ | ||
| - | | + | NAME: |
| - | PACKAGES: | + | PACKAGES: |
| endef | endef | ||
| Line 254: | Line 357: | ||
| $(eval $(call Profile, | $(eval $(call Profile, | ||
| </ | </ | ||
| - | </ | ||
| - | Alternately edit the hidden .profile.mk file at the top level directory of the image builder | + | Alternately edit the hidden .profile.mk file at the top level directory of the image builder and manually add the names of the desired packages to be added to the output image. |
| + | An "ls -a" will reveal the files hidden in the various directories. | ||
| - | ===== Remove | + | ==== Removing |
| - | < | + | <WRAP important> |
| This is not a standard feature of the Image Builder. | This is not a standard feature of the Image Builder. | ||
| - | It is highly recommended that you test file removal prior to incorporating such changes at the image builder level or that you have low level means to recover a device before attempting this type of mod. As bricking / non booting may result. | + | It is highly recommended that you test file removal prior to incorporating such changes at the image builder level or that you have low level means to recover a device before attempting this type of mod, as bricking / non booting may result. |
| Note that it requires patching of the '' | Note that it requires patching of the '' | ||
| + | |||
| + | It is based on older Chaos Calmer era code... and not applicable to modern ImageBuilders but useful as a reference... | ||
| </ | </ | ||
| - | 1. Create file ' | + | Create file '' |
| - | < | + | |
| + | < | ||
| / | / | ||
| / | / | ||
| Line 275: | Line 381: | ||
| </ | </ | ||
| - | 2. Patch Makefile | + | Patch Makefile: |
| - | < | + | |
| + | < | ||
| ifneq ($(USER_FILES), | ifneq ($(USER_FILES), | ||
| | | ||
| Line 285: | Line 392: | ||
| + @echo Remove useless files | + @echo Remove useless files | ||
| + | + | ||
| - | + while read filename; do \ | + | + while read filename; do \ |
| - | + rm -rfv " | + | + rm -rfv " |
| + done < $(FILES_REMOVE); | + done < $(FILES_REMOVE); | ||
| +endif | +endif | ||
| Line 294: | Line 401: | ||
| </ | </ | ||
| - | 3. Rebuild firmware | + | Rebuild firmware: |
| - | < | + | |
| - | # make image \ | + | |
| - | PROFILE=tlwr841 \ | + | |
| - | PACKAGES=" | + | |
| - | FILES_REMOVE=" | + | |
| - | </ | + | |
| - | + | <code bash> | |
| - | ===== Building the Image Generator with all packages inside ===== | + | make image \ |
| - | It is possible to build the Image Generator and integrate in it all packages so it will be able to generate images without downloading packages: | + | PROFILE="tlwr841" |
| - | + | PACKAGES=" | |
| - | In the graphical configuration, | + | FILES_REMOVE=" |
| - | Then build the image, including '' | + | |
| - | + | ||
| - | Enabling '' | + | |
| - | + | ||
| - | < | + | |
| </ | </ | ||
| - | |||
| - | **Note:** Don't call '' | ||
| - | |||