Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
docs:guide-developer:toolchain:use-buildsystem [2021/10/15 08:26] – ↷ Page moved from docs:guide-developer:build-system:use-buildsystem to docs:guide-developer:toolchain:use-buildsystem bobafetthotmaildocs:guide-developer:toolchain:use-buildsystem [2024/10/16 18:12] (current) – [Build system usage] jannoke
Line 1: Line 1:
 ====== Build system usage ====== ====== Build system usage ======
 <WRAP important> <WRAP important>
-  * Do everything as a normal user: do not log in as root and do not use sudo. +  * Do everything as an unprivileged user, not root, without sudo. 
-  * Do not build in a directory that has spaces in its full path.+  * Make sure there are no spaces in the full path to the build directory.
 </WRAP> </WRAP>
  
-Typical build steps in a [[docs:guide-developer:toolchain:install-buildsystem|set up build system]]:+<WRAP important> 
 +There is an issue affecting the current OpenWrt source tree (from at least 21.02 onwards): OpenWrt images built in certain setups will succeed, but they will hang on boot if installed on a device. To work around this issue, please follow the instructions posted [[https://github.com/openwrt/openwrt/issues/9545|here]] in the section titled "workaround" **before checking out the source tree**. 
 +</WRAP> 
 + 
 +After [[docs:guide-developer:toolchain:install-buildsystem|installing your build system's prerequisites]], these are typical build steps:
  
 <code bash> <code bash>
 # Download and update the sources # Download and update the sources
-git clone https://git.openwrt.org/openwrt/openwrt.git openwrt+git clone https://git.openwrt.org/openwrt/openwrt.git
 cd openwrt cd openwrt
 git pull git pull
Line 16: Line 20:
 git branch -a git branch -a
 git tag git tag
-git checkout v21.02.0+git checkout v23.05.5
  
 # Update the feeds # Update the feeds
Line 22: Line 26:
 ./scripts/feeds install -a ./scripts/feeds install -a
  
-# Configure the firmware image and the kernel+# Configure the firmware image
 make menuconfig make menuconfig
-make kernel_menuconfig+ 
 +# Optional: configure the kernel (usually not required) 
 +# Don't, unless have a strong reason to 
 +make -j$(nproc) kernel_menuconfig
  
 # Build the firmware image # Build the firmware image
-make -j $(nproc) defconfig download clean world+make -j$(nproc) defconfig download clean world
 </code> </code>
  
Line 93: Line 100:
   - Create a dir: ''mkdir -p <buildroot>/my_packages/<section>/<category>/<package_name>''.\\ Replace the ''<package_name>'' with the name of your package.\\ e.g. ''mkdir -p my_packages/net/network/rpcbind''.\\ The section and category can be found in the ''Makefile''.   - Create a dir: ''mkdir -p <buildroot>/my_packages/<section>/<category>/<package_name>''.\\ Replace the ''<package_name>'' with the name of your package.\\ e.g. ''mkdir -p my_packages/net/network/rpcbind''.\\ The section and category can be found in the ''Makefile''.
   - Write a Makefile or download one ''Makefile'' from another package, look at samples on github.\\ Edit your Makefile and add necessary files, sources...\\ More: [[docs:guide-developer:packages|Creating packages]] & [[docs:guide-developer:helloworld:chapter3|Creating a package from your application]]\\   - Write a Makefile or download one ''Makefile'' from another package, look at samples on github.\\ Edit your Makefile and add necessary files, sources...\\ More: [[docs:guide-developer:packages|Creating packages]] & [[docs:guide-developer:helloworld:chapter3|Creating a package from your application]]\\
-  - Append a line with your custom feed to ''feeds.conf.default'':\\ ''src-link my_packages <buildroot>/my_packages''\\ Replace the ''<buildroot>'' with cloned openwrt sources directory e.g. ''~/openwrt''\\ Move the line with your custom feed above standard feeds to override them.\\+  - Append a line with your custom feed to ''feeds.conf.default'':\\ ''src-link my_packages <buildroot>/my_packages''\\ Replace the ''<buildroot>'' with cloned openwrt sources directory e.g. ''/home///user///openwrt'' (the path must be absolute).\\ Move the line with your custom feed above standard feeds to override them.\\
   - Now run: ''./scripts/feeds update -a; ./scripts/feeds install <package_name>''\\ If you are doing this to resolve a dependency you can run ''./scripts/feeds install <package_name>'' one more time and you should notice the dependency has been resolved.\\   - Now run: ''./scripts/feeds update -a; ./scripts/feeds install <package_name>''\\ If you are doing this to resolve a dependency you can run ''./scripts/feeds install <package_name>'' one more time and you should notice the dependency has been resolved.\\
-  - Build your package.\\ Select it in the menu of ''Make menuconfig''\\ Build it with ''make package/my_package/{clean,compile}''\\ More: [[docs:guide-developer:single.package|Building a single package]]\\+  - Build your package.\\ Select it in the menu of ''Make menuconfig''\\ Build it with ''make package///my_package_name///{clean,compile}''\\ More: [[docs:guide-developer:toolchain:single.package|Building a single package]]\\
  
 ===== Selecting code revision ===== ===== Selecting code revision =====
Line 128: Line 135:
 git tag git tag
  
-# Use OpenWrt 21.02.0 release +# Use OpenWrt 21.02.1 release 
-git checkout v21.02.0+git checkout v21.02.1
 </code> </code>
  
Line 139: Line 146:
 git checkout ${REV_HASH} git checkout ${REV_HASH}
 REV_BRANCH="$(git rev-parse --abbrev-ref HEAD)" REV_BRANCH="$(git rev-parse --abbrev-ref HEAD)"
 +
 +# Replace all src-git with src-git-full: https://openwrt.org/docs/guide-developer/feeds#feed_configuration
 sed -e "/^src-git\S*/s//src-git-full/" feeds.conf.default > feeds.conf sed -e "/^src-git\S*/s//src-git-full/" feeds.conf.default > feeds.conf
 +
 ./scripts/feeds update -a ./scripts/feeds update -a
 +
 +# Edit every line of feeds.conf in a loop to set the chosen revision hash
 sed -n -e "/^src-git\S*\s/{s///;s/\s.*$//p}" feeds.conf \ sed -n -e "/^src-git\S*\s/{s///;s/\s.*$//p}" feeds.conf \
 | while read -r FEED_ID | while read -r FEED_ID
Line 148: Line 160:
 sed -i -e "/\s${FEED_ID}\s.*\.git$/s/$/^${REV_HASH}/" feeds.conf sed -i -e "/\s${FEED_ID}\s.*\.git$/s/$/^${REV_HASH}/" feeds.conf
 done done
 +
 ./scripts/feeds update -a ./scripts/feeds update -a
 ./scripts/feeds install -a ./scripts/feeds install -a
Line 155: Line 168:
 Compile OpenWrt in a way that it gets [[https://forum.openwrt.org/t/compiling-openwrt-exactly-like-the-official-one/23214|the same packages as the default official image]]: Compile OpenWrt in a way that it gets [[https://forum.openwrt.org/t/compiling-openwrt-exactly-like-the-official-one/23214|the same packages as the default official image]]:
  
 +For example if you are flashing for the [[toh:tp-link:archer_c7]] you can locate on its wiki page the factory firmware URL https://downloads.openwrt.org/releases/21.02.1/targets/ath79/generic/openwrt-21.02.1-ath79-generic-tplink_archer-c7-v2-squashfs-factory.bin
 +
 +'ath79' tells you the Target, 'generic' tells you the Subtarget while 'archer-c7-v2' tells you the Target profile for the very specific device.
 +
 +Download the config by removing everything up to the subtarget 'generic' and adding 'config.buildinfo':
 <code bash> <code bash>
 # OpenWrt 19.07 and later # OpenWrt 19.07 and later
-wget https://downloads.openwrt.org/releases/19.07.0/targets/ramips/mt7621/config.buildinfo -O .config+wget https://downloads.openwrt.org/releases/21.02.1/targets/ath79/generic/config.buildinfo -O .config
  
 # OpenWrt 18.06 and before # OpenWrt 18.06 and before
 wget https://downloads.openwrt.org/releases/18.06.0/targets/ramips/mt7621/config.seed -O .config wget https://downloads.openwrt.org/releases/18.06.0/targets/ramips/mt7621/config.seed -O .config
 </code> </code>
 +
 +When using this configuration the correct defaults will be already selected for the Target and Subtarget but **not** for the Target profile so you will have to tailor it for the specific device if you want to build only that one.
  
 ===== Image configuration ===== ===== Image configuration =====
Line 273: Line 293:
 :!: Also you won't be able to install kernel packages from the official repositories when you make changes here. :!: Also you won't be able to install kernel packages from the official repositories when you make changes here.
  
-While you won't typically need to do this, you can do it:+While you won't typically need to do this, first define the target, subtarget, and device in a ''.config'', for example: 
 +<code bash> 
 +CONFIG_TARGET_x86=y 
 +CONFIG_TARGET_x86_64=y 
 +CONFIG_TARGET_x86_64_DEVICE_generic=y 
 +</code>
  
 +Now run the following where subtarget is not a variable, it is a literal and it references the ''.config'' we just created (in this case x86/64):
 <code bash> <code bash>
 make kernel_menuconfig CONFIG_TARGET=subtarget make kernel_menuconfig CONFIG_TARGET=subtarget
Line 288: Line 314:
 </code> </code>
  
-=== Source mirrors ===+==== Source mirrors ====
 The 'Build system settings' include some efficient options for changing package locations which makes it easy to handle a local package set: The 'Build system settings' include some efficient options for changing package locations which makes it easy to handle a local package set:
   - Local mirror for source packages   - Local mirror for source packages
Line 347: Line 373:
 <code bash> <code bash>
 make download make download
-ionice -c 3 nice -n19 make -j9+ionice -c 3 chrt --idle 0 nice -n19 make -j9
 </code> </code>
  
Line 363: Line 389:
  
 It doesn't matter what feed the package is located in, this same syntax works for any installed package. It doesn't matter what feed the package is located in, this same syntax works for any installed package.
 +
 +Note: you must have done a full tree build (make, or make world) beforehand for this to work reliably.
  
 === Spotting build errors === === Spotting build errors ===
Line 407: Line 435:
 make -i ... make -i ...
 </code> </code>
 +
 +
 +=== Make a summary information of generated image ===
 +<code bash>
 +make json_overview_image_info
 +</code>
 +Generate a summary of the image (including default packages, type of target, etc... ) in JSON format. The output is available in ''<BUILD_DIR>/profiles.json''.
 +
 +
 +=== Calculate checksum for generated files ===
 +<code bash>
 +make checksum
 +</code>
 +The following action will take place: a checksum will be computed and saved for the output files. This checksum will then be stored in the '<BIN_DIR>/sha256sums' .
  
 ===== Locating images ===== ===== Locating images =====
Line 414: Line 456:
 E.g. if your ''<buildroot>'' is ''~/source'', the binaries are in ''~/source/bin/targets/ar71xx/generic'' and ''~/source/bin/packages/mips_24kc''. E.g. if your ''<buildroot>'' is ''~/source'', the binaries are in ''~/source/bin/targets/ar71xx/generic'' and ''~/source/bin/packages/mips_24kc''.
  
-See also: [[docs:guide-developer:build-system:start##directory_structure|Directory structure]]+See also: [[docs:guide-developer:toolchain:buildsystem_essentials#directory_structure|Directory structure]]
  
 ===== Cleaning up ===== ===== Cleaning up =====
 You might need to clean your //build environment// every now and then. You might need to clean your //build environment// every now and then.
-The following ''make''-targets are useful for that job. 
  
 +The build artefacts, toolchain, build tools and downloaded feeds & sources files can be cleaned selectively.\\
 +The following ''make''-targets are useful for that job.\\  ''make clean'' is the most frequently needed clean operation.
 +
 +^ > Cleaned components > \\ v make argument v^ Compiled binaries:\\ firmware, kernel, packages ^ Toolchain\\ (target-specific)  ^ Build tools,\\ tmp/\\ ^ Compiled\\ config tools ^.config\\ \\ ^ feeds, .ccache,\\ downloaded source files ^
 +| clean        |  x  | | | | | |
 +| targetclean  |  x  |  x  | | | | |
 +| dirclean     |  x  |  x  |  x  |  x  | | |
 +| config-clean | | |  |  x  | | |
 +| distclean    |  x  |  x  |  x  |  x  |  x  |  x  |
 ==== Clean ==== ==== Clean ====
 <code bash> <code bash>
Line 425: Line 475:
 </code> </code>
  
-Delete contents of the directories ''/bin'' and ''/build_dir''.+Deletes contents of the directories ''/bin'' and ''/build_dir''.
 This doesn't remove the toolchain, and it also avoids cleaning architectures/targets other than the one you have selected in your ''.config''. This doesn't remove the toolchain, and it also avoids cleaning architectures/targets other than the one you have selected in your ''.config''.
 It is a good practice to do ''make clean'' before a build to ensure that no outdated artefacts have been left from the previous builds. It is a good practice to do ''make clean'' before a build to ensure that no outdated artefacts have been left from the previous builds.
 That may not be necessary always, but as a general rule it helps to ensure quality builds. That may not be necessary always, but as a general rule it helps to ensure quality builds.
 +
 +==== Targetclean ====
 +<code bash>
 +make targetclean
 +</code>
 +
 +This cleans also the target-specific toolchain in addition of doing ''make clean''. This may be needed when the toolchain components like musl or gcc change.\\
 +Does a ''make clean'' and deletes also the directories ''/build_dir/toolchain*'' and ''/staging_dir/toolchain*'' (= the cross-compile tools).\\ \\
 +Note: ''make targetclean'' has been introduced in 22.03 and is not found in earlier OpenWrt versions.
  
 ==== Dirclean ==== ==== Dirclean ====
Line 435: Line 494:
 </code> </code>
  
-This is your basic "full clean" operation. +This is your basic "full clean" operation. Cleans all compiled binaries, tools, toolchain, tmp/ etc.\\ 
-Delete contents of the directories ''/bin'' and ''/build_dir'' and additionally ''/staging_dir'' and ''/toolchain'' (=the cross-compile tools), ''/tmp'' (e.g data about packages) and ''/logs''.+Deletes contents of the directories ''/bin'' and ''/build_dir'' and ''/staging_dir'' (= tools and the cross-compile toolchain), ''/tmp'' (e.g data about packages) and ''/logs''. 
  
 ==== Distclean ==== ==== Distclean ====
Line 443: Line 503:
 </code> </code>
  
-Nuke everything you have compiled or configured and also deletes all downloaded feeds contents and package sources.+Nukes everything you have compiled or configured and also deletes all downloaded feeds contents and package sources.
 :!: In addition to all else, this will **erase your build configuration ''<buildroot>/.config''**. :!: In addition to all else, this will **erase your build configuration ''<buildroot>/.config''**.
 Use only if you need a "factory reset" of the build system! Use only if you need a "factory reset" of the build system!
 +
  
 ==== Selective cleanup ==== ==== Selective cleanup ====
Line 467: Line 528:
  
 ===== Troubleshooting ===== ===== Troubleshooting =====
-  * Beware of [[docs:guide-developer:toolchain:install-buildsystem#build_system_setup_linux|unusual environment variables]].+  * Beware of [[docs:guide-developer:toolchain:install-buildsystem#build_system_setup|unusual environment variables]].
   * First get more information on the problem using the make option ''make V=sc'' or enable logging.   * First get more information on the problem using the make option ''make V=sc'' or enable logging.
   * Read more about make options: [[docs:techref:buildroot|Buildroot Techref]].   * Read more about make options: [[docs:techref:buildroot|Buildroot Techref]].
  
 ==== Missing source code file, due to download problems ==== ==== Missing source code file, due to download problems ====
-First check if the URL path in the make file contains a trailing slash, then try with it removed (helped several times).+First check if the URL path in the make file contains a trailing slash, then if it does, try with it removed (helped several times).
 Otherwise try to download the source code manually and put it into ''dl'' directory. Otherwise try to download the source code manually and put it into ''dl'' directory.
  
Line 501: Line 562:
  
 If running ''make'' does //not// yield images for one (or even all) of the boards linked to the device profile that you have selected, than you probably have selected/enabled too many options or packages, and the image was too big to be flashed onto your device. If running ''make'' does //not// yield images for one (or even all) of the boards linked to the device profile that you have selected, than you probably have selected/enabled too many options or packages, and the image was too big to be flashed onto your device.
- 
  • Last modified: 2021/10/15 08:26
  • by bobafetthotmail