Information may be outdated and obsolete information as of April, 2018; Overview, Preinit and Overview, Failsafe updated from April 2018 based on reading master code.

See Rootfs on External Storage for information on external rootfs mounting.

This document presents the preinit / firstboot boot sequence. The boot system is extensible via (new) packages such as rootfs on usb, or enhanced failsafe.

We describe the portion of the OpenWrt boot sequence that occurs before the 'init' program is executed (when booting in multiuser mode), as well as the script that is responsible for creating and initializing the root filesystem on the first boot after flashing the device with OpenWrt.

The basic OpenWrt boot sequence is:

1. boot loader loads kernel
2. kernel loads whilst scanning the mtd partition rootfs for a valid superblock for mounting the SquashFS partition (which contains /etc). More info at technical.details
3. kernel calls /etc/preinit (the kernel considers this to be the init (or root) process /sbin/init now the init process, which in turn launches /sbin/procd -h /etc/hotplug-preinit.json (a.k.a., “plugd”) followed by /bin/sh /etc/preinit. Each is launched via fork and execvp and managed by the uloop library in libubox.
4. /etc/preinit prepares system for multiuser mode
5. /etc/preinit execs /sbin/init which becomes the init (or root) process and launches multiuser
6. /sbin/init launches processes according to /etc/inittab.
7. Typically the first process launched is /etc/init.d/rcS which causes the scripts in /etc/rc.d which begin with 'S' to be launched (in glob sort order). The /etc/rc.d directory is populated with symlinks to the scripts in /etc/init.d. Each script in /etc/init.d accepts enable and disable arguments for creating and removing the symlinks.
8. These script initialize the system and also initialize daemons that wait for input, so that when all the scripts have executed the normal system is active. On first boot this initializing includes the process of preparing the root filesystem for use.

Preinit brings the system from raw kernel to ready for multiuser. To do so, as of April, 2018, /etc/preinit (ar71xx, Archer C7) performs the following tasks:

1. Checks to see if $PREINIT is empty; if so exec /sbin/init and skip the rest
2. Sources a set of files for common functions for boot/mount:
   • /lib/functions.sh
   • /lib/functions/preinit.sh
   • /lib/functions/system.sh
3. Prepares the “hooks” for the various stages of the preinit process
   • preinit_essential
   • preinit_main
   • failsafe
   • initramfs
   • preinit_mount_root
4. Source the contents of /lib/preinit/* which generally add sh functions to the appropriate hooks
5. Runs the preinit_essential hooks
6. Runs the preinit_main hooks

On an ar71xx device (Archer C7) the default functions added to the various hooks include (with functions from device-specific files in italics):

• preinit_main
  1. do_ar71xx
  2. define_default_set_state
  3. do_sysinfo_generic
  4. preinit_set_mac_address
  5. set_preinit_iface
  6. preinit_ip
  7. pi_indicate_preinit
  8. failsafe_wait
  9. run_failsafe_hook
     • failsafe (if “$pi_preinit_no_failsafe” != “y” and “$FAILSAFE” = “true”)
       1. indicate_failsafe
       2. failsafe_netlogin
       3. failsafe_shell
  10. indicate_regular_preinit
  11. do_mount_root (if “$INITRAMFS” != “1”)
  12. do_urandom_seed
  13. check_patch_ath10k_firmware
  14. run_init

Presently unused hooks, at least for the ar71xx Archer C7, appear to include

• preinit_essential
• initramfs
• preinit_mount_root

The root file system is actually an overlay which can be consisted of a read-only SquashFS file system (mounted at /rom) and a writable JFFS2 partition (mounted under /overlay). In Failsafe mode only the squashfs FS will be mounted (changes made to jffs2 partitons will be ignored), plus the following steps:

1. Notifies that failsafe mode is being entered (indicate_failsafe)
2. Launches daemon to allow network logins (failsafe_netlogin)
3. Allows login via serial console, if there is one (failsafe_shell)
   • If the serial console login process exits, failsafe doesn't exit, but restarts the process for additional logins.

all_jffs2 refers to a 'jffs2' target in menuconfig; e.g. firmware has no squashfs, but is purely a rw filesystem (jffs2), while, jffs2 in the following text refers to the jffs2 portion of a squashfs/jffs2 system.

1. Kernel has previously mounted squashfs partition by scanning the mtd partition rootfs for a valid superblock (see step 2 of contextboot.sequencel) Make sure it's correct
2. If there is no mtd device with label rootfs_data, then mounts /dev/root (e.g. squashfs or all_jffs2 with no squashfs) as root filesystem, and indicates that further steps should be skipped
3. If mtd device rootfs_data has not already been formatted, mounts a tmpfs (ramdisk) as root filesystem, and indicates that further steps should be skipped.
4. Mounts previously formatted jffs2 partition on /overlay and indicates successful mount.
5. Makes successfully mounted /overlay (if it exists) the new root filesystem and moves previous root filesystem to /rom, and indicates to skip further steps.
6. This is only reached on an error condition; attempts to mount a tmpfs (ramdisk) as root filesystem
7. This is only reached if no other step succeeds; attempt to mount /dev/root (e.g. squashfs/all_jffs2) as root filesystem.

* /overlay was previously named /jffs2.
* NOTE: If volatile files (e.g. a config) were preserved across firmware update via sysupgrade, step 3 is skipped. Instead, preinit_main hangs while the rootfs_data partition is formatted and the jffs2 overlay is mounted. Hypothetically, this is fatal on systems with weak cpu and exceptionally large rootfs_data partitions. For more information consult this forum post.

/sbin/firstboot may be referenced in three ways.

• It may be called as part of the system startup, in which cased it is called as /sbin/firstboot switch2jffs.
• It may be used as a standalone command with no parameters (.e.g. /sbin/firstboot)
• It may be sourced from another script

1. Source /lib/functions/boot.sh for common functions (e.g. also used by preinit)
2. Source files used by hooks
3. Determine how called, and branch to appropriate commands.

1. Determine (and set variable for) MTD rootfs_data partition
2. Determine (and set variable for) rom partition
3. Determine (and set variable for) jffs2 partition

• Resets jffs2 to original settings, if possible.
• If jffs2 is not mounted, erases mtd and attempts format, mount, and pivot jffs2 as root.

If jffs2 is mounted, firstboot runs hook jffs2reset

1. Determine (and set variable for) MTD rootfs_data partition
2. Determine (and set variable for) rom partition
3. Determine (and set variable for) jffs2 partition
4. Determine (and set variable to indicate) whether the mini overlay filesystem type is supported.
5. If overlay is supported, remove all files on jffs2 and remount it.
6. If overlay not supported, create directories and symlinks, copying only certain critical files

• Note: since r35712 the firstboot script requires an inputted 'y' as confirmation. If using firstboot in a reset button script, you need to get that y inputted, e.g. by using the yes command: yes | firstboot

1. Determine (and set variable for) MTD rootfs_data partition
2. Determine (and set variable for) rom partition
3. Determine (and set variable for) jffs2 partition
4. Determine if mini overlay is supported. If not run hook no_fo
5. Otherwise, if mounted, skip the rest, otherwise mount under squashfs (/rom/jffs)
6. Copy ramdisk to jffs2
7. Move /jffs to / (root) and move / (root) to /rom
8. Cleanup

1. Switch to kernel fs, get rid of union overlay and bind from /tmp/root
2. Mount jffs (and make it safe for union)
3. If not mounted, mount; copy from squashfs, and pivot so that /jffs is now / (root)
4. Copy files from ramdisk
5. Get rid of unnecessary mounts (cleanup)

Preinit consists of a number of scripts. The main script is /etc/preinit which reads in the scripts. The scripts define functions which they attach to hooks. These hooks are, when processed, launching the functions in the order they were added to the hooks.

Currently there are five hooks used by the preinit system:

• preinit_essential
• preinit_main
• failsafe
• initramfs
• preinit_mount_root

Each hook have a corresponding string variable containing the name of each function to be executed, separated by spaces. The hook variables have _hook appended to the hook name. Thus the name of the variable for the preinit_essential hook is preinit_essential_hook.

The main preinit script is actually quite empty. It:

1. Initializes some variables (including the hook variables)
2. Defines the function pi_hook_add, which is used to add functions to a hook
3. Defines the function pi_run_hook, which executes the functions that were added to a hook
4. Sources (reads) the shell scripts under folder /lib/preinit/, in glob sort order
5. Processes the hook preinit_essential
6. Initializes variables used by preinit_main
7. Processes the hook preinit_main

That's it.

There are a number of variables that control options of preinit. Defaults are defined in the main script /etc/preinit defined by the base-files package. However the variables are customizable via make menuconfig, in section “Preinit configuration options”. The OpenWrt build process will then create the file /lib/preinit/00_preinit.conf which will be sourced by the main script.

The variables defined at present are:

There are also variables used in the operation of preinit. They are:

The following sections describe the files and functions used by the various hooks.

NB: The files, even though divided by hook here are all in the single /lib/preinit directory, and are thus combined in the directory lists, and are processed in glob sort order, not by hook (when sourcing them, the hooks specify the order of the execution of functions, which is as listed below)

For the purposes of development, you will locate the files under $ROOTDIR/package/base-files/files/lib/preinit, for the existing files, and you can add new files anywhere that ultimately ends up in /lib/preinit on the router (while in preinit, e.g. not by user edits after read-write is mounted).

With the introduction of procd, effective in Chaos Calmer release, the tasks of this hook are done in c code and at least on some images (verified on a BRCM63xx one) this hook is unused.

The preinit_essentials hook takes care of mounting essential kernel filesystems such as proc, and initializing the console.

Files containing the functions executed by this hook

Functions, in order, executed by this hook (doesn't list the functions only called by other functions)

Functions which are called by other functions, rather than directly as part of a hook

The preinit_main hook performs all the functions required of preinit, except those functions, like console, that are essential even for preinit tasks.

Functions, in order, executed by this hook (doesn't list the functions only called by other functions)

Functions which are called by other functions, rather than directly as part of a hook.

Do what needs to done to prepare failsafe mode and enter it.

Functions, in order, executed by this hook (doesn't list the functions only called by other functions)

Functions which are called by other functions, rather than directly as part of a hook

Mount the root filesystem

Functions, in order, executed by this hook (doesn't list the functions only called by other functions)

Functions which are called by other functions, rather than directly as part of a hook

No files or functions at this time.

1. Source common functions
2. Source functions for hooks
3. if block:

if invoked as executable

       if called with `switch2jffs` parameter (i.e. from rcS)
           run hook `switch2jffs`
       if called standalone (e.g. from commandline)
           if there is a jffs2 partition mounted
                run hook `jffs2reset`
           else
                erase rootfs_data mtd partition
                format
                and remount it
           end
       end
 if sourced (that is not executed)
      set some variables
 end

Make the filesystem that we want to be the rootfs, to be the rootfs

Functions, in order, executed by this hook (doesn't list the functions only called by other functions)

Functions which are called by other functions, rather than directly as part of a hook

Make the filesystem that we want to be the rootfs, to be the rootfs, given that we have no mini\_fo overlay filesystem

Functions, in order, executed by this hook (doesn't list the functions only called by other functions)

Reset jffs2 to defaults

Functions, in order, executed by this hook (doesn't list the functions only called by other functions)

NB: These files must be added to the *squashfs* (or if using a all_jffs2 system, to the jffs2). That means, for instance adding it to the image's rootfs. This can be done, for instace, by creating `${ROOTDIR}/files/filename` (with appropriate substitutions of course).

Customizing the system is quite simple. We give an example of changing the message for preinit from '- preinit -' to '- setting the table for dinner -'

Create a file that replaces the function `indicate_regular_preinit_boot`. `pi_indicate_preinit` is defined in `20_indicate_preinit`, so we define our replace functions in `25_dinner_not_router`.

`/lib/preinit/25_dinner_not_router`

   pi_indicate_preinit() { 
         echo "- setting the table for dinner -"
         preinit_net_echo "Dinner is just about ready!"
         pi_indicate_led
   }
   

This results in the following boot log:

NET: Registered protocol family 17
802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
All bugs added by David S. Miller <davem@redhat.com>
VFS: Mounted root (squashfs filesystem) readonly on device 31:2.
Freeing unused kernel memory: 132k freed
Please be patient, while OpenWrt loads ...
eth1: link forced UP - 100/full - flow control off/off
- setting the table for dinner -
Press CTRL-C or Press f<ENTER> to enter failsafe mode
switching to jffs2
mini_fo: using base directory: /
mini_fo: using storage directory: /jffs
- init -

The default boot log is

NET: Registered protocol family 17
802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
All bugs added by David S. Miller <davem@redhat.com>
VFS: Mounted root (squashfs filesystem) readonly on device 31:2.
Freeing unused kernel memory: 132k freed
Please be patient, while OpenWrt loads ...
eth1: link forced UP - 100/full - flow control off/off
- preinit -
Press CTRL-C or Press f<ENTER> to enter failsafe mode
switching to jffs2
mini_fo: using base directory: /
mini_fo: using storage directory: /jffs
- init -
   

As another example we will add a message to failsafe, between the notice that we're in failsafe mode in the shell. You could use this, for example, to create a text menu system, or to launch a simple web server (with cgi scripts) to permit the user to do failsafe things.

We want to add the message, 'Remember, at this point there are no writable filesystems'

We create the file `50_failsafe_remember_no_rw`, in `/lib/preinit`

  remember_no_rw() {
      echo "Remember, at this point there are no writable filesystems"
  }
  
  boot_hook_add failsafe remember_no_rw
  

This creates the function `remember_no_rw` and adds it to the failsafe hook, in between `10_indicate_failsafe` and `99_10_failsafe_login` which define the other functions in the `failsafe` hook. This wasn't necessary for the previous example because the function was already in a hook.

The boot log for this, when entering failsafe, is:

VFS: Mounted root (squashfs filesystem) readonly on device 31:2.
Freeing unused kernel memory: 132k freed
Please be patient, while OpenWrt loads ...
eth1: link forced UP - 100/full - flow control off/off
- preinit -
Press CTRL-C or Press f<ENTER> to enter failsafe mode
f
- failsafe -
Remember, at this point there are no writable filesystems

BusyBox v1.15.3 (2010-01-20 19:26:26 EST) built-in shell (ash)
Enter 'help' for a list of built-in commands.

ash: can't access tty; job control turned off

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
                |__| W I R E L E S S   F R E E D O M

 KAMIKAZE (bleeding edge, r19235) ------------------
  * 10 oz Vodka       Shake well with ice and strain
  * 10 oz Triple sec  mixture into 10 shot glasses.
  * 10 oz lime juice  Salute!
 ---------------------------------------------------

Some architectures have additional files and functions (or overrides of the above functions) in order to accommodate specific needs of that hardware. In that case the files are located in the source tree under $ROOTDIR/target/linux/<architecture[/subarch]/base-files/lib/preinit. During build they are merged and appear under /lib/preinit along with the rest.