Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Next revisionBoth sides next revision
meta:infobox:432_warning [2017/02/05 16:47] – [Extensibility] fixed links, rephrased some sentences bobafetthotmailsupported_devices:432_warning [2021/11/29 01:38] – [Warning about 4/32 devices] richb-hanover
Line 1: Line 1:
-====== 4/32 warning ======+====== Warning about 4/32 devices ======
 ~~NOTOC~~ ~~NOTOC~~
  
-Every operating system requires +<WRAP round info 100%> 
-  Sufficient Flash to accommodate firmware image+**TL;DR Devices with ≤4MB flash and/or ≤32MB RAM (so-called "4/32 devices") may still work but they will be very limited (usually they can't install or run additional packages) because they have low RAM and Flash space.** Consider this when choosing a device to buy, or when deciding to flash OpenWrt on your device because it is listed as supported. 
 +</WRAP> 
 + 
 +Default builds of OpenWrt 21.02 can only run on 8/64 devices. We **recommend 16/128 devices.** Every operating system requires:
   - Sufficient RAM for stable operation   - Sufficient RAM for stable operation
 +  - Sufficient Flash to accommodate the firmware image
  
-**Devices with insufficient/low Flash and/or RAM have implications on usability, extensibility and supportability.**+===== Low RAM can lead to instability =====
  
-===== Usability =====+  * 32 MB can barely work for minimal router/AP functions, but may repeatedly "crash", depending on your hardware and use case 
 +  * 64 MB may still have issues with stability, depending on your hardware and use cases 
 +  * 128 MB or more is recommended (in 2021) if software past basic router/AP functionality is to be used
  
-**Insufficient RAM for stable operation**+===== Low Flash can't load new firmware or new packages =====
  
-  * 32MB minimum; can work quite well, but can also be borderlinedepending on your usecase+  * 4MB is absolute minimum (but you won't be able to install LuCI web interface) / 8MB is barely enough (will fit LuCI and some other applications) / 16MB offers more flexibility 
-  * 64MB are more comfortable to work withand in general the better choice.+  * 4MB devices can't fit anything noteworthy unless you use [[docs:guide-user:additional-software:imagebuilder|the Image Generator (Image Builder)]] (that requires a Linux system and some mild experience) or use [[docs:guide-user:additional-software:extroot_configuration|Extroot]]. Experienced users creating custom builds may be able to [[docs:guide-user:additional-software:saving_space|Saving firmware space]]but many packages won't ever fit no matter what you do
 +  * 8MB (or more) of Flash lets install at least a few additional software packages 
 +  * 16MB Flash (minimum) is recommended for the foreseeable future (in 2021)
  
-===== Extensibility =====+Most probably, you will not be able to install the following popular packages (and others) on a device with only 4MB flash:
  
-**Barely enough Flash to accommodate LEDE firmware image**+  [[docs:guide-user:network:tunneling_interface_protocols|VPNs]] and any other package requiring encryption 
 +  [[docs:guide-user:services:nas:samba_configuration|Samba]] (shared folders) 
 +  3G/4G dongle support 
 +  filesystem drivers/tools for formatting and checking a filesystem for [[docs:guide-user:additional-software:extroot_configuration|Extroot]]
  
-  * 4MB min (won't be able to install luci) / 8MB better (will fit luci and some other applications) +As the current stable 21.02 release uses kernel 5.4 that is roughly 0.5 MB larger than the kernel 4.14 used in the old 19.07.x releases, and SSL/HTTPS has been enabled by default needing at least some 0.3 MB flash space for the SSL libraries, the free flash space available on 8MB flash systems will be rather small in the current OpenWrt 21.02 releaseOpenWrt master has already partially moved to kernel 5.10which is still larger.
-  * 4MB can work, but are no fun to work with>4MB will make you happier than 4MB or below. +
-  * 4MB devices can't fit anything noteworthy unless you use [[docs:user-guide:imagebuilder|the Image Generator (Image Builder)]] (that requires a Linux system and some mild experience) or use [[docs:user-guide:extroot_configuration|Extroot]]Experienced users deploying specific or custom builds may not be affected by these restrictions; novices however will likely waste hours only to see that their expectations of an extensible operating system which LEDE claims to be can not be met by their device. +
-  * Especially in regards to extensibilitythe advice can only be: 8MB (or more) are the better choice.+
  
 +===== Supportability issues =====
  
-Devices with only 4MB Flash memory will barely be able to hold the LEDE firmware image, and will have limitations in their extensibility. With low flash, do not expect too much in this regard.+It is getting harder or even impossible over time to support devices with low Flash + RAM. You will be less likely to find people willing to assist on the forums.
  
-Most probably, you will not be able to install the following popular packages (and others) on device with only 4MB flash:+The 32 MB RAM is harder limitation than the flash size. The current Linux 5.4 barely works with 32 MB RAM system and spikes in memory consumption can easily crash the router with OOM (Out-of-Memory) errors.
  
-  [[docs:user-guide:tunneling_interface_protocols|VPNs]] and any other package requiring encryption +**OpenWrt support for those devices will end with the 19.07 release**, ie. 19.07 will be the last release with support for 4/32 MB devices. There will be no future releases and no images ready for download after 19.07.
-  [[docs:user-guide:samba_configuration|Samba]] (shared folders) +
-  * 3G/4G dongle support +
-  filesystem drivers/tools for formatting and checking a filesystem for [[docs:user-guide:extroot_configuration|Extroot]]+
  
  
-===== Supportability =====+===== Advice =====
  
-It is getting harder or even impossible over time to support devices with low Flash + RAM. +If you are not an expert user of OpenWrt (that is, if you do not build your own images), you should consider 
-LEDE support for those devices might end somewhere in the future.+
  
 +**16/64 as an //absolute// minimum for any device, with at least 128 MB of RAM being preferred.**  
  
-===== Advice =====+If a device has less than 16 MB of Flash and/or less than 64 MB of RAM, it may be unstable in basic operation under current versions of OpenWrt (21.02, 19.07). Further expect that support for the device may be dropped at any time and that security patches/updates to the kernel, drivers, and/or application software will not be available. While there is no warranty of ongoing support for any device under OpenWrt, those with insufficient resources are at great risk for "end of support".
  
-  * **new users** knowing what they want (or not)not knowing what they neednot knowing what to do -> get 8/64 +Previous versions of OpenWrt (such as earlier versions of 18.06.x17.01.x15.05.x "Chaos Calmer" and prior) contain now-known security vulnerabilities in the kernelwireless implementation, and/or application code. The OpenWrt community cannot support running known-vulnerable code under any situation. "It's just my router" is not justification as your router becoming compromised can impact others as a jump-pointcommand-and-control, or other participant in an attack. In many cases, these known vulnerabilities are being actively targeted, potentially including by [[https://blog.talosintelligence.com/2018/05/VPNFilter.html|advanced, likely state-sponsored or state-affiliated actor]] or actors.
-  * **experienced users** knowing what they wantneed, and do -> try if 4/32 suits your needs; if not, get 8/64+
  
-===== An analysis of the issue done by forum user slh ===== 
-As written in [[https://forum.lede-project.org/t/should-lede-support-devices-with-only-4mb-flash/1018/71| this forum post ]] by forum user **slh**. 
  
-First of all, I'm not pretending to speak for the LEDE team, however looking at the plain numbers presents a quite obvious situation.\\+===== Analysis of firmware size growth =====
  
-Taking "[[https://forum.lede-project.org/t/solved-why-no-images-generated-for-default-d-link-dir-600a1/990| Why no images generated for default D-Link DIR-600A1?]]" as an example (yes, some other 4 MB flash devices are a bit better than this particular specimen, the trend remains to be the same though).\\+As example, the size of the sysupgrade release image for WNDR3700v1, an ar71xx/ath79 device that has been supported by Openwrt for ten years: 
 + 
 +<code> 
 +master:        5056.3 KB (snapshot without LuCI) 
 +21.02.0:       5248.1 KB 
 +19.07.8:       4096.3 KB 
 +18.06.8:       3712.0 KB 
 +17.01.7:       3584.0 KB 
 +15.05.1:       3584.0 KB 
 +14.07:         3328.0 KB 
 +12.09:         2816.0 KB 
 +</code> 
 + 
 +Main reason is growth in size of the Linux kernel itself, but all included core packages (wifi, LuCI, etc.) also tend to grow as their features get expanded. 
 + 
 +==== Longer analysis of the issue done by forum user slh ==== 
 + 
 +As written in [[https://forum.openwrt.org/t/should-lede-support-devices-with-only-4mb-flash/1018/71| this forum post ]] by forum user **slh**. 
 + 
 +First of all, I'm not pretending to speak for the LEDE team, however looking at the plain numbers presents a quite obvious situation. 
 + 
 +Taking "[[https://forum.openwrt.org/t/solved-why-no-images-generated-for-default-d-link-dir-600a1/990| Why no images generated for default D-Link DIR-600A1?]]" as an example (yes, some other 4 MB flash devices are a bit better than this particular specimen, the trend remains to be the same though).
  
 <code>D-Link DIR-600A1: <code>D-Link DIR-600A1:
Line 63: Line 87:
 absolute firmware size......: 3735576 bytes absolute firmware size......: 3735576 bytes
 maximum usable firmware size: 3538944 bytes</code> maximum usable firmware size: 3538944 bytes</code>
-(all of these figures are for release images, including luci and a more or less identical feature set).\\+(all of these figures are for release images, including luci and a more or less identical feature set).
  
-The erase block size for this (and most other) devices is 64 KB, so you now end up with 256 KB (== 4 erase blocks) free space, compared to 320 KB (== 5 erase blocks before). While this may look comfortable at a first glance, you have to consider that free space can only be assigned in (full) block size chunks, so once you touch the overlay partition at all, you already have one erase block in use (64 KB). Therefore the firmware creation tools used by LEDE enforce at least 3 erase blocks reserve for the overlay filesystem (that's where the maximum usable firmware size comes from, compared to the total size of the firmware partition). In other words, with 17.01 you'll only have 1 erase block (64 KB) before the hard limit, while 15.05.1 still gave you 2 spare erase blocks (128 KB) for your own use. On top of this there is the file system overhead needed for formatting to jffs2 as well (jffs2 does some light compression, but its fs header and log (more or less directory entries) need some space, then there is the hard requirement to keep some free space (in erase block == 64 KB chunks) for the garbage collection to work) at all times, reducing the usable free space even further. Around 25 KB are used by the configuration overlay immediately after firstboot, before you actually get a chance to configure anything.\\+The erase block size for this (and most other) devices is 64 KB, so you now end up with 256 KB (== 4 erase blocks) free space, compared to 320 KB (== 5 erase blocks before). While this may look comfortable at a first glance, you have to consider that free space can only be assigned in (full) block size chunks, so once you touch the overlay partition at all, you already have one erase block in use (64 KB). Therefore the firmware creation tools used by LEDE enforce at least 3 erase blocks reserve for the overlay filesystem (that's where the maximum usable firmware size comes from, compared to the total size of the firmware partition). In other words, with 17.01 you'll only have 1 erase block (64 KB) before the hard limit, while 15.05.1 still gave you 2 spare erase blocks (128 KB) for your own use. On top of this there is the file system overhead needed for formatting to jffs2 as well (jffs2 does some light compression, but its fs header and log (more or less directory entries) need some space, then there is the hard requirement to keep some free space (in erase block == 64 KB chunks) for the garbage collection to work) at all times, reducing the usable free space even further. Around 25 KB are used by the configuration overlay immediately after firstboot, before you actually get a chance to configure anything.
  
-Assuming pure statistics, what will the situation be in 12 or 24 months[1]?\\+Assuming pure statistics, what will the situation be in 12 or 24 months[1]?
  
-No one has yet raised the suggestion to actually remove the hardware support for affected devices from LEDE's source repository. If you look deeper, you can still find full support for the Linksys WRT-54GL (4 MB flash, 16 MB RAM) in the repo, although support for devices with just 16 MB RAM had already been discontinued with Attitude Adjustment 12.09 and despite the fact that actually building a working up-to-date firmware for this device today is quite challenging[2].\\+No one has yet raised the suggestion to actually remove the hardware support for affected devices from LEDE's source repository. If you look deeper, you can still find full support for the Linksys WRT-54GL (4 MB flash, 16 MB RAM) in the repo, although support for devices with just 16 MB RAM had already been discontinued with Attitude Adjustment 12.09 and despite the fact that actually building a working up-to-date firmware for this device today is quite challenging[2].
  
 This thread merely serves as a reminder for 'normal users', who expect to download and flash LEDE under the expectation to use it as a full featured replacement for the vendor firmware (including a webinterface, luci). Also keep in mind that luci is enabled for release builds, which means that it either fits into the firmware image (with the mandatory safety margin of at least 3 erase blocks) or no release images can be created for the affected devices[3], [4]. In my very personal opinion, this needs to be documented quite obviously, avoiding to raise false hope and expectations for (especially new) users.\\ This thread merely serves as a reminder for 'normal users', who expect to download and flash LEDE under the expectation to use it as a full featured replacement for the vendor firmware (including a webinterface, luci). Also keep in mind that luci is enabled for release builds, which means that it either fits into the firmware image (with the mandatory safety margin of at least 3 erase blocks) or no release images can be created for the affected devices[3], [4]. In my very personal opinion, this needs to be documented quite obviously, avoiding to raise false hope and expectations for (especially new) users.\\
-Advanced users will obviously be able to delay the inevitable by quite some margin - depending on their use-cases and abilities to reduce LEDE's footprint below normal system requirements.\\+Advanced users will obviously be able to delay the inevitable by quite some margin - depending on their use-cases and abilities to reduce LEDE's footprint below normal system requirements.
  
-Despite the title, the RAM size is actually a much harder limit, affecting much more than just 4 MB flash devices. If you look through the commit log, you will notice quite some efforts to get opkg to use less RAM, but it's still a problem with just 32 MB RAM (even for normal operations, before touching opkg). Likewise you already are in trouble with sysupgrade and trying to flash a (larger) 6-8 MB firmware on a device with just 32 MB RAM, unless you really clean up services and loaded kernel modules manually beforehand, there is a high risk that you'll oom during the sysupgrade and brick your device for good (requiring bootloader/ tftp assistance to recover in the best case).\\+Despite the title, the RAM size is actually a much harder limit, affecting much more than just 4 MB flash devices. If you look through the commit log, you will notice quite some efforts to get opkg to use less RAM, but it's still a problem with just 32 MB RAM (even for normal operations, before touching opkg). Likewise you already are in trouble with sysupgrade and trying to flash a (larger) 6-8 MB firmware on a device with just 32 MB RAM, unless you really clean up services and loaded kernel modules manually beforehand, there is a high risk that you'll oom during the sysupgrade and brick your device for good (requiring bootloader/ tftp assistance to recover in the best case).
  
-[1] switching from mach files to device tree (post 17.01) offers a potential to free up a little space in the kernel, but this isn't a whole lot and will probably be eaten more or less completely by upgrading to kernel 4.9+ (also keep in mind that the FDT file needs to be appended to the kernel image, which might or might not compress as well as the mach files before), so don't expect a significant positive effect from this switch.\\+[1] switching from mach files to device tree (post 17.01) offers a potential to free up a little space in the kernel, but this isn't a whole lot and will probably be eaten more or less completely by upgrading to kernel 4.9+ (also keep in mind that the FDT file needs to be appended to the kernel image, which might or might not compress as well as the mach files before), so don't expect a significant positive effect from this switch.
  
-[2] no webinterface, better no pppd, quite some custom configurations to get the kernel's runtime memory requirements as small as possible, disabling whatever is humanly possible (definately no IPv6, better no wireless either).\\+[2] no webinterface, better no pppd, quite some custom configurations to get the kernel's runtime memory requirements as small as possible, disabling whatever is humanly possible (definately no IPv6, better no wireless either).
  
-[3] with LEDE's pretty new feature of device specific rootfs images, it would technically be possible to decide between installing luci or omitting it on a per device basis, e.g. based on flash sizes, but support for something like this hasn't been implemented yet and would require quite some attention (both as in source patches and tagging of device classes) from interested parties. As implemented right now, the decision is binary - either the default (release) config (including luci) fits XOR building a firmware image for the affected device fails and no firmware will be available.\\+[3] with LEDE's pretty new feature of device specific rootfs images, it would technically be possible to decide between installing luci or omitting it on a per device basis, e.g. based on flash sizes, but support for something like this hasn't been implemented yet and would require quite some attention (both as in source patches and tagging of device classes) from interested parties. As implemented right now, the decision is binary - either the default (release) config (including luci) fits XOR building a firmware image for the affected device fails and no firmware will be available.
  
-[4] I would expect that there already are a couple of 4 MB flash devices in the target list for which no release firmware can be built for 17.01, because of less ideal flash partitioning schemes chosen by the vendor (dropping free space below 3 erase blocks). Those are probably a minority, but given the close numbers, I'd be very suprised if there wouldn't be any affected.\\+[4] I would expect that there already are a couple of 4 MB flash devices in the target list for which no release firmware can be built for 17.01, because of less ideal flash partitioning schemes chosen by the vendor (dropping free space below 3 erase blocks). Those are probably a minority, but given the close numbers, I'd be very suprised if there wouldn't be any affected.
  
 +[[https://forum.openwrt.org/t/lede-a-bit-over-the-top-with-the-minimal-requirements/2009/4?u=tmomas|More explanations]] on this subject by forum user **slh**.
  
 ===== The opinion of a Core LEDE developer ===== ===== The opinion of a Core LEDE developer =====
-As originally written in [[https://forum.lede-project.org/t/should-lede-support-devices-with-only-4mb-flash/1018/79|this forum post]] by jow.+As originally written in [[https://forum.openwrt.org/t/should-lede-support-devices-with-only-4mb-flash/1018/79|this forum post]] by jow.
  
 Just providing my rather pragmatic opinion on the topic here:\\ Just providing my rather pragmatic opinion on the topic here:\\
Line 102: Line 127:
   *   **Small** => bootable images can be built when either sacrificing gui or opkg while still having configuration persistence   *   **Small** => bootable images can be built when either sacrificing gui or opkg while still having configuration persistence
   *   **Micro** => only choice are heavily tailored custom images that require special measures like pre-shipped configuration, NFS mounts, preconfigured extroot etc.   *   **Micro** => only choice are heavily tailored custom images that require special measures like pre-shipped configuration, NFS mounts, preconfigured extroot etc.
- 
- 
-===== Infoboxes ===== 
-These infoboxes can be included in other pages via the following section-includes: 
-<code> 
-{{section>meta:infobox:432_warning#infobox_for_dataentries&noheader&nofooter&noeditbutton}} 
-{{section>meta:infobox:432_warning#infobox_for_tohs&noheader&nofooter&noeditbutton}} 
-</code> 
- 
-==== Infobox for dataentries ==== 
- 
-<WRAP center round box info 650px> 
-Not recommended for future use with LEDE due to low flash/ram.\\ 
-Limitations in extensibility and stability of operation are to be expected. 
-</WRAP> 
- 
- 
-==== Infobox for ToHs ==== 
- 
-<WRAP round info 100%> 
-**Devices with ≤4MB flash and/or ≤32MB ram suffer from limitations in extensibility and stability of operation.** Consider this when chosing a device to buy, or when deciding to flash LEDE on your device because it is listed as supported. See [[432_warning]] for details. 
-</WRAP> 
  
  • Last modified: 2023/05/03 18:08
  • by tmomas