Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Next revisionBoth sides next revision
toh:linksys:whw03_v1 [2024/05/15 06:55] – [OpenWrt bootlog] lanchontoh:linksys:whw03_v1 [2024/07/17 06:09] – [Repartitioning] lanchon
Line 163: Line 163:
 </code> </code>
  
 +==== Repartitioning ====
 +
 +For compatibility with stock firmware, and to allow returning to stock firmware, OpenWrt fits the complete install in the kernel/rootfs partition (or alt_kernel/alt_rootfs partition, if booting to second copy of the firmware). Note that the rootfs partition is part of and contained within the kernel partition: the linux kernel is stored in the first 8 MiB of the 136 MiB kernel partition, and the remaining 128 MiB are the 128 MiB rootfs partition.
 +
 +OpenWrt uses whatever remains unused of the rootfs partition as rootfs_data, to store the overlay. This is why only about 80 MiB of storage is available to OpenWrt, even though the eMMC flash size is 4 GB. But if you do not plan to return to stock, you can repartition the eMMC so that much more space is available for use.
 +
 +The following steps will configure 512 MiB rootfs+rootfs_data for each firmware copy (this is a dual firmware device), and the remaining 2.4 GiB space will be set up to be used as persistent storage that is not wiped/restored during sysupgrades.
 +
 +- Setup SSH access and verify that your stock GPT is exactly the same as mine:
 +
 +<code>
 +root@OpenWrt:/# dd if=/dev/mmcblk0 bs=512 count=34 | md5sum
 +34+0 records in
 +34+0 records out
 +7a8c9529ee9e45be87407127e6905893  -
 +</code>
 +
 +WARNING: VERIFY THAT YOUR GPT HASH IS THE SAME AS MINE! DO NOT PROCEED OTHERWISE!
 +
 +- Backup your current configuration.
 +
 +- Install these packages: gdisk sgdisk blkdiscard 
 +
 +- Install luci-app-attendedsysupgrade if you use LuCI, or install auc if you do not.
 +
 +- Use "LuCI/Sytem/Attended Sysupgrade" (or the auc command) to upgrade the firmware TWICE. Yes TWICE, you want both firmware copies updated; so upgrade once (router will reboot) and then upgrade again. You will need to select advanced mode in the Attended Sysupgrade configuration tab to do the repeated upgrades.
 +
 +IMPORTANT: DO NOT PROCEED WITHOUT DOING THE 2 SYSUPGRADES! For the procedure to succeed, the installed commands need to be in rootfs, not rootfs_data, as rootfs_data will have to be unmounted.
 +
 +- Make backups of the GPT (partition table):
 +
 +<code>
 +root@OpenWrt:/# mkdir /tmp/lanchon
 +
 +# backup primary copy of GPT:
 +
 +root@OpenWrt:/# dd if=/dev/mmcblk0 bs=512 count=34 of=/tmp/lanchon/stock-gpt-pri.bin
 +34+0 records in
 +34+0 records out
 +
 +# backup secondary copy of GPT:
 +
 +root@OpenWrt:/# dd if=/dev/mmcblk0 bs=512 skip=7634911 of=/tmp/lanchon/stock-gpt-sec.bin
 +33+0 records in
 +33+0 records out
 +</code>
 +
 +- Make backups of the stock partitions:
 +
 +NOTE: The last partition (mmcblk0p19 "syscfg") has some configuration info used by stock firmware.
 +It is 3.4 GiB in size, with an ext4 filesystem that only contains about 250 KiB of files (when i looked).
 +However this filesystem is not trimmed, and as time goes on it accumulates stale data in unused areas.
 +If the backup commands below fail with /tmp running out of space, this stale data is the culprit.
 +
 +You then have 3 options:
 +
 +1) If one backup fits in /tmp, scp the first backup (see next step), remove it from /tmp, then do the second backup.
 +
 +2) Else pipe the backups over SSH (see below). Note that this might not work from a Windows PC due to character translations (but you could work some magic with base64 or similar).
 +
 +3) Or mount and fstrim the partition to remove the stale data (requires the "fstrim" package).
 +
 +<code>
 +# now you can backup the complete eMMC (GPTs and all partitions):
 +
 +root@OpenWrt:/# gzip -c /dev/mmcblk0 >/tmp/lanchon/mmcblk0-emmc.img.gz
 +
 +# or you can just backup the partition that will be wiped (but better if you do both backups):
 +
 +root@OpenWrt:/# gzip -c /dev/mmcblk0p19 >/tmp/lanchon/mmcblk0p19-syscfg.img.gz
 +
 +# or if you ran out of space in /tmp, you can pipe the backups over SSH from your Linux PC:
 +
 +you@your-pc:~$ ssh root@192.168.1.1 "gzip -c /dev/mmcblk0" >mmcblk0-emmc.img.gz
 +you@your-pc:~$ ssh root@192.168.1.1 "gzip -c /dev/mmcblk0p19" >mmcblk0p19-syscfg.img.gz
 +
 +# and test your backups after piping them, especially if done from a Windows PC:
 +
 +you@your-pc:~$ gzip -t mmcblk0*.img.gz
 +</code>
 +
 +- Using SCP from your PC, bring all backup files to your PC and store them safely:
 +
 +<code>
 +you@your-pc:~$ scp root@192.168.1.1:/tmp/lanchon/* .
 +</code>
 +
 +WARNING: DO NOT PROCEED WITHOUT HAVING THE BACKUPS IN YOUR PC! STORE THEM SAFELY.
 +
 +- Now comes the repartitioning, follow these steps via SSH:
 +
 +<code>
 +root@OpenWrt:~# 
 +
 +# before starting, verify that the GPT hash matches expectation:
 +
 +root@OpenWrt:~# dd if=/dev/mmcblk0 bs=512 count=34 | md5sum
 +34+0 records in
 +34+0 records out
 +7a8c9529ee9e45be87407127e6905893  -
 +
 +# DO NOT PROCEED IF THE ABOVE HEX STRING DOES NOT MATCH YOUR OUTPUT.
 +
 +# unmount overlay:
 +
 +root@OpenWrt:~# umount /overlay
 +
 +# wipe/discard/trim p19:
 +
 +root@OpenWrt:~# blkdiscard /dev/mmcblk0p19 -f
 +blkdiscard: Operation forced, data will be lost!
 +
 +# copy rootfs contents to the disk location where the rootfs partition will be in the future:
 +
 +root@OpenWrt:~# dd if=/dev/mmcblk0p15 of=/dev/mmcblk0p19 bs=1M seek=0
 +128+0 records in
 +128+0 records out
 +
 +# copy alt_rootfs contents to the disk location where the alt_rootfs partition will be in the future:
 +
 +root@OpenWrt:~# dd if=/dev/mmcblk0p17 of=/dev/mmcblk0p19 bs=1M seek=512
 +128+0 records in
 +128+0 records out
 +
 +root@OpenWrt:~# gdisk /dev/mmcblk0
 +GPT fdisk (gdisk) version 1.0.10
 +
 +Partition table scan:
 +  MBR: protective
 +  BSD: not present
 +  APM: not present
 +  GPT: present
 +
 +Found valid GPT with protective MBR; using GPT.
 +
 +Command (? for help): p
 +Disk /dev/mmcblk0: 7634944 sectors, 3.6 GiB
 +Sector size (logical/physical): 512/512 bytes
 +Disk identifier (GUID): 98101B32-BBE2-4BF2-A06E-2BB33D000C20
 +Partition table holds up to 20 entries
 +Main partition table begins at sector 2 and ends at sector 6
 +First usable sector is 34, last usable sector is 7634910
 +Partitions will be aligned on 2-sector boundaries
 +Total free space is 0 sectors (0 bytes)
 +
 +Number  Start (sector)    End (sector)  Size       Code  Name
 +                34            1057   512.0 KiB   A012  0:SBL1
 +              1058            2081   512.0 KiB   FFFF  0:BOOTCONFIG
 +              2082            3105   512.0 KiB   A016  0:QSEE
 +              3106            4129   512.0 KiB   FFFF  0:QSEE_1
 +              4130            4641   256.0 KiB   A01B  0:CDT
 +              4642            5153   256.0 KiB   FFFF  0:CDT_1
 +              5154            5665   256.0 KiB   FFFF  0:BOOTCONFIG1
 +              5666            7713   1024.0 KiB  A015  0:APPSBL
 +              7714            9761   1024.0 KiB  FFFF  0:APPSBL_1
 +  10            9762           10273   256.0 KiB   FFFF  0:ART
 +  11           10274           12321   1024.0 KiB  FFFF  u_env
 +  12           12322           14369   1024.0 KiB  FFFF  s_env
 +  13           14370           16417   1024.0 KiB  FFFF  devinfo
 +  14           16418          294945   136.0 MiB   FFFF  kernel
 +  15           32802          294945   128.0 MiB   FFFF  rootfs
 +  16          294946          573473   136.0 MiB   FFFF  alt_kernel
 +  17          311330          573473   128.0 MiB   FFFF  alt_rootfs
 +  18          573474          574497   512.0 KiB   FFFF  sysdiag
 +  19          574498         7634910   3.4 GiB     FFFF  syscfg
 +
 +# resize GPT from non-standard 20 entries to standard 128 entries:
 +
 +Command (? for help): x
 +
 +Expert command (? for help): s
 +Current partition table size is 20.
 +Enter new size (19 up, default 128): 
 +
 +Expert command (? for help): m
 +
 +Command (? for help): p
 +Disk /dev/mmcblk0: 7634944 sectors, 3.6 GiB
 +Sector size (logical/physical): 512/512 bytes
 +Disk identifier (GUID): 98101B32-BBE2-4BF2-A06E-2BB33D000C20
 +Partition table holds up to 128 entries
 +Main partition table begins at sector 2 and ends at sector 33
 +First usable sector is 34, last usable sector is 7634910
 +Partitions will be aligned on 2-sector boundaries
 +Total free space is 0 sectors (0 bytes)
 +
 +Number  Start (sector)    End (sector)  Size       Code  Name
 +                34            1057   512.0 KiB   A012  0:SBL1
 +              1058            2081   512.0 KiB   FFFF  0:BOOTCONFIG
 +              2082            3105   512.0 KiB   A016  0:QSEE
 +              3106            4129   512.0 KiB   FFFF  0:QSEE_1
 +              4130            4641   256.0 KiB   A01B  0:CDT
 +              4642            5153   256.0 KiB   FFFF  0:CDT_1
 +              5154            5665   256.0 KiB   FFFF  0:BOOTCONFIG1
 +              5666            7713   1024.0 KiB  A015  0:APPSBL
 +              7714            9761   1024.0 KiB  FFFF  0:APPSBL_1
 +  10            9762           10273   256.0 KiB   FFFF  0:ART
 +  11           10274           12321   1024.0 KiB  FFFF  u_env
 +  12           12322           14369   1024.0 KiB  FFFF  s_env
 +  13           14370           16417   1024.0 KiB  FFFF  devinfo
 +  14           16418          294945   136.0 MiB   FFFF  kernel
 +  15           32802          294945   128.0 MiB   FFFF  rootfs
 +  16          294946          573473   136.0 MiB   FFFF  alt_kernel
 +  17          311330          573473   128.0 MiB   FFFF  alt_rootfs
 +  18          573474          574497   512.0 KiB   FFFF  sysdiag
 +  19          574498         7634910   3.4 GiB     FFFF  syscfg
 +
 +# delete p19:
 +
 +Command (? for help): d
 +Partition number (1-19): 19
 +
 +Command (? for help): p
 +Disk /dev/mmcblk0: 7634944 sectors, 3.6 GiB
 +Sector size (logical/physical): 512/512 bytes
 +Disk identifier (GUID): 98101B32-BBE2-4BF2-A06E-2BB33D000C20
 +Partition table holds up to 128 entries
 +Main partition table begins at sector 2 and ends at sector 33
 +First usable sector is 34, last usable sector is 7634910
 +Partitions will be aligned on 2-sector boundaries
 +Total free space is 7060413 sectors (3.4 GiB)
 +
 +Number  Start (sector)    End (sector)  Size       Code  Name
 +                34            1057   512.0 KiB   A012  0:SBL1
 +              1058            2081   512.0 KiB   FFFF  0:BOOTCONFIG
 +              2082            3105   512.0 KiB   A016  0:QSEE
 +              3106            4129   512.0 KiB   FFFF  0:QSEE_1
 +              4130            4641   256.0 KiB   A01B  0:CDT
 +              4642            5153   256.0 KiB   FFFF  0:CDT_1
 +              5154            5665   256.0 KiB   FFFF  0:BOOTCONFIG1
 +              5666            7713   1024.0 KiB  A015  0:APPSBL
 +              7714            9761   1024.0 KiB  FFFF  0:APPSBL_1
 +  10            9762           10273   256.0 KiB   FFFF  0:ART
 +  11           10274           12321   1024.0 KiB  FFFF  u_env
 +  12           12322           14369   1024.0 KiB  FFFF  s_env
 +  13           14370           16417   1024.0 KiB  FFFF  devinfo
 +  14           16418          294945   136.0 MiB   FFFF  kernel
 +  15           32802          294945   128.0 MiB   FFFF  rootfs
 +  16          294946          573473   136.0 MiB   FFFF  alt_kernel
 +  17          311330          573473   128.0 MiB   FFFF  alt_rootfs
 +  18          573474          574497   512.0 KiB   FFFF  sysdiag
 +
 +Command (? for help): v
 +
 +Problem: partitions 15 and 14 overlap:
 +  Partition 15: 32802 to 294945
 +  Partition 14: 16418 to 294945
 +
 +Problem: partitions 17 and 16 overlap:
 +  Partition 17: 311330 to 573473
 +  Partition 16: 294946 to 573473
 +
 +Identified 2 problems!
 +
 +# delete rootfs and recreate it where p19 was with size 512 MiB:
 +
 +Command (? for help): d   
 +Partition number (1-18): 15
 +
 +Command (? for help): n
 +Partition number (15-128, default 15): 
 +First sector (574498-7634910, default = 574498) or {+-}size{KMGTP}: 
 +Last sector (574498-7634910, default = 7634909) or {+-}size{KMGTP}: +512m
 +Current type is 8300 (Linux filesystem)
 +Hex code or GUID (L to show codes, Enter = 8300): 
 +Changed type of partition to 'Linux filesystem'
 +
 +Command (? for help): c
 +Partition number (1-18): 15
 +Enter name: rootfs
 +
 +Command (? for help): p
 +Disk /dev/mmcblk0: 7634944 sectors, 3.6 GiB
 +Sector size (logical/physical): 512/512 bytes
 +Disk identifier (GUID): 98101B32-BBE2-4BF2-A06E-2BB33D000C20
 +Partition table holds up to 128 entries
 +Main partition table begins at sector 2 and ends at sector 33
 +First usable sector is 34, last usable sector is 7634910
 +Partitions will be aligned on 2-sector boundaries
 +Total free space is 6011837 sectors (2.9 GiB)
 +
 +Number  Start (sector)    End (sector)  Size       Code  Name
 +                34            1057   512.0 KiB   A012  0:SBL1
 +              1058            2081   512.0 KiB   FFFF  0:BOOTCONFIG
 +              2082            3105   512.0 KiB   A016  0:QSEE
 +              3106            4129   512.0 KiB   FFFF  0:QSEE_1
 +              4130            4641   256.0 KiB   A01B  0:CDT
 +              4642            5153   256.0 KiB   FFFF  0:CDT_1
 +              5154            5665   256.0 KiB   FFFF  0:BOOTCONFIG1
 +              5666            7713   1024.0 KiB  A015  0:APPSBL
 +              7714            9761   1024.0 KiB  FFFF  0:APPSBL_1
 +  10            9762           10273   256.0 KiB   FFFF  0:ART
 +  11           10274           12321   1024.0 KiB  FFFF  u_env
 +  12           12322           14369   1024.0 KiB  FFFF  s_env
 +  13           14370           16417   1024.0 KiB  FFFF  devinfo
 +  14           16418          294945   136.0 MiB   FFFF  kernel
 +  15          574498         1623073   512.0 MiB   8300  rootfs
 +  16          294946          573473   136.0 MiB   FFFF  alt_kernel
 +  17          311330          573473   128.0 MiB   FFFF  alt_rootfs
 +  18          573474          574497   512.0 KiB   FFFF  sysdiag
 +
 +# delete alt_rootfs and recreate it after the new rootfs with size 512 MiB:
 +
 +Command (? for help): d  
 +Partition number (1-18): 17
 +
 +Command (? for help): n
 +Partition number (17-128, default 17): 
 +First sector (1623074-7634910, default = 1623074) or {+-}size{KMGTP}: 
 +Last sector (1623074-7634910, default = 7634909) or {+-}size{KMGTP}: +512M
 +Current type is 8300 (Linux filesystem)
 +Hex code or GUID (L to show codes, Enter = 8300): 
 +Changed type of partition to 'Linux filesystem'
 +
 +Command (? for help): c
 +Partition number (1-18): 17
 +Enter name: alt_rootfs
 +
 +Command (? for help): p
 +Disk /dev/mmcblk0: 7634944 sectors, 3.6 GiB
 +Sector size (logical/physical): 512/512 bytes
 +Disk identifier (GUID): 98101B32-BBE2-4BF2-A06E-2BB33D000C20
 +Partition table holds up to 128 entries
 +Main partition table begins at sector 2 and ends at sector 33
 +First usable sector is 34, last usable sector is 7634910
 +Partitions will be aligned on 2-sector boundaries
 +Total free space is 4963261 sectors (2.4 GiB)
 +
 +Number  Start (sector)    End (sector)  Size       Code  Name
 +                34            1057   512.0 KiB   A012  0:SBL1
 +              1058            2081   512.0 KiB   FFFF  0:BOOTCONFIG
 +              2082            3105   512.0 KiB   A016  0:QSEE
 +              3106            4129   512.0 KiB   FFFF  0:QSEE_1
 +              4130            4641   256.0 KiB   A01B  0:CDT
 +              4642            5153   256.0 KiB   FFFF  0:CDT_1
 +              5154            5665   256.0 KiB   FFFF  0:BOOTCONFIG1
 +              5666            7713   1024.0 KiB  A015  0:APPSBL
 +              7714            9761   1024.0 KiB  FFFF  0:APPSBL_1
 +  10            9762           10273   256.0 KiB   FFFF  0:ART
 +  11           10274           12321   1024.0 KiB  FFFF  u_env
 +  12           12322           14369   1024.0 KiB  FFFF  s_env
 +  13           14370           16417   1024.0 KiB  FFFF  devinfo
 +  14           16418          294945   136.0 MiB   FFFF  kernel
 +  15          574498         1623073   512.0 MiB   8300  rootfs
 +  16          294946          573473   136.0 MiB   FFFF  alt_kernel
 +  17         1623074         2671649   512.0 MiB   8300  alt_rootfs
 +  18          573474          574497   512.0 KiB   FFFF  sysdiag
 +
 +# create new 'extra' partition p19 with the remaining 2.4 GiB space:
 +
 +Command (? for help): n
 +Partition number (19-128, default 19): 
 +First sector (2671650-7634910, default = 2671650) or {+-}size{KMGTP}: 
 +Last sector (2671650-7634910, default = 7634909) or {+-}size{KMGTP}: 
 +Current type is 8300 (Linux filesystem)
 +Hex code or GUID (L to show codes, Enter = 8300): 
 +Changed type of partition to 'Linux filesystem'
 +
 +Command (? for help): c
 +Partition number (1-19): 19
 +Enter name: extra
 +
 +Command (? for help): p
 +Disk /dev/mmcblk0: 7634944 sectors, 3.6 GiB
 +Sector size (logical/physical): 512/512 bytes
 +Disk identifier (GUID): 98101B32-BBE2-4BF2-A06E-2BB33D000C20
 +Partition table holds up to 128 entries
 +Main partition table begins at sector 2 and ends at sector 33
 +First usable sector is 34, last usable sector is 7634910
 +Partitions will be aligned on 2-sector boundaries
 +Total free space is 1 sectors (512 bytes)
 +
 +Number  Start (sector)    End (sector)  Size       Code  Name
 +                34            1057   512.0 KiB   A012  0:SBL1
 +              1058            2081   512.0 KiB   FFFF  0:BOOTCONFIG
 +              2082            3105   512.0 KiB   A016  0:QSEE
 +              3106            4129   512.0 KiB   FFFF  0:QSEE_1
 +              4130            4641   256.0 KiB   A01B  0:CDT
 +              4642            5153   256.0 KiB   FFFF  0:CDT_1
 +              5154            5665   256.0 KiB   FFFF  0:BOOTCONFIG1
 +              5666            7713   1024.0 KiB  A015  0:APPSBL
 +              7714            9761   1024.0 KiB  FFFF  0:APPSBL_1
 +  10            9762           10273   256.0 KiB   FFFF  0:ART
 +  11           10274           12321   1024.0 KiB  FFFF  u_env
 +  12           12322           14369   1024.0 KiB  FFFF  s_env
 +  13           14370           16417   1024.0 KiB  FFFF  devinfo
 +  14           16418          294945   136.0 MiB   FFFF  kernel
 +  15          574498         1623073   512.0 MiB   8300  rootfs
 +  16          294946          573473   136.0 MiB   FFFF  alt_kernel
 +  17         1623074         2671649   512.0 MiB   8300  alt_rootfs
 +  18          573474          574497   512.0 KiB   FFFF  sysdiag
 +  19         2671650         7634909   2.4 GiB     8300  extra
 +
 +# write the altered GPT and reboot the router:
 +
 +Command (? for help): w
 +
 +Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
 +PARTITIONS!!
 +
 +Do you want to proceed? (Y/N): y
 +OK; writing new GUID partition table (GPT) to /dev/mmcblk0.
 +Warning: The kernel is still using the old partition table.
 +The new table will be used at the next reboot or after you
 +run partprobe(8) or kpartx(8)
 +The operation has completed successfully.
 +
 +root@OpenWrt:~# reboot
 +root@OpenWrt:~# Connection to 192.168.1.1 closed by remote host.
 +Connection to 192.168.1.1 closed.
 +
 +</code>
 +
 +- Finally, you need to do a sysupgrade for the rootfs_data filesystem to grow and fill its new larger partition. So do one last Attended Sysupgrade and profit!
 +
 +- You can also format the 2.4 GiB "extra" partition /dev/mmcblk0p19 (ext4 is always available in this device) and use it for general persistent storage. Note that having extremely large rootfs_data partitions is problematic, as their content need to fit the RAM of the device (512 MiB in this case) during sysupgrade. And for large partitions, it is safer and more efficient to not touch them during sysupgrades anyway. If you so choose, you can mount this "extra" space on boot as /extra or /opt or whatever. You can do this with scripts, or via LuCI by installing the block-mount package. Another recommended package is fstrim.
 ==== OEM easy installation ==== ==== OEM easy installation ====
  
  • Last modified: 2024/07/17 08:31
  • by lanchon