[OpenWrt Wiki] Update package lists

The package list and package Indexes are generated by a script, so we will first need to move the current data to a permanent place (so it can still be viewed in the wiki as a “old stable” release), rename it, and then edit two strings in the script to make it point to the new release.

In the following instructions, the “new release” is 21.02, the “old release” is 19.07 and the “legacy/older release” is lede 17.1. Adjust the versions accordingly to your current situation.

Download the current release of the “CSV dump of the package database, updated daily” from this page and rename it to have the release version in it. For example packages_dump_tab_separated_19.07.zip. We will upload this later when we adjust the page.

You will need ssh access and after you became root user in the openwrt.org server to do the following

we can find current package files in this folder

root@wiki-01:~# ls -l /var/www/dokuwiki/data/pages/packages
total 1120
-rw-r--r-- 1 www-data www-data   4021 Jan 16  2017 _template.txt
-rw-r--r-- 1 www-data www-data    146 Nov 26  2016 architectures.txt
drwxr-xr-x 2 www-data www-data   4096 Sep 12 03:57 index
drwxr-xr-x 2 www-data www-data   4096 Mar 23  2019 index_lede17_1
drwxr-xr-x 2 www-data www-data   4096 Feb  2  2020 index_owrt18_6
-rw-r--r-- 1 www-data www-data 384579 Sep 12 21:44 logfile.txt
-rw-r--r-- 1 www-data www-data   1823 Aug  5  2018 maintainer-view.txt
drwxr-xr-x 2 www-data www-data 290816 Sep 12 03:57 pkgdata
drwxr-xr-x 2 www-data www-data 176128 Mar 10  2019 pkgdata_lede17_1
drwxr-xr-x 2 www-data www-data 245760 Feb 14  2021 pkgdata_owrt18_6
-rw-r--r-- 1 www-data www-data   3098 Dec 19  2020 start.txt
drwxr-xr-x 2 www-data www-data   4096 Aug  5  2018 table
drwxr-xr-x 2 www-data www-data   4096 Aug  5  2018 table_lede17_1
drwxr-xr-x 2 www-data www-data   4096 Feb  3  2020 table_owrt18_6

We start by deleting older package tables and indexes. Legacy releases only get the CSV file with the data export (which is why you saved that file above).

rm -Rv /var/www/dokuwiki/data/pages/packages/pkgdata_lede17_1/
rm -Rv /var/www/dokuwiki/data/pages/packages/index_lede17_1/
rm -Rv /var/www/dokuwiki/data/pages/packages/table_lede17_1/

Then we rename the current pkgdata/index folders to a folder with release version in them (the ones with the current release will be re-generated later by the script), and copy the table folder (we will edit it later to load the data from these package lists)

cd /var/www/dokuwiki/data/pages/packages 
mv pkgdata pkgdata_owrt19_7
mv index index_owrt19_7
cp -R table table_owrt19_7
chown www-data:www-data -R /var/www/dokuwiki/data/pages/packages

Now we mass-edit the package indexes to point to the right package data folder

cd /var/www/dokuwiki/data/pages/packages/index_owrt19_7/
sed -i 's/pkgdata/pkgdata_owrt19_7/g' *

Now we mass-edit the package data to use different variable names

cd /var/www/dokuwiki/data/pages/packages/pkgdata_owrt19_7/
sed -i 's/Dependencies_pkg-dependencies/Dependencies_pkg-dependency_19s/g' *
sed -i 's/Categories_pkg-categorys/Categories_pkg-category_19s/g' *

Now we mass-edit the table page to use different variable names (and be consistent with the changes above)

cd /var/www/dokuwiki/data/pages/packages/table_owrt19_7
sed -i 's/Dependencies_pkg-dependencies/Dependencies_pkg-dependency_19s/g' *
sed -i 's/Categories_pkg-categorys/Categories_pkg-category_19s/g' *
sed -i 's/Name_pkg-dependencies/%pageid%/g' *
sed -i 's/Package table/Package table 19.07/g' *

Now we edit the package indexing script

cd /root
nano listpackages-server

Scroll down a bit and edit

#package list generic link
package_list_main_URL="https://downloads.openwrt.org/releases/19.07.0/packages/"
#"https://downloads.openwrt.org/snapshots/packages/"

#targets list
OpenWrt_targets_link="https://downloads.openwrt.org/releases/19.07.0/targets"
#"https://downloads.openwrt.org/snapshots/targets/"

#types of repos there are currently
OpenWrt_repos="base luci routing telephony packages"

#the only release we have currently
package_release="OpenWrt-19.07.0"   #"snapshot"

#setting variables for release

github_folder="openwrt-19.07"  #"master"

The first two links are the folders you find in the release folder from this webpage on the download server https://downloads.openwrt.org/releases/

The “package_release” variable is just a description string that is used to write the release version in the packages

Thhe “github_folder” variable is the name of the branch of the release in the git source, you can see the branch names on this github page from OpenWrt's github mirror https://github.com/openwrt/openwrt/branches

So with this in mind, the above becomes

#package list generic link
package_list_main_URL="https://downloads.openwrt.org/releases/21.02.0/packages/"
#"https://downloads.openwrt.org/snapshots/packages/"

#targets list
OpenWrt_targets_link="https://downloads.openwrt.org/releases/21.02.0/targets"
#"https://downloads.openwrt.org/snapshots/targets/"

#types of repos there are currently
OpenWrt_repos="base luci routing telephony packages"

#the only release we have currently
package_release="OpenWrt-21.02.0"   #"snapshot"

#setting variables for release

github_folder="openwrt-21.02"  #"master"

Save and close the script.

start the script so it starts to index the new packages.

nohup /root/listpackages-server &

If launched this way it is run as background task. You can disconnect from SSH and it will still run.

Now it's time to edit the page in here to point to the right package lists and indexes, and remove legacy release entries. You can also upload the CSV file.

If you don't want to wait for the script to finish, you can also launch a page rescan so the wiki will re-read all text files and notice this manual change.

nohup  sudo -u www-data /usr/bin/php7.0 /var/www/dokuwiki/bin/indexer.php -c  &