Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Next revisionBoth sides next revision
infrastructure [2021/05/22 09:24] – [Donations] zoruninfrastructure [2024/06/01 08:40] – [Buildbot workers] update with latest state ynezz
Line 10: Line 10:
 | [[:start|Website and documentation]] | [[doku>|Dokuwiki]] | Primary OpenWrt website and documentation/wiki | DigitalOcean droplet | [[https://openwrt.org/stats/|awstats]] | bobafetthotmail, [[developers:jow|jow]], [[developers:thess|thess]], [[user:tmomas|tmomas]], [[developers:zorun|zorun]]  | | [[:start|Website and documentation]] | [[doku>|Dokuwiki]] | Primary OpenWrt website and documentation/wiki | DigitalOcean droplet | [[https://openwrt.org/stats/|awstats]] | bobafetthotmail, [[developers:jow|jow]], [[developers:thess|thess]], [[user:tmomas|tmomas]], [[developers:zorun|zorun]]  |
 | [[https://forum.openwrt.org/|Forum]] | [[https://www.discourse.org/|Discourse]] | Provides a place for people to ask and answer questions about using OpenWrt, developing it, and other topics | DigitalOcean droplet | [[https://forum.openwrt.org/stats/|awstats]] | [[developers:hauke|hauke]], [[developers:jow|jow]], [[developers:thess|thess]], [[user:tmomas|tmomas]]  | | [[https://forum.openwrt.org/|Forum]] | [[https://www.discourse.org/|Discourse]] | Provides a place for people to ask and answer questions about using OpenWrt, developing it, and other topics | DigitalOcean droplet | [[https://forum.openwrt.org/stats/|awstats]] | [[developers:hauke|hauke]], [[developers:jow|jow]], [[developers:thess|thess]], [[user:tmomas|tmomas]]  |
-| [[https://git.openwrt.org/|Git server]] | [[https://git.wiki.kernel.org/index.php/Gitweb|gitweb]] | Primary OpenWrt source code repositories | DigitalOcean droplet | [[https://github.com/openwrt/openwrt/graphs/contributors|gitstats]] | | +| [[https://git.openwrt.org/|Git server]] | [[https://git.wiki.kernel.org/index.php/Gitweb|gitweb]] | Primary OpenWrt source code repositories | DigitalOcean droplet | [[https://github.com/openwrt/openwrt/graphs/contributors|gitstats]] | [[developers:jow|jow]], [[developers:thess|thess]] 
-| [[https://bugs.openwrt.org/|Bug tracker]] | [[https://www.flyspray.org/|Flyspray]] | The place where bugs and feature requests are tracked | DigitalOcean droplet | | | +| [[https://bugs.openwrt.org/|Bug tracker]] | [[https://www.flyspray.org/|Flyspray]] | The place where bugs and feature requests are tracked | DigitalOcean droplet | | [[developers:jow|jow]], [[developers:thess|thess]] 
-| Email | ? | Handles incoming emails to @openwrt.org domain | DigitalOcean droplet | | | +| Email | ? | Handles incoming emails to @openwrt.org domain | DigitalOcean droplet | | [[developers:jow|jow]], [[developers:thess|thess]], [[developers:stintel|stintel]] 
-| [[https://buildbot.openwrt.org|Buildbot master]] | [[https://buildbot.net/|Buildbot]] | Coordinates builders | Hetzner | | | +| [[https://buildbot.openwrt.org|Buildbot master]] | [[https://buildbot.net/|Buildbot]] | Coordinates builders | Hetzner | | [[developers:jow|jow]], [[developers:ynezz|ynezz]] 
-| [[https://downloads.openwrt.org/|Downloads]] | | Hosts all current and released images, SDK, Imagebuilder | Hetzner | [[https://downloads.openwrt.org/stats/|awstats]] | | +| [[https://downloads.openwrt.org/|Downloads]] | | Hosts all current and released images, SDK, Imagebuilder | Hetzner | [[https://downloads.openwrt.org/stats/|awstats]] | [[developers:jow|jow]], [[developers:thess|thess]] 
-| [[https://archive.openwrt.org|Downloads archive]] | | Hosts old OpenWrt images and releases | | | [[developers:wigyori|wigyori]] +| [[https://archive.openwrt.org|Downloads archive]] | | Hosts old OpenWrt images and releases | private co-lo | | [[developers:wigyori|wigyori]] 
-| [[https://sources.openwrt.org/|Sources archive]] | | Hosts a copy of sources of all software used in OpenWrt | Hetzner | | |+| [[https://oldwiki.archive.openwrt.org|Old wiki archive]] | | Old OpenWrt wiki with legacy / non-migrated content | private co-lo | | [[developers:wigyori|wigyori]] 
 +| [[https://sources.openwrt.org/|Sources archive]] | | Hosts a copy of sources of all software used in OpenWrt | Hetzner | | [[developers:jow|jow]], [[developers:thess|thess]] |
 | [[https://freescout.openwrt.org|Freescout]] | [[https://freescout.net|Freescout]] | Facilitates handling of shared mailboxes (testing) | DigitalOcean droplet | | [[developers:stintel|stintel]] | | [[https://freescout.openwrt.org|Freescout]] | [[https://freescout.net|Freescout]] | Facilitates handling of shared mailboxes (testing) | DigitalOcean droplet | | [[developers:stintel|stintel]] |
 +| [[https://lists.openwrt.org/mailman/listinfo|Mailing lists]] | Mailman | Main medium of communication for developers and admin | infradead | | [[developers:dwmw2|dwmw2]] | 
 +| [[https://firmware-selector.openwrt.org/|Firmware selector]] | | Search and download OpenWrt firmware for your device | | | | 
 +| [[https://sysupgrade.openwrt.org/|Sysupgrade server]] | | Online ImageBuilder and upgrade server | | [[https://sysupgrade.openwrt.org/stats|stats]] | [[developers:aparcar|aparcar]] |
  
 ==== Externally-managed services ==== ==== Externally-managed services ====
 We also rely on the following services that are run by external entities: We also rely on the following services that are run by external entities:
  
-^ Service ^ Technology ^ Description ^ Host ^ Liaison ^ +^ Service                                                                ^ Technology                                              ^ Description                                                   ^ Host                                    ^ Liaison  
-| [[https://lists.openwrt.org/mailman/listinfo/openwrt-devel|openwrt-devel mailing list]] | Mailman Main medium of communication for developers infradead | | +| [[https://patchwork.ozlabs.org/project/openwrt/list/|Patch tracking]]  | [[http://jk.ozlabs.org/projects/patchwork/|Patchwork]]  | Used to track patches sent to the openwrt-devel mailing list  | Ozlab                                            | 
-| [[https://lists.openwrt.org/mailman/listinfo/openwrt-adm|openwrt-adm mailing list]] | Mailman Main medium of communication for administrative matters infradead | | +| [[https://github.com/openwrt/openwrt|Github mirror]]                   Github                                                  Mirror of OpenWrt repository. Pull-requests are accepted.     Github                                           
-| [[https://patchwork.ozlabs.org/project/openwrt/list/|Patch tracking]] | [[http://jk.ozlabs.org/projects/patchwork/|Patchwork]] | Used to track patches sent to the openwrt-devel mailing list Ozlab | | +| CDN                                                                    | Fastly                                                  | Content delivery network                                      | {{:fastlylogo-red-png.png?200&nolink}}  | aparcar 
-| [[https://github.com/openwrt/openwrt|Github mirror]] | Github Mirror of OpenWrt repository. Pull-requests are accepted. | Github | |+ 
 +===== Hosting infrastructure and costs ===== 
 +==== Servers ==== 
 +All services that are managed internally are run on servers devoted to the OpenWrt project, except for mailing lists which run on Infradead servers. 
 + 
 +^ Host                                                                                                                                                                                             ^ Servers                                                                                ^ Services                    ^ Admin(s)                                                                            ^ Expected cost     ^ Actual cost       ^ Comment                      ^ 
 +| {{:netcup_logo_rgb_color_vorschau.png?400|}} | [[https://www.netcup.de/bestellen/produkt.php?produkt=2898|RS 8000 G9.5 a1 12M]] + 6TB storage Downloads [[developers:aparcar|aparcar]] 100€ / month     | Free      | Open source project support 
 +| [[https://www.digitalocean.com/?utm_medium=opensource&utm_source=OpenWrt|{{:do_powered_by_badge_blue.png?400|}}]]  | 7 droplets (prod) | wiki, forum, git, mails...  | [[developers:jow|jow]], [[developers:ynezz|ynezz]], [[developers:stintel|stintel]]  | 344.00 $ / month  | Free              | Open source project support 
 +| Microsoft | 2 VMs | Buildbot workers            | [[developers:ynezz|ynezz]]                                                          | 10.000 $ year   | Free              | Open source project support 
 +| Hetzner | [[https://www.hetzner.com/dedicated-rootserver/matrix-ex|ex40]] | Buildbot master             | [[developers:nbd|nbd]], ? | 50 € month      | 50 € month      | Paid privately               | 
 +| Hetzner | [[https://www.hetzner.com/sb/|sb59]] | Archive                     [[developers:blogic|blogic]]                                                        | 49.50 € / month   | 49.50 € / month   | Paid privately               
 +| Hetzner | [[https://www.hetzner.com/dedicated-rootserver/matrix-ax|ax61-nvme]] + 2x 1.92 TB SSD  Downloads                   [[developers:jow|jow]]                                                              | 158.27 € / month  158.27 € / month  Paid privately               |
  
-===== Servers ===== +==== Buildbot workers ==== 
-All services that are managed internally are run on servers devoted to the OpenWrt project.+Buildbot workers detect updates to the source and kick off the build process for each supported device.
  
-^ Host ^ Servers Services Money ^ Admin(s) ^ +^ Host       Name                Server spec                                                                    Nb servers   ^ Admin(s)                                                                         ^ Cost                                                                                            ^ Comment   
-| Hetzner | 3x [[https://www.hetzner.com/dedicated-rootserver/matrix-ex|ex40]] | buildbot master, downloads | Paid privately | jow, ? +| Hetzner    | truecz-dock-01   | Xeon E3/32GB/2TB                                                               | 1            | [[developers:ynezz|ynezz]]                                                        32€ / month  | Paid privately                                                                                  
-Hetzner 2x XeonE3/32GB/2TB buildbot builders Paid privately | ynezz | +OSUOSL     osuosl-vm-*         [[infrastructure:openstack|OpenStack VM]] (12 cores, 32 GB RAM, 320 GB disk)   4            | [[developers:zorun|zorun]], [[developers:ynezz|ynezz]], [[developers:jow|jow]]   | Free             | Open source project support                                                                     
-DigitalOcean Many droplets wikiforumgit, mails | Free (open source project support) | jow, ? +OSUOSL     osuosl-dock-*       12C24T64 GB RAM (2x Xeon X5680 3.33 GHz)                                   | 2            | OpenWrt sysadmins                           | Free             | Open source project support                                                                     
-OSUOSL | [[infrastructure:openstack|OpenStack VMs]] | buildbot builders Free (open source project support) bjonglezynezzjow |+VZFFNRMO   vzffnrmo-dock-*     | Xeon E31230/16GB/100GB                                                         | 1            | OpenWrt sysadmins                                                                | ?                | Provided by the [[https://karlsruhe.freifunk.net|Freifunk Karlsruhe Project / VZFFNRMO e.V.]]   | 
 +FFFFM      ffffm-dock-*             | 28C56T448GB RAM, 1.8TB (2x Xeon E5-2680v4 2.4GHz)                          | 1            | OpenWrt sysadmins                                                                                 | Free             | Provided by [[https://ffm.freifunk.net/|Freifunk Frankfurt am Main e.V.]]                       |
  
-===== Buildbot ===== +=== Worker hardware requirements ===
-==== Build matrix ==== +
-There are several buildbot instances: one for master, and one for each release. In addition, there is phase1 to build images, and phase2 to build packages.+
  
-^ Version ^ Phase 1 Images ^ Phase 2 - Packages ^ Fail logs ^ Image downloads ^ +  * **CPU**: a good worker needs 4-8 CPU cores for phase1/images, and 8-16 CPU cores for phase2/packages. Larger machines are split into several workers using Docker
-| master | [[https://buildbot.openwrt.org/master/images/|Phase 1 Images]] | [[https://buildbot.openwrt.org/master/packages/|Phase 2 - Packages]] | [[https://downloads.openwrt.org/snapshots/faillogs/|Fail logs]] | [[https://downloads.openwrt.org/snapshots/targets/|images]] | +  * **Memory**each worker needs a minimum of 1 GB of RAM for each CPU hardware thread, 2 GB thread is preferred For example, a 16 cores 32 threads machine needs between 32 GB and 64 GB of RAM in total
-| 21.02 | [[https://buildbot.openwrt.org/openwrt-21.02/images/|21.02 Images]] | [[https://buildbot.openwrt.org/openwrt-21.02/packages/|21.02 Packages]] | [[https://downloads.openwrt.org/releases/faillogs-21.02/|Fail logs]] | [[https://downloads.openwrt.org/releases/21.02-SNAPSHOT/targets/|images]] | +  * **Disk:** each worker needs 50 GB of disk space (phase1/images) or 100 GB of disk space (phase2/packages). 
-| 19.07 | [[https://buildbot.openwrt.org/openwrt-19.07/images/|19.07 Images]] | [[https://buildbot.openwrt.org/openwrt-19.07/packages/|19.07 Packages]] [[https://downloads.openwrt.org/releases/faillogs-19.07/|Fail logs]] [[https://downloads.openwrt.org/releases/19.07-SNAPSHOT/targets/|images]] | +==== Other costs ==== 
-| 18.06 | [[https://buildbot.openwrt.org/openwrt-18.06/images/|18.06 Images]] | [[https://buildbot.openwrt.org/openwrt-18.06/packages/|18.06 Packages]] [[https://downloads.openwrt.org/releases/faillogs-18.06/|Fail logs]] | [[https://downloads.openwrt.org/releases/18.06-SNAPSHOT/targets/|images]] |+^ What ^ Provider ^ Admin(s) ^ Expected cost ^ Actual cost ^ Comment ^ 
 +Domain names Tucows SPI Hostmaster / [[developers:jow|jow]] | 20-40 € year Free Held by Tucows on behalf of SPI |
  
-==== Buildbot machines ==== +===== Buildbot build matrix ===== 
-Buildbot machines detect updates to the source and kick off the build process for each supported device.+There are several buildbot instances, phase1 to build images, and phase2 to build packages.
  
-Server Hardware Host Contact +Version  Phase 1 - Images                                                                             Phase 2 - Packages                                                               Fail logs                                                             ^ Image downloads                                                            
-fsf-dock-4x 16C/32T gcc compile farm OSUOSL bjonglez |+master   | [[https://buildbot.openwrt.org/images/|Phase 1 Images]]                             | [[https://buildbot.openwrt.org/master/packages/|Phase 2 Packages]]             [[https://downloads.openwrt.org/snapshots/faillogs/|Fail logs]]       | [[https://downloads.openwrt.org/snapshots/targets/|images]]                | 
 +| 23.05    | [[https://buildbot.openwrt.org/images/#/waterfall?tags=openwrt-23.05|23.05 Images]]  | [[https://buildbot.openwrt.org/openwrt-23.05/packages/|23.05 Packages]]  | [[https://downloads.openwrt.org/releases/faillogs-23.05/|Fail logs]]  | [[https://downloads.openwrt.org/releases/23.05-SNAPSHOT/targets/|images]] 
 +| 22.03    | [[https://buildbot.openwrt.org/images/#/waterfall?tags=openwrt-22.03|22.03 Images]]  | [[https://buildbot.openwrt.org/openwrt-22.03/packages/|22.03 Packages]]          | [[https://downloads.openwrt.org/releases/faillogs-22.03/|Fail logs]]  | [[https://downloads.openwrt.org/releases/22.03-SNAPSHOT/targets/|images]]  |
  
 ===== Finances ===== ===== Finances =====
Line 61: Line 77:
  
 ===== Donations ===== ===== Donations =====
 +See the [[:donate|Donations]] page.
  
-See the [[:donate|Donations]] page 
  • Last modified: 2024/11/06 18:45
  • by ynezz