| Both sides previous revision Previous revision Next revision | Previous revision |
| docs:user-guide:services:bandwidthd [2018/02/17 18:11] – ↷ Page moved from doc:howto:bandwidthd to docs:user-guide:services:bandwidthd bobafetthotmail | docs:guide-user:services:network_monitoring:bandwidthd [2021/12/19 15:35] (current) – Add the information to add option index_page 'index.php' in the uhttpd config file jmlacroix |
|---|
| ====== Bandwidthd ====== | ====== Bandwidthd ====== |
| [[http://bandwidthd.sourceforge.net/]] | |
| |
| **Note: bandwidthd in trunk (DD) or LEDE tracks the SMTP data while the version in the previous version of OpenWrt (Chaos Calmer and older) tracks the P2P data (Gnutella, eDonkey, etc.). All of the other data (FTP, UDP, TCP, etc.) is tracked the same way in all versions.** | [[http://bandwidthd.sourceforge.net/]] (openWRT older than 17) |
| | |
| | [[https://github.com/NethServer/bandwidthd]] (openWRT 17 or higher) |
| | |
| | **Note: bandwidthd in LEDE/OpenWRT 17.01 or higher tracks the SMTP data (ports 25, 465 and 587) while the version in the previous version of OpenWrt (Chaos Calmer and older) tracks the P2P data (Gnutella, eDonkey, etc.). All of the other data (FTP, UDP, TCP, etc.) is tracked the same way in all versions.** |
| |
| **Bandwidthd is composed of 4 packages: | **Bandwidthd is composed of 4 packages: |
| ** | ** |
| - **bandwidthd**: this package cannot save in a postgresql database. This package is currently available in trunk, 15.05.1 and OpenWrt version older than Chaos Calmer (ex: 14.07): for 15.05, it has to be compiled. This allows the router to generate bandwidth data that is accessible with a browser at http://192.168.1.1/bandwidthd. **__//Use this package if you have no intention of storing the data on a postgresql or sqlite database.//__** | - **bandwidthd**: this package cannot save in a postgresql or sqlite database. This allows the router to generate bandwidth data that is accessible with a browser at http://192.168.1.1/bandwidthd. **__//Use this package if you have no intention of storing the data on a postgresql or sqlite database.//__** |
| - **bandwidthd-pgsql**: this package can do everything the bandwidthd package can do, but in addition it can save the data to a postgresql database that can reside on the router or on another server. This data can be analyzed by a series of PHP scripts and visualized with a browser. Note that the router, the posgresql database and the graphing of the data by PHP can be on three different systems, one for each task (i.e.: __collecting the data__, __storing the postgresql data__ and __generating the graphs with PHP__). Note that postgresql does not have to be installed on the router if the postgresql data is not stored on the router. __bandwidthd-pgsql__ requires the package __libpq__ to be installed: this package is already installed if postgresql is installed. **NOTE: install only one of the package: bandwidthd, bandwidthd-pgsql or bandwidthd-sqlite.** | - **bandwidthd-pgsql**: this package can do everything the bandwidthd package can do, but in addition it can save the data to a postgresql database that can reside on the router or on another server. This data can be analyzed by a series of PHP scripts and visualized with a browser. Note that the router, the posgresql database and the graphing of the data by PHP can be on three different systems, one for each task (i.e.: __collecting the data__, __storing the postgresql data__ and __generating the graphs with PHP__). Note that postgresql does not have to be installed on the router if the postgresql data is not stored on the router. **NOTE: install only one of the package: bandwidthd, bandwidthd-pgsql or bandwidthd-sqlite.** |
| - **bandwidthd-php**: This package contains the PHP files that are installed in /www/phphtdocs on the router. It is only required if the OpenWrt router serves as the web server to generate the graphs from the data on the postgresql database by pointing the browser to http://192.168.1.1/phphtdocs. **NOTE: if you installed bandwidthd-sqlite, do not install bandwidthd-php as bandwidthd-sqlite contains the required PHP files to graph the data.** | - **bandwidthd-php**: This package contains the PHP files that are installed in /www/phphtdocs on the router. It is only required if the OpenWrt router serves as the web server to generate the graphs from the data on the postgresql database by pointing the browser to http://192.168.1.1/phphtdocs. **NOTE: if you installed bandwidthd-sqlite, do not install bandwidthd-php as bandwidthd-sqlite contains the required PHP files to graph the data.** |
| - **bandwidthd-sqlite**. This package, in addition to do what the plain bandwidthd package can do, stores the data in a sqlite database on the router and graph the data using PHP. In a way, it is a combination of bandwidthd-pgsql and bandwidthd-php together, but using sqlite instead of postgresql. **NOTE: install only one of the package: bandwidthd, bandwidthd-pgsql or bandwidthd-sqlite.** | - **bandwidthd-sqlite**. This package, in addition to do what the plain bandwidthd package can do, stores the data in a sqlite database on the router and graph the data using PHP. In a way, it is a combination of bandwidthd-pgsql and bandwidthd-php together, but using sqlite instead of postgresql. **NOTE: install only one of the package: bandwidthd, bandwidthd-pgsql or bandwidthd-sqlite.** |
| |
| The availability of each package varies according to which version of OpenWrt you have: | The availability of each package varies according to which version of OpenWrt/LEDE you have: |
| |
| **bandwidthd** | **bandwidthd** |
| * In Barrier Breaker (14.07) in "oldpackages" as a package. | * In Barrier Breaker (14.07) in "oldpackages" as a package. |
| * In original Chaos Calmer (15.05) __it is not available as a precompiled package__: you have to compile it yourself and install it. See https://wiki.openwrt.org/doc/howto/build to have details on how to compile a package. | * In original Chaos Calmer (15.05) __it is not available as a precompiled package__: you have to compile it yourself and install it. See https://wiki.openwrt.org/doc/howto/build to have details on how to compile a package. |
| * In the minor fix release of Chaos Calmer (15.05.1) it is available as a package. | * As a package in 15.05.1 or higher. |
| * In trunk (DD) or LEDE as a package (https://downloads.openwrt.org/snapshots/trunk/). | |
| |
| **bandwidthd-pgsql** | **bandwidthd-pgsql** |
| * In the original Chaos Calmer (15.05) __it is not available as a precompiled package__: you have to compile it yourself and install it. See https://wiki.openwrt.org/doc/howto/build to have details on how to compile a package. | * In the original Chaos Calmer (15.05) __it is not available as a precompiled package__: you have to compile it yourself and install it. See https://wiki.openwrt.org/doc/howto/build to have details on how to compile a package. |
| * In the minor fix release of Chaos Calmer (15.05.1) it is available as a package. | * In the minor fix release of Chaos Calmer (15.05.1) it is available as a package. |
| * In trunk (DD) or LEDE as a package (https://downloads.openwrt.org/snapshots/trunk/). | * As a package in 15.05.1 or higher. |
| | * |
| **bandwidthd-php** | **bandwidthd-php** |
| * Available in trunk (DD) or LEDE as a package and uses php7 (php5 is not available anymore in trunk). The PHP files found in the trunk package work fine with Chaos Calmer 15.05 or 15.05.1 with php5 (tested) and should work for php7 (not tested and php7 is not available in Chaos Calmer) | * Available in LEDE/OpenWRT 17.01 or higher as a package and uses php7 or php8. The PHP files found in these versions work fine with Chaos Calmer 15.05 or 15.05.1 with php5 (tested). |
| |
| **bandwidthd-sqlite** | **bandwidthd-sqlite** |
| * Available trunk (DD) and LEDE as a package. | * Available in LEDE/OpenWRT 17.01 or higher. |
| ===== Installation ===== | ===== Installation ===== |
| |
| |
| With all dependencies it uses around 500 kB of storage space. If you have not changes the IP address of your router (192.168.1.1), then the web page with the bandwidthd data will be available at http://192.168.1.1/bandwidthd. If you have changed the IP address of your router, then edit the /etc/config/bandwidthd file to correct the address. | With all dependencies it uses around 500 kB of storage space. If you have not changes the IP address of your router (192.168.1.1), then the web page with the bandwidthd data will be available at http://192.168.1.1/bandwidthd. If you have changed the IP address of your router, then edit the /etc/config/bandwidthd file to correct the address. |
| |
| ===== Limitations and CPU usage ===== | |
| |
| {{page>meta:infobox:outdated&noheader&nofooter&noeditbtn}} | |
| |
| **NOTE: This information is old. If you have more recent information about this, please correct the wiki or contact the maintainer of the package to correct the wiki.** | |
| |
| Bandwidthd uses a lot of cpu power to process and analyse packages, on average [[:toh:tp-link/tl-wr841nd|TP-Link TL-WR841ND]] router with 400 Mhz CPU it maxes out CPU with 100% usage around 3 Mpbs speeds. So if you have faster connection it throttles it down to around 3 Mbps because CPU can't handle more packages. | |
| |
| Note: As of build r31249 this is not necessarily true. While bandwidthd still maxes out at 100% cpu and >4 load average even while idle, the router will happily exceed 70mbps download speeds even with QoS on, which by itself uses >80% cpu at those speeds. In other words, even while bandwidthd is using 100% cpu, the above router will still easily exceed most people's connection speeds without the CPU becoming a limitation. | |
| |
| If the data is stored in a postgresql database with bandwidthd-pgsql and the graphs are made by another system than the OpenWrt router, the CPU requirements is much lighter. | |
| |
| ===== Configuration ===== | ===== Configuration ===== |
| |
| **__Web Server Setup:__** Note that the web server can be on a remote web server and does not have to reside on the OpenWrt router. Consult http://wiki.openwrt.org/doc/howto/php for installation of PHP on OpenWrt. For other OSes (Linux, NetBSD, Unix, etc.) consult the help on the respective OS. You will also have to configure the web server to work with PHP. Running this on OpenWrt takes about 2 MB of storage space for the various packages. | **__Web Server Setup:__** Note that the web server can be on a remote web server and does not have to reside on the OpenWrt router. Consult http://wiki.openwrt.org/doc/howto/php for installation of PHP on OpenWrt. For other OSes (Linux, NetBSD, Unix, etc.) consult the help on the respective OS. You will also have to configure the web server to work with PHP. Running this on OpenWrt takes about 2 MB of storage space for the various packages. |
| - Copy the contents of phphtdocs into your web tree somewhere: these files are available at http://bandwidthd.sourceforge.net/ or in the **bandwidthd-php** package: the package will install them automatically and puts then in /www/phphtdocs. | - Copy the contents of phphtdocs into your web tree somewhere: these files are available at http://bandwidthd.sourceforge.net/, https://github.com/NethServer/bandwidthd or in the **bandwidthd-php** package: the package will install them automatically and puts then in /www/phphtdocs. |
| - Edit the file bandwidthd-php in /etc/config to set your db connect string (__$db_connect_string = "host=192.168.1.1 user=postgres dbname=bandwidthd"__). The variables __host__, __user__ and __dbname__ have to be edited in order to connect to the database. | - Edit the file bandwidthd-php in /etc/config to set your db connect string (__$db_connect_string = "host=192.168.1.1 user=postgres dbname=bandwidthd"__). The variables __host__, __user__ and __dbname__ have to be edited in order to connect to the database. |
| - On OpenWrt, the following packages will be installed automatically: **libpcre**; **libxml2**; **php7**; **php7-cgi**; **php7-mod-pgsql** and **php7-mod-gd**. On another system, the corresponding packages should be installed. Note that it also works with php5 which to be used on Chaos Calmer as php7 is not available. | - On OpenWrt, the following packages will be installed automatically: **libpcre**; **libxml2**; **php7**; **php7-cgi**; **php7-mod-pgsql** and **php7-mod-gd**. On another system, the corresponding packages should be installed. Note that it also works with php5 which to be used on Chaos Calmer as php7 is not available. |
| | - Starting at the end of 2021, **php8** maybe installed instead of **php7**. |
| - With php5, a zoneinfo package should be installed (ex: **zoneinfo-northamerica** or **zoneinfo-europe**). This is required to set the date.timezone value in php.ini. | - With php5, a zoneinfo package should be installed (ex: **zoneinfo-northamerica** or **zoneinfo-europe**). This is required to set the date.timezone value in php.ini. |
| - If you are not using the **bandwidthd-php** package on OpenWrt, the file /etc/php.ini should be edited to have the following: **short_open_tag = On**: the reason is that the php files taken from http://bandwidthd.sourceforge.net/ have the php short tag (<? ?>) instead of the normal tag (<?php ?>). If you do not do it, then you will get a bunch of garbage on the screen. If you installed the **bandwidthd-php** package on your router, then this is not necessary to set the **short_open_tag =** at **On** as the tags in the php files have been corrected to have <?php ?>. | - If you are not using the **bandwidthd-php** package from OpenWrt (i.e. you got the php files on the web at one of the links above), the file /etc/php.ini should be edited to have the following: **short_open_tag = On**: the reason is that the php files taken from http://bandwidthd.sourceforge.net/ or https://github.com/NethServer/bandwidthd have the php short tag (<? ?>) instead of the normal tag (<?php ?>). If you do not do it, then you will get a bunch of garbage on the screen. If you installed the **bandwidthd-php** package on your router, then this is not necessary to set the **short_open_tag =** at **On** as the tags in the php files have been corrected to have <?php ?>. |
| - In the /etc/php.ini file with php5, the **date.timezone =** should be edited to a valid value (ex:**date.timezone = "America/Montreal"**). Without the **date.timezone** set to something valid, the graphs will not be drawn. | - In the /etc/php.ini file a **date.timezone =** whould be present (ex:**date.timezone = "America/Montreal"**). __Without the **date.timezone** set to something valid, the graphs will not be drawn__. |
| - If you are using uhttpd, the following line should be added to /etc/config/uhttpd: **list interpreter ".php=/usr/bin/php-cgi"** | - In the /etc/php.ini file the **display_errors** should be set to off as many variables are not defined. If you have **display_errors = On** the web page will most probably not work. |
| You should now be able to access the web application and see you graphs. All graphing | - If you are using uhttpd, the following two lines should be added to /etc/config/uhttpd: **list interpreter '.php=/usr/bin/php-cgi'** and **option index_page 'index.php'**. |
| is done by graph.php, all parameters are passed to it in it's url. You can create | You should now be able to access the web application and see you graphs. All graphing is done by graph.php, all parameters are passed to it in it's url. You can create custom urls to pull custom graphs from your own index pages, or use the canned reporting system. |
| custom urls to pull custom graphs from your own index pages, or use the canned | |
| reporting system. | |
| | |
| ===== Monitoring the Internet bandwidth of each host in a network ===== | |
| One advantage of bandwidthd is the ability to provide the bandwidth usage of each host in a network. If you want to monitor the Internet usage of each host in your network, you can set bandwidthd in an OpenWrt system between the Internet router and the network. | |
| | |
| Here is the way that I do it in my home: | |
| * I have a router (non-OpenWrt, although an OpenWrt system can be used) connected to the Internet on the WAN side that creates a private network (192.168.XXX.XXX/24) on the LAN side. | |
| * I have a TP-WDR3600, with OpenWrt (Chaos Calmer 15.05) setup to monitor the bandwidth using bandwidthd (from the package bandwidthd-pgsql) that stores the data on a postgresql database on a server on the private network: this server also has the PHP files to generate the graphs with the web server application (Apache. Any web server supporting PHP can be used): the PHP files are the one from bandwidthd-php. | |
| * The non-OpenWrt router is connected to port 3 of the TP-WDR3600. | |
| * On the TP-WDR3600, port 4 is connected to port 2: yes there is a small Cat5 cable between port 4 and 2 of the TP-WDR3600. | |
| * Port 1 of the TP-WDR3600 is connected to a switch to which the whole private network is connected: the TP-WDR3600 is between the non-OpenWrt router (on port 3) and the private network (on port 1). | |
| * Traffic from the Internet arrives from the non-OpenWrt router to port 3 of the TP-WDR3600 and goes out from port 4 to go back in on port 2 and then go out on port 1 to the switch on the private network. | |
| * The same way, traffic to the internet leaves the switch on the private network to enter the TP-WDR3600 on port 1, goes out on port 2, enter again on port 4 and then goes out on port 3 to the non-OpenWrt router that direct it to the modem and the Internet. | |
| * Ports 3 and 4 are specifically configured just to sniff the packages: they do not have any IP address: see the configuration below. | |
| * Ports 1 and 2 are regular ports that can be accessed with the IP address of the router: they are used to access the router from the private network by HTTP or SSH. The WAN port is not used as the TP-WDR3600 is not performing any router activities. | |
| | |
| NOTE: the TP-WDR3600 also provides WiFi for the private network. | |
| | |
| This small diagram may help you understand: | |
| | |
| <file> | |
| +----------+ +---------------------+ +--------+ +-----------------+ | |
| | Internet | | TP-WDR3600 1|----| Switch |---| Private network | | |
| +----------+ | OpenWrt | +--------+ +-----------------+ | |
| | | | | | |
| | |WAN: no connection 2|-- +-----------------+ | |
| +--------+ | | | | Postgresql and | | |
| | Router |------|3 4|-- | web php server | | |
| +--------+ +---------------------+ +-----------------+ | |
| </file> | |
| | |
| Here is the **/etc/network** configuration files of my TP-WDR3600: | |
| | |
| <file> | |
| config interface 'loopback' | |
| option ifname 'lo' | |
| option proto 'static' | |
| option ipaddr '127.0.0.1' | |
| option netmask '255.0.0.0' | |
| | |
| config globals 'globals' | |
| option ula_prefix 'xxxx:xxxx:xxxx::/48' | |
| | |
| config interface 'lan' | |
| option ifname 'eth0.1' | |
| option force_link '1' | |
| option type 'bridge' | |
| option proto 'static' | |
| option netmask '255.255.255.0' | |
| option ip6assign '60' | |
| option dns '192.168.xxx.xxx' | |
| option ipaddr '192.168.xxx.xxx' | |
| option gateway '192.168.xxx.xxx' | |
| option delegate '0' | |
| | |
| config switch | |
| option name 'switch0' | |
| option reset '1' | |
| option enable_vlan '1' | |
| option mirror_source_port '0' | |
| option mirror_monitor_port '0' | |
| | |
| config switch_vlan | |
| option device 'switch0' | |
| option vlan '1' | |
| option vid '1' | |
| option ports '0t 2 3' | |
| | |
| config switch_vlan | |
| option device 'switch0' | |
| option vlan '2' | |
| option ports '0t 1' | |
| option vid '2' | |
| | |
| config switch_vlan | |
| option device 'switch0' | |
| option vlan '3' | |
| option vid '3' | |
| option ports '0t 4' | |
| | |
| config switch_vlan | |
| option device 'switch0' | |
| option vlan '4' | |
| option vid '4' | |
| option ports '0t 5' | |
| | |
| config interface 'mon1' | |
| option proto 'none' | |
| option ifname 'eth0.3' | |
| | |
| config interface 'mon2' | |
| option proto 'none' | |
| option ifname 'eth0.4' | |
| | |
| config interface 'sniff' | |
| option proto 'static' | |
| option ifname 'eth0.3 eth0.4' | |
| option type 'bridge' | |
| </file> | |
| | |
| And here is the /etc/config/bandwidthd file: | |
| <file> | |
| config bandwidthd | |
| option dev br-sniff | |
| option subnets "192.168.xxx.xxx/24" | |
| option skip_intervals 0 | |
| option graph_cutoff 1024 | |
| option promiscuous true | |
| option output_cdf false | |
| option recover_cdf false | |
| option filter ip | |
| option graph true | |
| option meta_refresh 150 | |
| option pgsql_connect_string "user = postgres dbname = bandwidthd host = 192.168.xxx.xxx" | |
| option sensor_id "my_sensor_name" | |
| </file> | |