Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
| docs:guide-user:network:wan:multiwan:mwan3 [2021/05/01 10:51] – [Creating additional WAN interfaces] jamesmacwhite | docs:guide-user:network:wan:multiwan:mwan3 [2024/11/15 19:05] (current) – Internet connectivity may take up to 180 seconds to settle in some routers. Update sleep from 10 to 180 seconds. jeperez | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== mwan3 (Load balancing/ | + | ====== mwan3 (Multi WAN load balancing/ |
| - | * Latest | + | * 23.05: |
| - | * Latest | + | * 22.03: Latest release: 2.11.7 |
| + | * 21.02: | ||
| ==== About mwan3 ==== | ==== About mwan3 ==== | ||
| Line 14: | Line 15: | ||
| * The firewall mask (default '' | * The firewall mask (default '' | ||
| - | ==== Why should I use mwan3? ==== | + | === Overview of how routing with mwan3 works === |
| - | * If you have multiple internet connections and you want to control what traffic goes through which specific WAN interface. | + | |
| - | * Mwan3 can handle multiple levels of primary and backup interfaces, load-balanced or not. Different sources can have different primary or backup WANs. | + | |
| - | * Mwan3 uses netfilter mark mask to be compatible with other packages (such as OpenVPN, PPTP VPN, QoS-script, Tunnels, etc) as you can configure traffic to use the default routing table. | + | |
| - | * Mwan3 can also load-balance traffic originating from the router itself | + | |
| - | + | ||
| - | ==== How mwan3 load-balancing works ==== | + | |
| - | * mwan3 uses normal Linux policy routing to balance outgoing traffic over multiple WAN connections | + | |
| - | * Linux outgoing network traffic load-balancing is performed on a per-IP connection basis -- it is not channel-bonding, | + | |
| - | * As such load-balancing will help speed multiple separate downloads or traffic generated from a group of source PCs all accessing different sites but it will not speed up a single download from one PC (unless the download is spread across multiple IP streams such as by using a download manager) | + | |
| - | + | ||
| - | ==== Architecture of mwan3 ==== | + | |
| - | * mwan3 is triggered by [[docs: | + | |
| - | * Once all the routes and rules are initially set up, mwan3 exits. The kernel takes care of all the routing decisions. If a new interface hotplug event occurs, mwan3 will run again to adjust route and tables as needed. | + | |
| - | * A monitoring script (mwan3track) runs in the background checking if each WAN interface is up using a connectivity test (default is ping). If an interface goes down, the script issues a hotplug event to cause mwan3 to adjust the routing tables of the interface that has gone down. | + | |
| - | * Any routing table changes are constantly monitored by an another component (mwan3rtmon) which is responsible for keeping the main routing table in sync with the interface routing tables. | + | |
| - | ==== Overview of how routing with mwan3 works ==== | + | |
| The following steps are taken to route a packet with mwan3: | The following steps are taken to route a packet with mwan3: | ||
| Line 52: | Line 37: | ||
| Next up are the routing tables. These are really simple. There is just the standard main routing table and one routing table containing one gateway for each wan interface. Route table 1 for the first wan, route table 2 for the second and so on. Hopes this make troubleshooting easier. | Next up are the routing tables. These are really simple. There is just the standard main routing table and one routing table containing one gateway for each wan interface. Route table 1 for the first wan, route table 2 for the second and so on. Hopes this make troubleshooting easier. | ||
| - | ==== IPv6 support ==== | + | === Why should I use mwan3? |
| + | * If you have multiple internet connections and you want to control what traffic goes through which specific WAN interface. | ||
| + | * Mwan3 can handle multiple levels of primary and backup interfaces, load-balanced or not. Different sources can have different primary or backup WANs. | ||
| + | * Mwan3 uses netfilter mark mask to be compatible with other packages (such as OpenVPN, PPTP VPN, QoS-script, Tunnels, etc) as you can configure traffic to use the default routing table. | ||
| + | * Mwan3 can also load-balance traffic originating from the router itself | ||
| - | <WRAP center info 100%> | + | === How mwan3 load-balancing works === |
| - | Using mwan3 with IPv6 requires additional configuration such as IPv6 masquerading through methods like NETMAP or NAT6. This is currently | + | |
| - | </ | + | * Linux outgoing network traffic load-balancing is performed on a per-IP connection basis -- it is not channel-bonding, |
| + | * As such load-balancing will help speed multiple separate downloads or traffic generated from a group of source PCs all accessing different sites but it will not speed up a single download from one PC (unless the download is spread across multiple IP streams such as by using a download manager) | ||
| - | mwan3 does support IPv6 interfaces, but there are few guidelines you need to follow | + | === Architecture of mwan3 === |
| + | * mwan3 is triggered by [[docs: | ||
| + | * Once all the routes and rules are initially set up, mwan3 exits. The kernel takes care of all the routing decisions. If a new interface hotplug event occurs, mwan3 will run again to adjust route and tables as needed. | ||
| + | * A monitoring script (mwan3track) runs in the background checking if each WAN interface is up using a connectivity test (default is ping). If an interface goes down, the script issues a hotplug event to cause mwan3 to adjust the routing tables of the interface that has gone down. | ||
| + | * Any routing table changes are constantly monitored by an another component (mwan3rtmon) which is responsible for keeping the main routing table in sync with the interface routing tables. | ||
| - | * Where possible you should use the latest version of mwan3 available from the package repository for your router. Continuous improvements are being made regularly to mwan3 in relation to IPv6 support and other areas. | + | ===== Prerequisites ===== |
| - | * If your IPv6 is using a tunnel type connection such as 6in4 or Wireguard, you are strongly advised to be using OpenWrt 19.07.5 and above or a recent snapshot build which has important kernel patches that fix issues with ping which affects the mwan3track component. | + | |
| - | * You will need to split your network interfaces if they are dual-stack. i.e. wan and wan6, wanb and wanb6 etc. You can create alias interfaces to achieve this requirement. You would then define each interface in mwan3 with the address family of either '' | + | |
| - | * You will likely need to implement some form of IPv6 masquerading such as NETMAP or [[docs: | + | |
| - | NAT6 is potentially a controversial suggestion for many, but realistically without BGP, it is often required for IPv6 multihoming. Currently | + | Ensure no other multiple WAN or policy routing packages are installed such as '' |
| - | The [[# | + | ==== OpenWrt version ==== |
| - | ===Preventing IPv6 rules=== | + | === 23.05 === |
| - | You can prevent | + | The mwan3 package is mostly unchanged between 22.03 and 23.05, with some additional fixes but otherwise mostly |
| - | ===== Prerequisites ===== | + | **Known issues:** |
| - | Ensure no other multiple WAN or policy routing | + | * [[https:// |
| - | ==== OpenWrt version ==== | + | |
| + | === 22.03 === | ||
| + | |||
| + | 22.03 switched to firewall4/ | ||
| + | |||
| + | **Known issues:** | ||
| + | |||
| + | There are a few regressions between 2.10 and 2.11 identified with sticky rules and ipset. | ||
| + | An issue with fwmark and tunnel connections can cause traffic to be incorrectly routed e.g. L2TP, 6in4 and IPv6 traffic within | ||
| + | |||
| + | * [[https:// | ||
| + | * < | ||
| + | * < | ||
| + | * [[https:// | ||
| + | |||
| + | === 21.02 === | ||
| + | |||
| + | **No longer supported.** | ||
| + | |||
| + | The version of mwan3 in 21.02 is 2.10.13-1, it has a lot of improvements over the version in 19.07 for both performance and stability. | ||
| + | |||
| + | For those running some form of tunnel based protocol e.g. L2TP, 6in4 and IPv6 traffic within the tunnel may encounter routing issues due to fwmark behaviour that unintentionally marks all incoming traffic which can break routing in many cases. | ||
| - | Using the latest [[releases:19.07: | + | **Known issues:** |
| - | === Why it is not recommended to use snapshot builds currently? === | + | * [[https:// |
| - | Due to on-going development with mwan3 to be made compatible with the next major OpenWrt version, the general advisory is to avoid using snapshot builds currently if stable | + | Older versions beyond |
| - | If you want to test the latest | + | You can find the current open issues for mwan3 on the [[https://github.com/openwrt/ |
| - | You can find the current open issues for mwan3 on the [[https:// | ||
| ==== Hardware requirements ==== | ==== Hardware requirements ==== | ||
| Line 92: | Line 103: | ||
| Check the [[: | Check the [[: | ||
| - | ==== Single WAN Port ==== | + | === Single WAN Port === |
| If having LAN ports repurposed as WAN ports is not possible, it is also possible create virtual eths with **kmod-macvlan**. | If having LAN ports repurposed as WAN ports is not possible, it is also possible create virtual eths with **kmod-macvlan**. | ||
| Line 114: | Line 125: | ||
| option ifname ' | option ifname ' | ||
| </ | </ | ||
| + | |||
| + | ===== Installation ===== | ||
| + | ==== Command line (SSH) ==== | ||
| + | |||
| + | <code bash> | ||
| + | opkg update | ||
| + | opkg install mwan3 | ||
| + | opkg install luci-app-mwan3 | ||
| + | </ | ||
| + | |||
| + | '' | ||
| + | |||
| + | <WRAP center info 100%> | ||
| + | For routers using 22.03 or above the default firewall uses firewall4/ | ||
| + | </ | ||
| + | |||
| + | **For 22.03 or later:** | ||
| + | |||
| + | <code bash> | ||
| + | opkg install iptables-nft | ||
| + | opkg install ip6tables-nft | ||
| + | </ | ||
| + | |||
| + | ==== Web interface (LuCI) === | ||
| + | * Go to System -> Software | ||
| + | * click " | ||
| + | * In the " | ||
| + | * For 22.03: Install the '' | ||
| + | |||
| + | === Restart LuCI or reboot if needed === | ||
| + | |||
| + | To ensure the new menu item for mwan3 appears, logout of your existing session and restart the service hosting the LuCI interface i.e. uhttpd or just reboot the router. | ||
| + | |||
| + | * Go to System > Startup | ||
| + | * click the " | ||
| + | * Login into the web interface again. | ||
| + | |||
| + | A new menu entry " | ||
| + | |||
| + | ==== Upgrading ==== | ||
| + | |||
| + | If there is a newer version of mwan3 available, you can upgrade mwan3 through either opkg or LuCI. | ||
| + | |||
| + | < | ||
| + | opkg upgrade mwan3 | ||
| + | </ | ||
| + | |||
| + | Or through LuCI: **System** -> **Software** -> **Updates** | ||
| + | |||
| + | Your existing configuration will not be modified and instead if there any changes from the default, these will be able to be viewed in a '' | ||
| + | |||
| + | ==== IPv6 support ==== | ||
| + | |||
| + | <WRAP center info 100%> | ||
| + | Using mwan3 with load balancing or failover routing policies for IPv6 requires additional configuration such as NETMAP, NPTv6 or NAT66. None of these methods are currently implemented in mwan3 directly and hence requires additional configuration. | ||
| + | </ | ||
| + | |||
| + | **Using IPv6 with mwan3:** | ||
| + | |||
| + | - Newer versions of mwan3 have better IPv6 support, ensure you are running a supported OpenWrt version, as various IPv6 related areas have been addressed in recent versions. | ||
| + | - You will need to split your WAN network interfaces, so one interface has your IPv4 WAN and another for the IPv6 WAN. A common example convention is wan and wan6 (default with OpenWrt), along with an additional WAN interfaces such as wanb and wanb6 etc. Your IPv6 interface can be an alias interface in most cases. You then define each interface in mwan3 with the address family of either '' | ||
| + | - You will likely need to implement some form of IPv6 masquerading such as NETMAP or NPTv6 or [[docs: | ||
| + | |||
| + | NETMAP, NPTv6 and NAT66 all are configuration options that can work with mwan3, but it is up to you to implement the IPv6 configuration required. mwan3 does not currently implement any IPv6 masquerading by itself. | ||
| + | |||
| + | The [[# | ||
| + | |||
| + | === Disable mwan3 from routing IPv6 traffic === | ||
| + | |||
| + | You can prevent mwan3 from routing IPv6 traffic by declaring '' | ||
| + | |||
| ===== Pre-configuration ===== | ===== Pre-configuration ===== | ||
| - | You will need a minimum of two WAN interfaces for mwan3 to work effectively. While mwan3 is primarily designed for physical WAN connections it can also be used with logical interfaces like OpenVPN or Wireguard. | + | You will need a minimum of two WAN interfaces for mwan3 to work effectively. While mwan3 is primarily designed for physical |
| ==== Creating additional WAN interfaces ==== | ==== Creating additional WAN interfaces ==== | ||
| - | The simplest way to create more WAN interfaces is to have a VLAN-capable router. This allows | + | The simplest way to create more WAN interfaces is to have a VLAN-capable router. This will allow you to convert |
| - | Here is the general procedure using LuCI to create a new VLAN and assign a single port to it so as to create a second WAN interface. | + | Here is the general procedure using LuCI to create a new VLAN and assign a single port to it in order to create a second WAN interface. |
| - | * Go to Network > Switch | + | === Routers using swconfig === |
| - | | + | |
| - | | + | - Go to **Network > Switch** |
| - | | + | - Remove a single physical port from the default VLAN 1; this port will be the new physical WANB port |
| - | | + | |
| - | | + | |
| - | | + | |
| - | | + | |
| + | | ||
| + | | ||
| For routers that have more than one CPU, make sure to only tag one of the CPUs for any new VLAN created. One methodology for dual-CPU routers is that CPU1 will often be assigned to the built in WAN port, and you can tag CPU0 for any VLANs you wish to create. | For routers that have more than one CPU, make sure to only tag one of the CPUs for any new VLAN created. One methodology for dual-CPU routers is that CPU1 will often be assigned to the built in WAN port, and you can tag CPU0 for any VLANs you wish to create. | ||
| - | Create as many additional WAN interfaces as desired if you have more than two WAN connections. More information on [[https://wiki.openwrt.org/ | + | === Routers using Distributed Switch Architecture (DSA) === |
| + | |||
| + | From 21.02 onwards most targets will use [[docs:techref: | ||
| + | |||
| + | - Go to **Network > Interfaces** and select the Devices tab. Click configure on the br-lan device. | ||
| + | - Remove a lan port from the switch bridge ports option by selecting the menu and unchecking a switch port such as " | ||
| + | - Apply these changes to remove the selected LAN port from the LAN bridge. | ||
| + | - While still on the Devices page, scroll down and click the "Add device configuration" | ||
| + | - For device type select "VLAN (802.1q)" | ||
| + | - For base device select the lan port e.g. lan1 which was removed from the LAN bridge earlier. | ||
| + | - Assign the desired VLAN ID for this device. | ||
| + | - Save the changes and apply. | ||
| + | - Go to **Network > Interfaces** and "Add new interface" | ||
| + | - Give the interface a name such as " | ||
| + | - Select whatever protocol is required for this interface DHCP, PPPoE etc. For device select the lan port you removed from br-lan earlier. | ||
| + | - Assign the new interface to the wan firewall zone | ||
| + | - Apply any remaining changes. | ||
| - | **Note for PPPoE WAN interfaces: | + | **Note for PPPoE WAN interfaces: |
| - | ==== The routable loopback (self) interface | + | === The routable loopback (self) interface === |
| <WRAP center info 100%> | <WRAP center info 100%> | ||
| - | If you are using a newer release branch build of OpenWrt after 18.06, this step is **not necessary anymore**. Router initiated traffic | + | **If you are using 19.07 or newer this part is not required**. Router initiated traffic |
| A new service [[https:// | A new service [[https:// | ||
| </ | </ | ||
| Line 187: | Line 287: | ||
| Extra advantage is that configuring mwan3 rules for router only traffic is much easier. | Extra advantage is that configuring mwan3 rules for router only traffic is much easier. | ||
| - | ==== Prepare and the check the default routing table for the WAN interfaces ==== | + | ==== Prepare and verify |
| - | <WRAP left important | + | <WRAP left info 100%> |
| Before doing anything with mwan3 (installing or configuring), | Before doing anything with mwan3 (installing or configuring), | ||
| </ | </ | ||
| Line 195: | Line 295: | ||
| === Configure a different metric for each WAN interface === | === Configure a different metric for each WAN interface === | ||
| - | This is an important | + | <WRAP left important 100%> |
| + | Ensure that every WAN interface has a gateway IP and metric defined! | ||
| + | </ | ||
| * You must configure each WAN interface with a **different** routing metric. This metric will only have an effect on the default routing table, not on the mwan3 routing tables. | * You must configure each WAN interface with a **different** routing metric. This metric will only have an effect on the default routing table, not on the mwan3 routing tables. | ||
| Line 234: | Line 336: | ||
| default via 10.0.4.2 dev eth2 proto static | default via 10.0.4.2 dev eth2 proto static | ||
| </ | </ | ||
| - | |||
| - | <WRAP left important 100%> | ||
| - | Ensure that every WAN interface has a gateway IP and metric defined! This is very important as otherwise mwan3 will likely not work! | ||
| - | </ | ||
| ==== Verify outbound traffic on each WAN interface ==== | ==== Verify outbound traffic on each WAN interface ==== | ||
| Line 305: | Line 403: | ||
| This is the IP of // | This is the IP of // | ||
| - | |||
| - | |||
| - | ===== Installation ===== | ||
| - | === On the command line (SSH)=== | ||
| - | |||
| - | <code bash> | ||
| - | opkg update | ||
| - | opkg install mwan3 luci-app-mwan3 | ||
| - | </ | ||
| - | |||
| - | '' | ||
| - | |||
| - | === On the web interface (LuCI) === | ||
| - | * Go to System -> Software | ||
| - | * click " | ||
| - | * In the " | ||
| - | |||
| - | ==== Restart LuCI or reboot if needed ==== | ||
| - | |||
| - | To ensure the new menu item for mwan3 appears, logout of your existing session and restart the service hosting the LuCI interface i.e. uhttpd or just reboot the router. | ||
| - | |||
| - | * Go to System > Startup | ||
| - | * click the " | ||
| - | * Login into the web interface again. | ||
| - | |||
| - | A new menu entry " | ||
| - | |||
| - | ==== Upgrading ==== | ||
| - | |||
| - | If there is a newer version of mwan3 available, you can upgrade mwan3 through either opkg or LuCI. | ||
| - | |||
| - | Your existing configuration will not be modified and instead if there any changes from the default, these will be able to be viewed in a '' | ||
| ===== mwan3 configuration ===== | ===== mwan3 configuration ===== | ||
| - | The mwan3 configuration consists of five section elements, namely: | + | The mwan3 configuration consists of five main sections: |
| - | + | ||
| - | * Globals | + | |
| - | * Interfaces | + | |
| - | * Members | + | |
| - | * Policies | + | |
| - | * Rules | + | |
| - | + | ||
| - | Essentially the configuration can be summarised to the following. Globals are settings that apply to all of mwan3, interfaces are configured in mwan3 to be tracked and relate to the interface names present in your network configuration. For routing rules, one or more members must be defined targeting a specific interface, which in turn are assigned to a policy and a policy is assigned to one or more rules. | + | |
| - | + | ||
| + | * Globals - Global settings that apply to mwan3 overall. | ||
| + | * Interfaces - Network interfaces to be used/ | ||
| + | * Members - For a network interface to be used in mwan3, it must be defined as a member, which can then be used in policies. | ||
| + | * Policies - How the traffic should be routed according to the metric value and weight set in the member configuration. This allows you to define configurations like load balancing/ | ||
| + | * Rules - Defining one or more specific routing rules according to the defined policy set. A variety of rules can be configured using source/ | ||
| ==== Globals configuration ===== | ==== Globals configuration ===== | ||
| Line 371: | Line 432: | ||
| | '' | | '' | ||
| | '' | | '' | ||
| - | | '' | + | | '' |
| - | | '' | + | | '' |
| | '' | | '' | ||
| | '' | | '' | ||
| Line 385: | Line 446: | ||
| | '' | | '' | ||
| | '' | | '' | ||
| - | | '' | + | | '' |
| | '' | | '' | ||
| | '' | | '' | ||
| Line 394: | Line 455: | ||
| In most cases the default values should work for most configurations. The primary reason to change the default settings is to shorten the time before an interface is failed-over (by reducing the ping interval and number of pings before the interface is down) or lengthen the time to avoid a false link failure report. Please note that if you change the timeout value on low bandwidth interfaces (e.g. 3G) or busy interfaces, that false positives of marking a WAN down can occur. A timeout value of less then 2 seconds is not recommended. | In most cases the default values should work for most configurations. The primary reason to change the default settings is to shorten the time before an interface is failed-over (by reducing the ping interval and number of pings before the interface is down) or lengthen the time to avoid a false link failure report. Please note that if you change the timeout value on low bandwidth interfaces (e.g. 3G) or busy interfaces, that false positives of marking a WAN down can occur. A timeout value of less then 2 seconds is not recommended. | ||
| - | A typical interface section looks like this, mostly using the default values of all options described above: | + | A typical interface section |
| <code bash> | <code bash> | ||
| config interface ' | config interface ' | ||
| option enabled ' | option enabled ' | ||
| - | list track_ip '8.8.4.4' | + | list track_ip '1.0.0.1' |
| - | list track_ip '8.8.8.8' | + | list track_ip '1.1.1.1' |
| list track_ip ' | list track_ip ' | ||
| list track_ip ' | list track_ip ' | ||
| Line 407: | Line 468: | ||
| === Reliable public IP addresses to ping === | === Reliable public IP addresses to ping === | ||
| - | Below are a collection of public IPv4 and IPv6 endpoints that accept ICMP and can be used with mwan3track for tracking the connection state of interfaces. These are [[wp> | + | |
| + | Below are a collection of public IPv4 and IPv6 endpoints that accept ICMP and can be used with mwan3track for tracking the connection state of interfaces | ||
| + | |||
| + | **Note:** Some public DNS services may not respond to ICMP requests or intermittently drop requests due to throttling or rate limiting. This has been seen with Google public DNS, but can occur with any provider depending on their policy. You may see mwan3track ping failures due to this behaviour. To avoid this scenario marking an interface as down, ensure you have multiple '' | ||
| ^ DNS service ^ IPv4 resolvers ^ IPv6 resolvers ^ | ^ DNS service ^ IPv4 resolvers ^ IPv6 resolvers ^ | ||
| Line 477: | Line 541: | ||
| config policy ' | config policy ' | ||
| list use_member ' | list use_member ' | ||
| - | list use_member 'wanb_m1_w2' | + | list use_member 'wanb_m1_w3' |
| list use_member ' | list use_member ' | ||
| - | list use_member 'wanb6_m1_w2' | + | list use_member 'wanb6_m1_w3' |
| option last_resort ' | option last_resort ' | ||
| </ | </ | ||
| Line 489: | Line 553: | ||
| ==== Rule configuration ==== | ==== Rule configuration ==== | ||
| A rule describes what traffic to match and what policy to assign for that traffic. | A rule describes what traffic to match and what policy to assign for that traffic. | ||
| - | |||
| - | <WRAP center info 100%> | ||
| - | When creating rules from LuCI, the family option is currently not available to configure in 19.07 and older builds. This will mean rules created through LuCI default to the family value of '' | ||
| - | </ | ||
| **Key points about rules:** | **Key points about rules:** | ||
| - | * Rules specify which traffic will use a particular policy | + | * Rules specify which traffic will use a particular policy. |
| - | * Rules are based on IP address, port or protocol | + | * Rules are based on IP address, port or protocol. |
| - | * Rules are matched from top to bottom | + | * Rules are matched from top to bottom. |
| - | * Rules below a matching rule are ignored | + | * Rules below a matching rule are ignored. |
| - | * Traffic not matching any rule is routed using the main routing table | + | * Traffic not matching any defined |
| - | * Traffic destined for known (other than default) networks is handled by the main routing table | + | * Traffic destined for known (other than default) networks is handled by the main routing table. |
| - | * Traffic matching a rule where all interfaces for that policy are down will be blackholed | + | * Traffic matching a rule where all interfaces for that policy are down will be blackholed. |
| - | * Rule names may contain characters A-Z, a-z, 0-9, _ and no spaces | + | * Rule names may contain characters A-Z, a-z, 0-9, _ and no spaces. |
| - | * Rules may not share the same name as configured interfaces, members or policies | + | * Rules may not share the same name as configured interfaces, members or policies. |
| ^ Name ^ Type ^ Required ^ Default ^ Description ^ | ^ Name ^ Type ^ Required ^ Default ^ Description ^ | ||
| Line 544: | Line 604: | ||
| For rules that require a large amount of destination IP addresses, it is recommended to use ipset as this more optimised to group large amounts of IP addresses, or CIDR ranges. | For rules that require a large amount of destination IP addresses, it is recommended to use ipset as this more optimised to group large amounts of IP addresses, or CIDR ranges. | ||
| - | === ipset and sticky | + | === Sticky |
| - | Stickiness | + | Sticky (or sticky sessions) can be enabled on a per-rule basis and lets you route a new session over the same WAN interface as the previous session, as long as the time between the new and the previous session is shorter then the specified |
| + | |||
| + | By default mwan3 treats all https traffic with a sticky rule. | ||
| <code bash> | <code bash> | ||
| - | config rule 'youtube' | + | config rule 'https' |
| option sticky ' | option sticky ' | ||
| - | option timeout '300' | + | option |
| + | option proto ' | ||
| + | option use_policy ' | ||
| + | </ | ||
| + | |||
| + | With sticky set to 1, this rule now uses sticky sessions. When a packet for a new session matches this rule, its source IP address and interface mark are stored in an ipmark. When a packet for a second new session from the same LAN host within the timeout | ||
| + | |||
| + | === ipset support === | ||
| + | |||
| + | <WRAP center important 100%> | ||
| + | ipset functionality is broken in 23.05 due to the '' | ||
| + | </ | ||
| + | |||
| + | ipset is designed to store multiple IP addresses in a single collection, while being performant and easier to maintain. Common usages of ipset include storing large amounts of IP addresses or ranges in a single set as well as conditional routing by domain. As routing ultimately works at the IP layer, being able to use ipset with domain based policies is useful for many websites or services which use multiple IP addresses or large Content Delivery Networks which means the IP address of that domain is constantly changing, individually adding these IP addresses would become unmanageable very quickly, ipset can help maintain this for you. | ||
| + | |||
| + | A set can be populated manually, by a DNS resolver (triggered by a DNS lookup), or your own script. Rules enabled with ipset option will check for the existence of the destination address in the ipset chain defined in the rule to determine what routing needs to take place. If the destination address is found, the packet will be routed according to the policy, otherwise the ipset policy will not apply. | ||
| + | |||
| + | <code bash> | ||
| + | config rule 'youtube' | ||
| option ipset ' | option ipset ' | ||
| + | option sticky ' | ||
| option dest_port ' | option dest_port ' | ||
| option proto ' | option proto ' | ||
| Line 558: | Line 639: | ||
| </ | </ | ||
| - | With sticky | + | **Tip:** ipset rules also support |
| - | **Stickiness is on a per rule basis. With this example, all traffic from LAN hosts will use the same WAN interface | + | The example creates an ipset rule for a collection called youtube, with an additional |
| - | The option | + | For having |
| - | If the ipset chain does not already exist, mwan3 will create the ipset set for you. For this to work you need to configure a rule for dnsmasq in your ''/ | + | **dnsmasq: |
| + | |||
| + | **Note:** dnsmasq-full is required for ipset functionality. | ||
| <code bash> | <code bash> | ||
| Line 578: | Line 661: | ||
| </ | </ | ||
| - | You will then need to restart dnsmasq | + | Add more domains by separating each domain with a ''/'' |
| + | |||
| + | **AdGuard Home:** | ||
| + | |||
| + | Add to ''/ | ||
| + | |||
| + | <code yaml> | ||
| + | dns: | ||
| + | | ||
| + | - youtube.com/ | ||
| + | ... | ||
| + | </ | ||
| + | |||
| + | Add more domains by separating each domain with a '','' | ||
| + | |||
| + | Restart your DNS resolver and make a DNS lookup | ||
| + | |||
| + | <code bash> | ||
| + | ipset -L youtube | ||
| + | </ | ||
| + | |||
| + | If all is working correctly, you should see the resolved IP address or addresses in the ipset collection. | ||
| + | |||
| + | Be aware if the domain has been recently resolved by your DNS resolver, it may return a cache response which may not hit the ipset collection, clear the DNS cache and confirm your lookup is not a cached result. | ||
| ==== Default configuration example ==== | ==== Default configuration example ==== | ||
| - | This is an example configuration | + | This is a copy of the example configuration |
| <code bash / | <code bash / | ||
| Line 589: | Line 695: | ||
| config interface ' | config interface ' | ||
| option enabled ' | option enabled ' | ||
| - | list track_ip '8.8.4.4' | + | list track_ip '1.0.0.1' |
| - | list track_ip '8.8.8.8' | + | list track_ip '1.1.1.1' |
| list track_ip ' | list track_ip ' | ||
| list track_ip ' | list track_ip ' | ||
| - | option reliability ' | ||
| option family ' | option family ' | ||
| + | option reliability ' | ||
| config interface ' | config interface ' | ||
| option enabled ' | option enabled ' | ||
| - | list track_ip '2001:4860:4860::8844' | + | list track_ip '2606:4700:4700::1001' |
| - | list track_ip '2001:4860:4860::8888' | + | list track_ip '2606:4700:4700::1111' |
| list track_ip ' | list track_ip ' | ||
| list track_ip ' | list track_ip ' | ||
| + | option family ' | ||
| option reliability ' | option reliability ' | ||
| - | option family ' | ||
| config interface ' | config interface ' | ||
| option enabled ' | option enabled ' | ||
| - | list track_ip '8.8.4.4' | + | list track_ip '1.0.0.1' |
| - | list track_ip '8.8.8.8' | + | list track_ip '1.1.1.1' |
| list track_ip ' | list track_ip ' | ||
| list track_ip ' | list track_ip ' | ||
| + | option family ' | ||
| option reliability ' | option reliability ' | ||
| - | option family ' | ||
| config interface ' | config interface ' | ||
| option enabled ' | option enabled ' | ||
| - | list track_ip '2001:4860:4860::8844' | + | list track_ip '2606:4700:4700::1001' |
| - | list track_ip '2001:4860:4860::8888' | + | list track_ip '2606:4700:4700::1111' |
| list track_ip ' | list track_ip ' | ||
| list track_ip ' | list track_ip ' | ||
| - | option reliability ' | ||
| option family ' | option family ' | ||
| + | option reliability ' | ||
| config member ' | config member ' | ||
| Line 637: | Line 743: | ||
| option metric ' | option metric ' | ||
| option weight ' | option weight ' | ||
| + | |||
| + | config member ' | ||
| + | option interface ' | ||
| + | option metric ' | ||
| + | option weight ' | ||
| config member ' | config member ' | ||
| Line 657: | Line 768: | ||
| option metric ' | option metric ' | ||
| option weight ' | option weight ' | ||
| + | |||
| + | config member ' | ||
| + | option interface ' | ||
| + | option metric ' | ||
| + | option weight ' | ||
| config member ' | config member ' | ||
| Line 673: | Line 789: | ||
| config policy ' | config policy ' | ||
| list use_member ' | list use_member ' | ||
| - | list use_member 'wanb_m1_w2' | + | list use_member 'wanb_m1_w3' |
| list use_member ' | list use_member ' | ||
| - | list use_member 'wanb6_m1_w2' | + | list use_member 'wanb6_m1_w3' |
| config policy ' | config policy ' | ||
| Line 697: | Line 813: | ||
| config rule ' | config rule ' | ||
| option dest_ip ' | option dest_ip ' | ||
| - | option family ' | ||
| option use_policy ' | option use_policy ' | ||
| + | option family ' | ||
| config rule ' | config rule ' | ||
| option dest_ip '::/ | option dest_ip '::/ | ||
| - | option family ' | ||
| option use_policy ' | option use_policy ' | ||
| + | option family ' | ||
| </ | </ | ||
| - | ===== Verification of basic operation | + | ===== Testing/ |
| + | Once mwan3 has been configured and is enabled you will want to verify that mwan3 is working and correctly routing traffic according to your policies and rules. | ||
| - | ==== Check status | + | ==== Interface |
| - | * Network | + | |
| + | * Status | ||
| * Overview | * Overview | ||
| * MWAN3 Multi-WAN Interface Live Status | * MWAN3 Multi-WAN Interface Live Status | ||
| * this area should show all WAN interfaces as " | * this area should show all WAN interfaces as " | ||
| - | * MWAN3 Multi-WAN Interface | + | * MWAN3 Multi-WAN Interface |
| * this area will show recent mwan3 log messages | * this area will show recent mwan3 log messages | ||
| - | ==== Check kernel routing | + | **Note:** Older versions of mwan3 will use the label "Load Balancing" |
| - | * "ip route show table x" | + | |
| + | ==== Routing | ||
| + | |||
| + | * '' | ||
| + | |||
| + | ==== Verification of WAN interface load balancing ==== | ||
| - | ===== Verification of WAN interface load-balancing ===== | ||
| * Go to Network > Interfaces | * Go to Network > Interfaces | ||
| * Send traffic from a test inside PC | * Send traffic from a test inside PC | ||
| Line 727: | Line 849: | ||
| * Verify that traffic is going out all expected WAN interfaces | * Verify that traffic is going out all expected WAN interfaces | ||
| - | ===== Verification of WAN interface failover ===== | + | ==== Verification of WAN interface failover ===== |
| - | ==== Test interface failover ==== | + | |
| * Go to Network > Load Balancing > Overview | * Go to Network > Load Balancing > Overview | ||
| * Manually disconnect a WAN connection | * Manually disconnect a WAN connection | ||
| * Wait for interface failure detection to happen -- the mwan3 status display should update | * Wait for interface failure detection to happen -- the mwan3 status display should update | ||
| + | |||
| * Go to Network > Interfaces | * Go to Network > Interfaces | ||
| * Send traffic from a test inside PC and observe the interface packet counts to ensure traffic is now going out the alternate WAN port (counters are updated automatically) | * Send traffic from a test inside PC and observe the interface packet counts to ensure traffic is now going out the alternate WAN port (counters are updated automatically) | ||
| * Check that the external IP address has changed to the wanb interface (such as by going to [[http:// | * Check that the external IP address has changed to the wanb interface (such as by going to [[http:// | ||
| - | ==== Test interface | + | === Test WAN interface |
| * Restore the primary WAN connection | * Restore the primary WAN connection | ||
| * Wait for detection that the WAN link is back up | * Wait for detection that the WAN link is back up | ||
| Line 767: | Line 891: | ||
| **Changes in version 2.10.0:** | **Changes in version 2.10.0:** | ||
| - | '' | + | '' |
| + | |||
| + | <code bash> | ||
| + | mwan3 use < | ||
| + | </ | ||
| + | |||
| + | **Ping using the primary WAN interface: | ||
| + | |||
| + | <code bash> | ||
| + | mwan3 use wan ping -4 google.co.uk | ||
| + | </ | ||
| + | |||
| + | **iperf3 using the secondary WAN interface: | ||
| + | |||
| + | <code bash> | ||
| + | mwan3 use wanb iperf3 -4 -c speed.nimag.net -R | ||
| + | </ | ||
| **Changes in version 2.8.11:** | **Changes in version 2.8.11:** | ||
| Line 844: | Line 985: | ||
| '' | '' | ||
| - | * Status > Load Balancing | + | * Status > MultiWAN Manager |
| - | * Network > Load Balancing | + | * Network > MultiWAN Manager |
| - | The status section | + | In previous versions of '' |
| - | The network section allows for editing the mwan3 configuration through LuCI, being able change any part of the config file. | + | * The status section is designed to show the same information from mwan3 using CLI directly in LuCI with diagnostics and troubleshooting information. |
| + | * The network section allows for editing the mwan3 configuration through LuCI, being able change any part of the config file. | ||
| - | **Note:** The '' | + | **Note:** The '' |
| Line 873: | Line 1015: | ||
| **Note:** '' | **Note:** '' | ||
| - | There are various use cases for the ''/ | + | There are various use cases for the ''/ |
| + | |||
| + | === Example 1: Target ifup event on the wan interface === | ||
| <code bash> | <code bash> | ||
| Line 880: | Line 1024: | ||
| fi | fi | ||
| </ | </ | ||
| + | |||
| + | === Example 2: Target any ifup and ifdown events excluding certain interfaces === | ||
| <code bash> | <code bash> | ||
| Line 886: | Line 1032: | ||
| if [ " | if [ " | ||
| # Exclude events for interfaces loopback and self | # Exclude events for interfaces loopback and self | ||
| - | | + | |
| -smtp mail.example.com -sub " | -smtp mail.example.com -sub " | ||
| -pass " | -pass " | ||
| Line 895: | Line 1041: | ||
| Some notes about the last example: | Some notes about the last example: | ||
| - | * The " | + | * The " |
| * mailsend with SSL support was chosen as mail client, for other options and SMTP clients: [[: | * mailsend with SSL support was chosen as mail client, for other options and SMTP clients: [[: | ||
| * Finally observe that the whole sleep/ | * Finally observe that the whole sleep/ | ||
| Line 1178: | Line 1324: | ||
| ==== iperf3 ==== | ==== iperf3 ==== | ||
| - | Testing WAN links using iperf3 with mwan3 enabled can be a little tricky due to the fact mwan3 routing rules will often override the desired behaviour. More recently [[https:// | + | Testing WAN links using iperf3 with mwan3 enabled can be a little tricky due to the fact mwan3 routing rules will often override the desired behaviour. More recently [[https:// |
| - | In order to be able to use iperf3 successfully with mwan3 enabled you have two options. | + | In order to be able to use iperf3 successfully with mwan3 enabled you have a few options |
| + | - Upgrade to iperf3 version 3.10 or later, this may not be available in all OpenWrt package repositories currently. | ||
| - Upgrade to mwan3 2.10.0 or above, which provides the '' | - Upgrade to mwan3 2.10.0 or above, which provides the '' | ||
| - | - Implement custom mwan3 rules in ''/ | + | - Implement custom mwan3 rules in ''/ |
| As an example you can have something like this: | As an example you can have something like this: | ||
| Line 1206: | Line 1353: | ||
| option use_policy ' | option use_policy ' | ||
| </ | </ | ||
| + | |||
| + | ==== nft2ipset init script ==== | ||
| + | |||
| + | Due to the default firewall (fw4) now being based on nftables (rather than iptables), the ipset functionality commonly used in conjunction with dnsmasq and mwan3 no longer works in 23.05 releases. This is due to mwan3 not being fully compatible with nftables and requiring iptables compatibility/ | ||
| + | |||
| + | You will need to use nfset with dnsmasq for ipset polices to be created, which mwan3 only supports at this time. mwan3 currently does not support nfset in rules directly, hence the need to create ipset policies. | ||
| + | |||
| + | For help with this init script, please message @Kishi on the forum thread and also thank them if you found this useful! | ||
| + | |||
| + | The script is [[https:// | ||
| + | |||
| + | **Installation instructions: | ||
| + | |||
| + | < | ||
| + | wget -O / | ||
| + | chmod +x / | ||
| + | service nft2ipset enable | ||
| + | service nft2ipset start | ||
| + | </ | ||
| + | |||
| + | **Usage:** | ||
| + | |||
| + | - Define the nftables sets in LuCI " | ||
| + | - Then add those sets to dnsmasq resolving under "DHCP & DNS -> IP Sets" (Note: add both the IPv4 and the IPv6 set to the IP set option of the element as necessary. (Multiple nftables sets are possible to be specified for each group). | ||
| + | - Finally add them to mwan3 rules. I use specific, separate rules for IPv4/IPv6, but IPv4+IPv6 works as well because it'll match the family if the ipset anyway (which is IPv4 by default if not explicitly defined under " | ||
| ===== Original creators/ | ===== Original creators/ | ||