Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Next revisionBoth sides next revision
docs:guide-user:network:wan:multiwan:mwan3 [2022/09/25 16:07] – [Default configuration example] jamesmacwhitedocs:guide-user:network:wan:multiwan:mwan3 [2024/05/17 08:30] – [nft2ipset init script] jamesmacwhite
Line 1: Line 1:
 ====== mwan3 (Multi WAN load balancing/failover) ====== ====== mwan3 (Multi WAN load balancing/failover) ======
  
-  * 22.03: Latest release: 2.11.1 +  * 23.05: Latest release: [[:packages:pkgdata:mwan3|2.11.8]] 
-  * 21.02: Latest release: [[:packages:pkgdata:mwan3|2.10.13-1]] +  * 22.03: Latest release: 2.11.7 
-  * 19.07: Latest release: 2.8.16-1 +  * 21.02Latest release: 2.10.13-1
-  * 18.06 or olderNo longer supported or maintained.+
  
 ==== About mwan3 ==== ==== About mwan3 ====
Line 58: Line 57:
  
 Ensure no other multiple WAN or policy routing packages are installed such as ''multiwan''. Having ''multiwan'' installed at the same time as mwan3 is known not to work and is an obsolete package. Equally make sure you aren't using an other package that makes use of the same firewall mask value mwan3 uses as this will cause conflicts. The firewall mask value used by mwan3 is able to be changed in the configuration to avoid this problem. Ensure no other multiple WAN or policy routing packages are installed such as ''multiwan''. Having ''multiwan'' installed at the same time as mwan3 is known not to work and is an obsolete package. Equally make sure you aren't using an other package that makes use of the same firewall mask value mwan3 uses as this will cause conflicts. The firewall mask value used by mwan3 is able to be changed in the configuration to avoid this problem.
 +
 ==== OpenWrt version ==== ==== OpenWrt version ====
  
-**Using the latest [[:#current_stable_seriesopenwrt_2102|stable branch build]] is recommended as this is actively supported and maintained.**+=== 23.05 ===
  
-The old stable 19.07 branch should also work but it has an older version of mwan3 which does not include some newer features and fixes. It was also discovered there were a few 4.14 kernel issues that affect certain network configurations which can cause problems for mwan3. This has been resolved by a specific [[commit>?p=openwrt/openwrt.git;a=commitdiff;h=c9c7b4b3945c01c2aadf3ef5d9a77c8200db80f1|kernel patch related to netfilter]]. These patches help fix issues with tunnel based network connections like 6in4, P2TP, PPPoE, Wireguard etc. This kernel patch is only present in 19.07.5 and aboveso it is recommended to use at least this version if using 19.07.+The mwan3 package is mostly unchanged between 22.03 and 23.05with some additional fixes but otherwise mostly the same.
  
-Older branches before 19.07 are no longer supported.+**Known issues:**
  
-You can find the current open issues for mwan3 on the [[https://github.com/openwrt/packages/issues?q=is%3Aissue+is%3Aopen+in%3Atitle+mwan3|OpenWrt packages repository]]. User feedback is welcome to help with identifying bugs and issues found with different network setupsFeatures requests or contributions are also welcome!+   [[https://github.com/openwrt/packages/issues/22474|mwan3: ipset functionality broken on 23.05]]. [[docs:guide-user:network:wan:multiwan:mwan3#nft2ipset_init_script|Workaround init script available]].
  
-==== IPv6 support ====+=== 22.03 ===
  
-<WRAP center info 100%> +22.03 switched to firewall4/nftables for firewall management, mwan3 has not been updated to natively support nftables yet and therefore needs the ''iptables-nft'' and ''ip6tables-nft'' packages installed for a iptables compatibility layer for firewall rules to work. [[docs:guide-user:network:wan:multiwan:mwan3#installation|See installation steps]] for more information.
-Using mwan3 with IPv6 requires additional configuration such as IPv6 masquerading through methods like NETMAP or NAT6This is currently not implemented in mwan3 directly and requires additional configuration. +
-</WRAP>+
  
-mwan3 does support IPv6 interfaces, but there are few guidelines you need to follow in order to configure IPv6 with mwan3.+**Known issues:**
  
-  - If your IPv6 is using tunnel type connection such as 6in4 or Wireguard, you are strongly advised to use OpenWrt 19.07.and above which has important kernel patches that fix issues with ping which affects the mwan3track component+There are few regressions between 2.10 and 2.11 identified with sticky rules and ipset
-  - You will need to split your network interfaces, so one interface has your IPv4 and another for IPv6i.e. wan and wan6wanb and wanb6 etc. You can create alias interfaces to achieve this requirement. You would then define each interface in mwan3 with the address family of either ''ipv4'' or ''ipv6'' and create a member profile for each to be used in policies assigned to your rules so IPv4 and IPv6 traffic is handled. mwan3 cannot currently handle IPv4 and IPv6 on a single interface. +An issue with fwmark and tunnel connections can cause traffic to be incorrectly routed e.gL2TP6in4 and IPv6 traffic within the tunnel is also present under certain configurations.
-  - You will likely need to implement some form of IPv6 masquerading such as NETMAP or [[docs:guide-user:network:ipv6:ipv6.nat6|NAT6]] for mwan3 to work properly across multiple WAN interfaces.+
  
-NAT6 is potentially a controversial suggestion for many, but realistically without BGP, it is often required for IPv6 multihomingCurrently mwan3 does not implement any IPv6 masquerading as part of it's configurationThis is something that needs to be configured outside of mwan3 itself.+  * [[https://github.com/openwrt/packages/issues/19472|mwan3: Legacy rules detected]] (See installation steps) 
 +  * <del>[[https://github.com/openwrt/packages/pull/20900|mwan3: fix addition of routes to mwan3_connected ipset]]</del> 
 +  * <del>[[https://github.com/openwrt/packages/pull/20901|mwan3: fix addition of iptables rules for mwan3 sticky rules]]</del> 
 +  * [[https://github.com/openwrt/packages/pull/20923|mwan3: fix some tunnels assigned the wrong mark]]
  
-The [[#default_configuration_example|default configuration that ships with mwan3]] provides an example configuration of having two WAN interfaces with dual-stack connectivity (note that the second example interface is not enabled by default). This is a good template to start with if you wish to explore routing IPv6 with mwan3.+=== 21.02 ===
  
-=== Disable mwan3 from routing IPv6 traffic ===+**No longer supported.**
  
-You can prevent mwan3 from routing IPv6 traffic by declaring ''option family 'ipv4%%'%%'' [[#rule_configuration|on all rules]] and removing the default IPv6 ruleThis will prevent any mwan3 IPv6 routing rules being created by mwan3. You should also add ''option last_resort 'default%%'%%'' on your policies to fall back to the main routing table to allow IPv6 traffic (if present)However, doing this means your IPv6 traffic cannot be balanced or fail over if not handled by mwan3.+The version of mwan3 in 21.02 is 2.10.13-1, it has a lot of improvements over the version in 19.07 for both performance and stability.  
 + 
 +For those running some form of tunnel based protocol e.g. L2TP, 6in4 and IPv6 traffic within the tunnel may encounter routing issues due to fwmark behaviour that unintentionally marks all incoming traffic which can break routing in many cases. 
 + 
 +**Known issues:** 
 + 
 +  * [[https://github.com/openwrt/packages/pull/20923|mwan3: fix some tunnels assigned the wrong mark]] 
 + 
 +Older versions beyond the old and current stable are no longer supported and unlikely to receive support. 
 + 
 +You can find the current open issues for mwan3 on the [[https://github.com/openwrt/packages/issues?q=is%3Aissue+is%3Aopen+in%3Atitle+mwan3|OpenWrt packages repository]]. User feedback is welcome to help with identifying bugs and issues found with different network setupsFeatures requests or contributions are also welcome!
  
 ==== Hardware requirements ==== ==== Hardware requirements ====
Line 115: Line 125:
     option ifname 'eth1'     option ifname 'eth1'
 </code> </code>
 +
 +===== Installation =====
 +==== Command line (SSH) ====
 +
 +<code bash>
 +opkg update
 +opkg install mwan3
 +opkg install luci-app-mwan3
 +</code>
 +
 +''luci-app-mwan3'' is optional, if you don't wish to manage rules through LuCI.
 +
 +<WRAP center info 100%>
 +For routers using 22.03 or above the default firewall uses firewall4/nftables, the packages ''iptables-nft'' and ''ip6tables-nft'' are needed for mwan3 functionality to work. mwan3 does not currently natively support nftables, but does function with the iptables compatibility backend which will translate rules to be compatible with nftables.
 +</WRAP>
 +
 +**For 22.03 or later:**
 +
 +<code bash>
 +opkg install iptables-nft
 +opkg install ip6tables-nft
 +</code>
 +
 +==== Web interface (LuCI) ===
 +  * Go to System -> Software
 +    * click "Update lists" to get the latest package databases
 +    * In the "Download and install package:" box, enter ''luci-app-mwan3'' and click OK to download and install the package, dependencies including mwan3 itself will be installed.
 +    * For 22.03: Install the ''iptables-nft'' and ''ip6tables-nft'' backend which is required for translating mwan3 rules to work with nftables.
 +
 +=== Restart LuCI or reboot if needed ===
 +
 +To ensure the new menu item for mwan3 appears, logout of your existing session and restart the service hosting the LuCI interface i.e. uhttpd or just reboot the router.
 +
 +  * Go to System > Startup
 +    * click the "Restart" button next to the process running LuCI i.e. uhttpd, nginx etc.
 +    * Login into the web interface again.
 +
 +A new menu entry "Network > MultiWAN Manager" should now be present. In older versions of ''luci-app-mwan3'' this will be labelled as "Load Balancing".
 +
 +==== Upgrading ====
 +
 +If there is a newer version of mwan3 available, you can upgrade mwan3 through either opkg or LuCI.
 +
 +<code>
 +opkg upgrade mwan3
 +</code>
 +
 +Or through LuCI: **System** -> **Software** -> **Updates**
 +
 +Your existing configuration will not be modified and instead if there any changes from the default, these will be able to be viewed in a ''mwan3-opkg'' file alongside your mwan3 configuration file in ''/etc/config''. Occasionally there may be changes to the configuration options so it is a good idea to inspect the default configuration on upgrades to ensure your configuration has the latest changes in various sections.
 +
 +==== IPv6 support ====
 +
 +<WRAP center info 100%>
 +Using mwan3 with load balancing or failover routing policies for IPv6 requires additional configuration such as NETMAP, NPTv6 or NAT66. None of these methods are currently implemented in mwan3 directly and hence requires additional configuration.
 +</WRAP>
 +
 +**Using IPv6 with mwan3:**
 +
 +  - Newer versions of mwan3 have better IPv6 support, ensure you are running a supported OpenWrt version, as various IPv6 related areas have been addressed in recent versions.
 +  - You will need to split your WAN network interfaces, so one interface has your IPv4 WAN and another for the IPv6 WAN. A common example convention is wan and wan6 (default with OpenWrt), along with an additional WAN interfaces such as wanb and wanb6 etc. Your IPv6 interface can be an alias interface in most cases. You then define each interface in mwan3 with the address family of either ''ipv4'' or ''ipv6'' and create a member profile for each to be used in policies assigned to your rules so IPv4 and IPv6 traffic is handled. mwan3 cannot currently handle IPv4 and IPv6 configuration on a single interface.
 +  - You will likely need to implement some form of IPv6 masquerading such as NETMAP or NPTv6 or [[docs:guide-user:network:ipv6:ipv6.nat6|NAT66]] for mwan3 and IPv6 traffic to work properly across multiple WAN interfaces.
 +
 +NETMAP, NPTv6 and NAT66 all are configuration options that can work with mwan3, but it is up to you to implement the IPv6 configuration required. mwan3 does not currently implement any IPv6 masquerading by itself.
 +
 +The [[#default_configuration_example|default configuration that ships with mwan3]] provides an example configuration of having two WAN interfaces with dual-stack connectivity (note that the second example interface is not enabled by default). This is a good template to start with if you wish to explore routing IPv6 with mwan3.
 +
 +=== Disable mwan3 from routing IPv6 traffic ===
 +
 +You can prevent mwan3 from routing IPv6 traffic by declaring ''option family 'ipv4%%'%%'' [[#rule_configuration|on all rules]] and removing the default IPv6 rule. This will prevent any mwan3 IPv6 routing rules being created by mwan3. You should also add ''option last_resort 'default%%'%%'' on your policies to fall back to the main routing table to allow IPv6 traffic (if present). However, doing this means your IPv6 traffic cannot be balanced or fail over if not handled by mwan3.
  
 ===== Pre-configuration ===== ===== Pre-configuration =====
Line 142: Line 222:
 === Routers using Distributed Switch Architecture (DSA) === === Routers using Distributed Switch Architecture (DSA) ===
  
-From 21.02 onwards some targets will use [[docs:techref:hardware:switch|DSA]] which is different and not compatible with the instructions for swconfig. You can find a [[:docs:guide-user:network:dsa:converting-to-dsa|converting to DSA guide]] for additional guidance for switch/VLAN management for router targets using DSA.+From 21.02 onwards most targets will use [[docs:techref:hardware:switch|DSA]] which is different and not compatible with the instructions for swconfig. You can find a [[:docs:guide-user:network:dsa:converting-to-dsa|converting to DSA guide]] for additional guidance for switch/VLAN management for router targets using DSA.
  
   - Go to **Network > Interfaces** and select the Devices tab. Click configure on the br-lan device.   - Go to **Network > Interfaces** and select the Devices tab. Click configure on the br-lan device.
Line 163: Line 243:
  
 <WRAP center info 100%> <WRAP center info 100%>
-If you are using a newer release branch build of OpenWrt after 18.06, this step is **not necessary anymore**. Router initiated traffic is also load-balanced and can fail-over correctly.+**If you are using 19.07 or newer this part is not required**. Router initiated traffic can also be load-balanced or use failover correctly.
 A new service [[https://github.com/openwrt/packages/commits/master/net/mwan3/files/usr/sbin/mwan3rtmon?author=ptpt52|mwan3rtmon]] was added by [[https://github.com/ptpt52|Chen Minqiang]]. The service is responsible for syncing the main routing table with the interface routing tables. Also as inbound traffic has no dedicated firewall tables anymore. This is now working out of the box without any workarounds needed. A new service [[https://github.com/openwrt/packages/commits/master/net/mwan3/files/usr/sbin/mwan3rtmon?author=ptpt52|mwan3rtmon]] was added by [[https://github.com/ptpt52|Chen Minqiang]]. The service is responsible for syncing the main routing table with the interface routing tables. Also as inbound traffic has no dedicated firewall tables anymore. This is now working out of the box without any workarounds needed.
 </WRAP> </WRAP>
Line 207: Line 287:
 Extra advantage is that configuring mwan3 rules for router only traffic is much easier. Extra advantage is that configuring mwan3 rules for router only traffic is much easier.
  
-==== Prepare and the check the default routing table for the WAN interfaces ====+==== Prepare and verify the default routing table for WAN interfaces ====
  
-<WRAP left important 100%>+<WRAP left info 100%>
 Before doing anything with mwan3 (installing or configuring), ensure that each WAN interface is working and that the default routing table is correctly configured for each WAN connection. Test each interface with a manual ping before installing mwan3! It is strongly recommended to do some pre-configuration and test your connectivity for each WAN interface prior to enabling mwan3, this will help with troubleshooting and ensure your WAN interfaces are correctly configured before using mwan3. Before doing anything with mwan3 (installing or configuring), ensure that each WAN interface is working and that the default routing table is correctly configured for each WAN connection. Test each interface with a manual ping before installing mwan3! It is strongly recommended to do some pre-configuration and test your connectivity for each WAN interface prior to enabling mwan3, this will help with troubleshooting and ensure your WAN interfaces are correctly configured before using mwan3.
 </WRAP> </WRAP>
Line 215: Line 295:
 === Configure a different metric for each WAN interface === === Configure a different metric for each WAN interface ===
  
-This is an important step and is compulsory. Time and time again fail to configure this and have a none working setup.+<WRAP left important 100%> 
 +Ensure that every WAN interface has a gateway IP and metric defined! This is very important as otherwise mwan3 will likely not work! 
 +</WRAP>
  
   * You must configure each WAN interface with a **different** routing metric. This metric will only have an effect on the default routing table, not on the mwan3 routing tables.   * You must configure each WAN interface with a **different** routing metric. This metric will only have an effect on the default routing table, not on the mwan3 routing tables.
Line 254: Line 336:
 default via 10.0.4.2 dev eth2  proto static  src 10.0.4.15  metric 20 default via 10.0.4.2 dev eth2  proto static  src 10.0.4.15  metric 20
 </code> </code>
- 
-<WRAP left important 100%> 
-Ensure that every WAN interface has a gateway IP and metric defined! This is very important as otherwise mwan3 will likely not work! 
-</WRAP> 
  
 ==== Verify outbound traffic on each WAN interface ==== ==== Verify outbound traffic on each WAN interface ====
Line 325: Line 403:
  
 This is the IP of //smtp.virginmedia.com//, you may need to add more IP addresses in order to cover IMAP, POP3 and other SMTP servers if used. You could also add use the ''proto'' and ''dest_port'' on rules to limit it to mail related ports. This is the IP of //smtp.virginmedia.com//, you may need to add more IP addresses in order to cover IMAP, POP3 and other SMTP servers if used. You could also add use the ''proto'' and ''dest_port'' on rules to limit it to mail related ports.
- 
- 
-===== Installation ===== 
-==== Command line (SSH) ==== 
- 
-<code bash> 
-opkg update 
-opkg install mwan3 
-opkg install luci-app-mwan3 
-</code> 
- 
-''luci-app-mwan3'' is optional, if you don't wish to manage rules through LuCI. 
- 
-==== Web interface (LuCI) === 
-  * Go to System -> Software 
-    * click "Update lists" to get the latest package databases 
-    * In the "Download and install package:" box, enter "luci-app-mwan3" and click OK to download and install the luci-app-mwan3 package and all related packages, including mwan3 itself and all dependencies. 
- 
-=== Restart LuCI or reboot if needed === 
- 
-To ensure the new menu item for mwan3 appears, logout of your existing session and restart the service hosting the LuCI interface i.e. uhttpd or just reboot the router. 
- 
-  * Go to System > Startup 
-    * click the "Restart" button next to the process running LuCI i.e. uhttpd, nginx etc. 
-    * Login into the web interface again. 
- 
-A new menu entry "Network > MultiWAN Manager" should now be present. In older versions of ''luci-app-mwan3'' this will be labelled as "Load Balancing". 
- 
-==== Upgrading ==== 
- 
-If there is a newer version of mwan3 available, you can upgrade mwan3 through either opkg or LuCI. 
- 
-<code> 
-opkg upgrade mwan3 
-</code> 
- 
-Or through LuCI: **System** -> **Software** -> **Updates** 
- 
-Your existing configuration will not be modified and instead if there any changes from the default, these will be able to be viewed in a ''mwan3-opkg'' file alongside your mwan3 configuration file in ''/etc/config''. Occasionally there may be changes to the configuration options so it is a good idea to inspect the default configuration on upgrades to ensure your configuration has the latest changes in various sections. 
  
 ===== mwan3 configuration ===== ===== mwan3 configuration =====
-The mwan3 configuration consists of five section elements, namely: +The mwan3 configuration consists of five main sections:
- +
-  * Globals +
-  * Interfaces +
-  * Members +
-  * Policies +
-  * Rules +
- +
-Essentially the configuration can be summarised to the following. Globals are settings that apply to all of mwan3, interfaces are configured in mwan3 to be tracked and relate to the interface names present in your network configuration. For routing rules, one or more members must be defined targeting a specific interface, which in turn are assigned to a policy and a policy is assigned to one or more rules. +
- +
  
 +  * Globals - Global settings that apply to mwan3 overall.
 +  * Interfaces - Network interfaces to be used/tracked by mwan3, the interfaces configured in mwan3 need to match what is set in ''/etc/config/network''.
 +  * Members - For a network interface to be used in mwan3, it must be defined as a member, which can then be used in policies.
 +  * Policies - How the traffic should be routed according to the metric value and weight set in the member configuration. This allows you to define configurations like load balancing/failover or forcing traffic through a specific WAN.
 +  * Rules - Defining one or more specific routing rules according to the defined policy set. A variety of rules can be configured using source/destination IP/port, domain names (using ipset) and more.
 ==== Globals configuration ===== ==== Globals configuration =====
  
Line 522: Line 556:
 **Key points about rules:** **Key points about rules:**
  
-  * Rules specify which traffic will use a particular policy +  * Rules specify which traffic will use a particular policy. 
-  * Rules are based on IP address, port or protocol +  * Rules are based on IP address, port or protocol. 
-  * Rules are matched from top to bottom +  * Rules are matched from top to bottom. 
-  * Rules below a matching rule are ignored +  * Rules below a matching rule are ignored. 
-  * Traffic not matching any rule is routed using the main routing table +  * Traffic not matching any defined rule will be routed using the main routing table. 
-  * Traffic destined for known (other than default) networks is handled by the main routing table +  * Traffic destined for known (other than default) networks is handled by the main routing table. 
-  * Traffic matching a rule where all interfaces for that policy are down will be blackholed +  * Traffic matching a rule where all interfaces for that policy are down will be blackholed. 
-  * Rule names may contain characters A-Z, a-z, 0-9, _ and no spaces +  * Rule names may contain characters A-Z, a-z, 0-9, _ and no spaces. 
-  * Rules may not share the same name as configured interfaces, members or policies+  * Rules may not share the same name as configured interfaces, members or policies.
  
 ^ Name ^ Type ^ Required ^ Default ^ Description ^ ^ Name ^ Type ^ Required ^ Default ^ Description ^
Line 570: Line 604:
 For rules that require a large amount of destination IP addresses, it is recommended to use ipset as this more optimised to group large amounts of IP addresses, or CIDR ranges. For rules that require a large amount of destination IP addresses, it is recommended to use ipset as this more optimised to group large amounts of IP addresses, or CIDR ranges.
  
-=== ipset and sticky support ===+=== Sticky support ===
  
-Stickiness lets you route a new session over the same WAN interface as the previous session, as long as the time between the new and the previous session is shorter then the timeout value (default 600 seconds). This can solve some problems with HTTPS siteswhich don't allow a new source address within the same cookie/HTTPS session. Using ipset lets you route traffic over WAN interfaces based on set of IP addresses. A set can be created by hand, by dnsmasq based on domain names, or your own script. Rules with ipset option will try to match destination IP address to the configured ipset.+Sticky (or sticky sessions) can be enabled on a per-rule basis and lets you route a new session over the same WAN interface as the previous session, as long as the time between the new and the previous session is shorter then the specified timeout value. This is mainly useful for load balanced routing and can solve some problems with HTTPS sites which don't allow a new source address within the same cookie/HTTPS session. 
 + 
 +By default mwan3 treats all https traffic with a sticky rule.
  
 <code bash> <code bash>
-config rule 'youtube'+config rule 'https'
     option sticky '1'     option sticky '1'
-    option timeout '300'+    option dest_port '443' 
 +    option proto 'tcp' 
 +    option use_policy 'balanced' 
 +</code> 
 + 
 +With sticky set to 1, this rule now uses sticky sessions. When a packet for a new session matches this rule, its source IP address and interface mark are stored in an ipmark. When a packet for a second new session from the same LAN host within the timeout period matches this rule, it will use the same WAN interface as the first packet and the timeout counter is reset back to specified timeout value. The default timeout value is 600 seconds. 
 + 
 +=== ipset support === 
 + 
 +<WRAP center important 100%> 
 +ipset functionality is broken in 23.05 due to the ''dnsmasq-full'' package no longer being compiled with ipset support in favour of nftables. As mwan3 does not currently support nftables natively, this functionality no longer works. [[https://forum.openwrt.org/t/23-05-dnsmasq-ipsets-and-mwan3-incompatibility/174926|More information and further discussion]]. A [[docs:guide-user:network:wan:multiwan:mwan3#nft2ipset init script|workaround init script that converts nfset to ipset is available]] to use until mwan3 is updated to natively support nfset. 
 +</WRAP> 
 + 
 +ipset is designed to store multiple IP addresses in a single collection, while being performant and easier to maintain. Common usages of ipset include storing large amounts of IP addresses or ranges in a single set as well as conditional routing by domain. As routing ultimately works at the IP layer, being able to use ipset with domain based policies is useful for many websites or services which use multiple IP addresses or large Content Delivery Networks which means the IP address of that domain is constantly changing, individually adding these IP addresses would become unmanageable very quickly, ipset can help maintain this for you. 
 + 
 +A set can be populated manually, by a DNS resolver (triggered by a DNS lookup), or your own script. Rules enabled with ipset option will check for the existence of the destination address in the ipset chain defined in the rule to determine what routing needs to take place. If the destination address is found, the packet will be routed according to the policy, otherwise the ipset policy will not apply. 
 + 
 +<code bash> 
 +config rule 'youtube'
     option ipset 'youtube'     option ipset 'youtube'
 +    option sticky '1'
     option dest_port '80,443'     option dest_port '80,443'
     option proto 'tcp'     option proto 'tcp'
Line 584: Line 639:
 </code> </code>
  
-With sticky set to 1, this rule has now sticky enabled. When a packet for a new session matches this rule, its source IP address and interface mark are stored in an ipmark set with a timeout of 300 seconds. When a packet for a second new session from the same LAN host within the timeout period matches this rule, it will use the same WAN interface as the first packet and the timeout counter is reset back to 300 again.+**Tip:** ipset rules also support sticky sessions.
  
-**Stickiness is on per rule basisWith this exampleall traffic from LAN hosts will use the same WAN interface for all hosts in the ipseteven if the source or destination IP address differs.**+The example creates an ipset rule for collection called youtube, with an additional rule of only matching destination ports TCP 80/443 i.e. HTTP/HTTPS. If the ipset chain does not already existmwan3 will create the ipset set for you. However to ensure all network conditions are metyou should ensure ipset collections are created on router startup.
  
-The option ipset matches only destination IP addresses. This example will only work if your LAN clients use the dnsmasq server as their one and only DNS server or have your configured existing upstream DNS resolvers use the dnsmasq server as their forwarder.+For having ipset collections automatically populated on DNS lookups matching the domain required, you will need to add an ipset configuration to your DNS resolver, two common DNS resolvers dnsmasq (default in OpenWrt) or Adguard Home.
  
-If the ipset chain does not already exist, mwan3 will create the ipset set for you. For this to work you need to configure a rule for dnsmasq in your ''/etc/config/dhcp''.+**dnsmasq:** 
 + 
 +**Note:** dnsmasq-full is required for ipset functionality.
  
 <code bash> <code bash>
Line 604: Line 661:
 </code> </code>
  
-You will then need to restart dnsmasq for the ipset change to be applied.+Add more domains by separating each domain with a ''/'' character. 
 + 
 +**AdGuard Home:** 
 + 
 +Add to ''/etc/adguardhome.yaml''
 + 
 +<code yaml> 
 +dns: 
 + ipset: 
 + - youtube.com/youtube 
 +... 
 +</code> 
 + 
 +Add more domains by separating each domain with a '','' character. 
 + 
 +Restart your DNS resolver and make a DNS lookup for the domain in the ipset. To check the contents of an ipset collection you can run the command: 
 + 
 +<code bash> 
 +ipset -L youtube 
 +</code> 
 + 
 +If all is working correctly, you should see the resolved IP address or addresses in the ipset collection. 
 + 
 +Be aware if the domain has been recently resolved by your DNS resolver, it may return a cache response which may not hit the ipset collection, clear the DNS cache and confirm your lookup is not a cached result.
 ==== Default configuration example ==== ==== Default configuration example ====
  
Line 746: Line 826:
 Once mwan3 has been configured and is enabled you will want to verify that mwan3 is working and correctly routing traffic according to your policies and rules. Once mwan3 has been configured and is enabled you will want to verify that mwan3 is working and correctly routing traffic according to your policies and rules.
  
-==== Check status in the MWAN3 overview page ====+==== Interface status ====
  
   * Network > MultiWAN Manager   * Network > MultiWAN Manager
Line 757: Line 837:
 **Note:** Older versions of mwan3 will use the label "Load Balancing" in LuCI. **Note:** Older versions of mwan3 will use the label "Load Balancing" in LuCI.
  
-==== Check kernel routing tables ====+==== Routing tables ====
  
-  * "ip route show table x(where x is interface ID) should show a routing table specifically for that interface -- these tables are generated by mwan3+  * ''ip route show table x'' (where x is interface ID) should show a routing table specifically for that interface -- these tables are generated by mwan3.
  
-==== Verification of WAN interface (load-balancing====+==== Verification of WAN interface load balancing ====
  
   * Go to Network > Interfaces   * Go to Network > Interfaces
Line 769: Line 849:
     * Verify that traffic is going out all expected WAN interfaces     * Verify that traffic is going out all expected WAN interfaces
  
-==== Verification of WAN interface (failover=====+==== Verification of WAN interface failover =====
  
   * Go to Network > Load Balancing > Overview   * Go to Network > Load Balancing > Overview
Line 779: Line 859:
     * Check that the external IP address has changed to the wanb interface (such as by going to [[http://whatismyip.com]])     * Check that the external IP address has changed to the wanb interface (such as by going to [[http://whatismyip.com]])
  
-=== Test interface fail-back ===+=== Test WAN interface recovery ===
  
   * Restore the primary WAN connection   * Restore the primary WAN connection
Line 811: Line 891:
 **Changes in version 2.10.0:** **Changes in version 2.10.0:**
  
-''mwan3 use'' was added in version 2.10. This additional option is designed to allow you test network commands like ''ping'', ''iperf3'' etc by binding the command to a specific interface. A known issue with mwan3 is it can skew the output of commands that rely on binding to specific interfaces, as traffic will be routed according to the rules defined in ''/etc/config/mwan3'' and essentially bypass itAn example if you wanted to run an ''iperf'' test on a secondary WAN, your mwan3 policy will most likely override this and instead test your primary WAN which is not desirable for this purpose.+''mwan3 use'' was added in version 2.10. This additional option is designed to allow you test network commands like ''ping'', ''iperf3'' etc by binding the command to a specific interface reliably. A common side effect with mwan3 is it can skew the output of commands that rely on binding to specific interfaces, as traffic will be routed according to the rules defined in ''/etc/config/mwan3'' and essentially override the desired scenario in some cases. 
 + 
 +<code bash> 
 +mwan3 use <IFACE> <COMMAND> 
 +</code> 
 + 
 +**Ping using the primary WAN interface:** 
 + 
 +<code bash> 
 +mwan3 use wan ping -4 google.co.uk 
 +</code> 
 + 
 +**iperf3 using the secondary WAN interface:** 
 + 
 +<code bash> 
 +mwan3 use wanb iperf3 -4 -c speed.nimag.net -R 
 +</code> 
  
 **Changes in version 2.8.11:** **Changes in version 2.8.11:**
Line 1255: Line 1352:
  option family 'ipv4'  option family 'ipv4'
  option use_policy 'wanb_only'  option use_policy 'wanb_only'
 +</code>
 +
 +==== nft2ipset init script ====
 +
 +Due to the default firewall (fw4) now being based on nftables (rather than iptables), the ipset functionality commonly used in conjunction with dnsmasq and mwan3 no longer works in 23.05 releases. This is due to mwan3 not being fully compatible with nftables and requiring iptables compatibility/translation packages (see installation steps). While ipset functionality works in 23.02 without any changes, since the 23.05 release an important dnsmasq compile flag was changed to remove all ipset support in favour of nfset. To restore near like for like functionality a custom init script can be used, [[https://forum.openwrt.org/t/23-05-dnsmasq-ipsets-and-mwan3-incompatibility/174926/40|credit @Kishi on the OpenWrt community forum]]. This script monitors changes to nftables/nfset and creates or updates ipset equivalents, essentially replicating the behaviour of what dnsmasq would do with ipset support enabled.
 +
 +You will need to use nfset with dnsmasq for ipset polices to be created, which mwan3 only supports at this time. mwan3 currently does not support nfset in rules directly, hence the need to create ipset policies.
 +
 +For help with this init script, please message @Kishi on the forum thread and also thank them if you found this useful!
 +
 +The script is [[https://gist.github.com/Kishi85/b7f379f9aa19f4878af28b8e1a8887ab|published as gist on GitHub]] so the full code can be inspected and reviewed before installing.
 +
 +Installation instructions:
 +
 +<code>
 +wget -O /etc/init.d/nft2ipset https://gist.github.com/Kishi85/b7f379f9aa19f4878af28b8e1a8887ab/raw/
 +chmod +x /etc/init.d/nft2ipset
 +service nft2ipset enable
 +service nft2ipset start
 </code> </code>
  
  • Last modified: 2024/11/15 19:05
  • by jeperez