mwan3 package provides the following functionality and capabilities:
0x3F00) which is used to mark outgoing traffic can be configured in the
/etc/config/mwan3globals section. This is useful if you also use other packages (nodogsplash) which use the firewall masking feature. This value is also used to set how many interfaces are supported.
The following steps are taken to route a packet with mwan3:
Every incoming packet (this includes router originated traffic) is handled by the iptables mwan3_hook. This hook takes 5 steps:
Remember that iptables only marks the packet, it does not make routing decisions. Next in line are the ip rules. In following order they are:
Next up are the routing tables. These are really simple. There is just the standard main routing table and one routing table containing one gateway for each wan interface. Route table 1 for the first wan, route table 2 for the second and so on. Hopes this make troubleshooting easier.
IPv6 support is currently a work in progress. mwan3 does support IPv6 but it requires additional configuration, discussed in more detail below.
There are currently several caveats and potential issues you may encounter with IPv6 which makes it more complex to work with compared to IPv4. In more recent versions of mwan3, IPv6 support is an area being targeted to improve and several changes have already been made to help improve compatibility with dual stack configurations. In a majority of cases you will likely need to consider doing some form of IPv6 masquerading, so bear this in mind when implementing mwan3 with IPv6. To help configure IPv6 with mwan3, following the guidance specifically related for dual stack configurations below may help you.
ipv6and create a member profile for each to be used in policies assigned to your rules so IPv4 and IPv6 traffic is handled.
Currently mwan3 does not implement any IPv6 masquerading as part of it's configuration. This is something that needs to be configured at the firewall level.
The default configuration that ships with mwan3 provides an example configuration of having two WAN interfaces with dual stack connectivity (note that the second example interface is not enabled by default). This is a good template to start with if you wish to explore routing IPv6 with mwan3.
Preventing IPv6 rules
You can prevent mwan3 from trying to route IPv6 traffic by declaring
option family 'ipv4' on all rules and removing the default IPv6 rule, this will prevent any mwan3 IPv6 routing rules being created in ip6tables. You should also add
option last_resort 'default' on your policies to fallback to the main routing table to allow IPv6 traffic (if present). However doing this means your IPv6 traffic cannot be balanced or fail-over if not handled by mwan3.
There are various CLI commands to help you troubleshoot or show the current mwan3 configuration:
# mwan3 Syntax: mwan3 [command] Available commands: start Load iptables rules, ip rules and ip routes stop Unload iptables rules, ip rules and ip routes restart Reload iptables rules, ip rules and ip routes ifup <iface> Load rules and routes for specific interface ifdown <iface> Unload rules and routes for specific interface interfaces Show interfaces status policies Show currently active policy connected Show directly connected networks rules Show active rules status Show all status
An example of running
# mwan3 interfaces Interface status: interface wan is online 50h:39m:26s, uptime 345h:27m:03s and tracking is active interface wan6 is online 50h:39m:27s, uptime 345h:26m:05s and tracking is active interface wanb is online 50h:39m:26s, uptime 256h:12m:17s and tracking is active interface wanb6 is online 50h:39m:28s, uptime 256h:12m:14s and tracking is active
Note: In version 2.8.11 and above the
mwan3 interfaces command shows the online time and the overall interface uptime. This is not shown on older versions.
Ensure no other multiple WAN or policy routing packages are installed such as
multiwan installed at the same time as mwan3 is known not to work. Equally make sure you aren't using an other package that makes use of the same firewall mask value mwan3 uses as this will cause conflicts. The firewall mask value used by mwan3 is able to be changed in the configuration to avoid this problem.
Using the latest stable branch build is recommended.
There are currently known issues with using snapshot builds from the next version release branch (20.x). It is recommended to avoid using snapshot builds currently if mwan3 functionality is required unless you are comfortable with compiling with the SDK or building your own images and testing BETA code.
Several issues are currently being tracked by mwan3 maintainers on the issues with snapshot builds, relating to the 5.4 kernel and routing behaviour changes which impacts mwan3 functionality. Key issues include:
You can find the current open issues for mwan3 on the OpenWrt packages repository.
Any router with good OpenWrt support should be suitable, preferably using a supported router with VLAN support to have the additional interface flexibility VLAN support provides is recommended. The simplest way to create additional WAN interfaces is to use VLANs by putting individual LAN switch ports into their own VLANs, thus each becoming separate interfaces, creating independent WAN ports.
If having LAN ports as VLANs in not possible, it is also possible create virtual eths with kmod-macvlan.
opkg update && opkg install kmod-macvlan
Here is a basic example of creating virtual eth interfaces in
config device 'veth5' option name 'veth5' option type 'macvlan' option ifname 'eth1' config device 'veth7' option name 'veth7' option type 'macvlan' option ifname 'eth1'
Before installing mwan3, it is strongly recommended to do some pre-configuration and test your connectivity for each WAN interface prior to enabling mwan3, this will help with troubleshooting and ensure your WAN interfaces are correctly configured before using mwan3.
You will need a minimum of two WAN interfaces for mwan3 to work effectively, this could be two physical WANs, one physical and one logical i.e. OpenVPN/Wireguard.
The simplest way to create more WAN interfaces is to have a VLAN-capable router. This allows each individual port to become its own separate interface in OpenWrt. Here is the basic example procedure to create a new VLAN and assign a single port to it so as to create a second WAN interface
Create additional WAN interfaces as desired if more than two WAN connections will be used. More information on how to create interfaces.
Note for PPPoE WAN interfaces: If you are using PPPoE for multiple ADSL lines from the same company, you may need to use
option macaddr 'XX:XX:XX:XX:XX:XX' to give each interface a unique MAC. The symptom of this problem is that the ISP will drop the connection on one line when another connects with the same (default) MAC.
If you are using a newer release branch build of OpenWrt after 18.06, this step is not necessary anymore. Router initiated traffic is also load-balanced and can fail-over correctly. A new service mwan3rtmon was added by Chen Minqiang. The service is responsible for syncing the main routing table with the interface routing tables. Also as inbound traffic has no dedicated firewall tables anymore. This is now working out of the box without any workarounds needed.
On routers with just one WAN interface (and one default route), there is no issue on which source address to use for new initiated sessions. But with two or more wWAN interfaces you may wish to have control over this. Up until version 2.0, mwan3 did not respect the already set source address of router originated packets. Packets were load-balanced regardless of source address, based on configured user rules.
As of version 2.0 mwan3 does respect the already set source address. The advantage of this is that an applications can have control over which WAN interface to use. The downside of this is that when an application does not specify which source address to use (most of the time) the kernel will pick a source address based on the routing table. In practice this means the default route with the lowest metric is used to determine which source address to use. So if you don't configure a routable loopback address with corresponding more preferred default route, all traffic originating from the router itself will leave the primary WAN with the source address of that wan interface, regardless of configured user mwan3 rules.
This however only effects router initiated traffic. Traffic from LAN clients will always be balanced based on mwan3 configured rules even if no routable loopback address is configured.
Configuring a routable loopback (lede-17.01):
Add the following interface to
config interface 'self' option ifname 'lo' option proto 'static' option ipaddr '192.168.1.1' option netmask '255.255.255.255' option gateway '192.168.1.1'
After this all traffic originating from router itself (if no more specific route is found) will have source address of 192.168.1.1 (before NAT).
Extra advantage is that configuring mwan3 rules for router only traffic is much easier.
Configuring a routable loopback (openwrt-18.06):
You have to add into “/etc/config/mwan3” the option “local_source”. The value must be one of the interfaces in
/etc/config/network. Normally this would be lan.
config globals 'globals' option local_source 'lan' option mmx_mask '0x3F00'
After this all traffic originating from router itself (if no more specific route is found) will have source address of the
lan interface (before NAT).
The address will be configured to the loopback interface lo by netifd on the *ifup/ifdown* hotplug script.
Extra advantage is that configuring mwan3 rules for router only traffic is much easier.
Before doing anything with mwan3 (installing or configuring), ensure that each WAN interface is working and that the default routing table is correctly configured for each WAN connection. Test each interface with a manual ping before installing mwan3!
This is an important step and is compulsory. Time and time again fail to configure this and have a none working setup.
Note: PPPoE connections only show the “Use gateway metric” option if “Use default gateway” option is enabled.
WAN is the default wan interface in this example, and so will get a metric of 10.
WANB is the second wan interface in this example, and so will get the a metric of 20.
If configured correctly, you should have a default gateway (the lines with a target address of 0.0.0.0/0) with a unique metric set for each WAN interface. For example:
# ip route show default via 10.0.3.2 dev eth1 proto static src 10.0.3.15 metric 10 default via 10.0.4.2 dev eth2 proto static src 10.0.4.15 metric 20
Ensure that every WAN interface has a gateway IP and metric defined! This is very important as otherwise mwan3 will likely not work!
Check that each WAN interfaces works by trying to ping www.google.com out from each interface. Ensure all interfaces are correctly sending and receiving traffic before proceeding.
# ping -c 1 -I eth0.1 www.google.com PING www.google.com (220.127.116.11): 56 data bytes 64 bytes from 18.104.22.168: seq=0 ttl=54 time=19.637 ms --- www.google.com ping statistics --- 1 packets transmitted, 1 packets received, 0% packet loss round-trip min/avg/max = 19.637/19.637/19.637 ms
# ping -c 1 -I eth0.2 www.google.com PING www.google.com (22.214.171.124): 56 data bytes 64 bytes from 126.96.36.199: seq=0 ttl=56 time=25.552 ms --- www.google.com ping statistics --- 1 packets transmitted, 1 packets received, 0% packet loss round-trip min/avg/max = 25.552/25.552/25.552 ms
When implementing mwan3 you may experience issues with your ISPs DNS or email services depending your setup. This is due to many ISPs configuring their DNS/email servers to only allow source IP addresses within their network. Any attempts to use such services from “unknown” IP addresses will likely result in the traffic being dropped due to the source address not matching the ISP network.
For DNS, you can either use public open DNS resolvers outside of your ISP network to avoid this problem, or alternatively implement a specific rule in mwan3 to make sure DNS traffic goes over the right WAN interface for such requests. An example rule in mwan3 could be:
config rule 'example_dns_wan' option dest_ip '188.8.131.52,184.108.40.206' option family 'ipv4' option use_policy 'wan_only'
Here I am using Virgin Media UK as the example ISP. Their DNS resolvers are 220.127.116.11 and 18.104.22.168. Any request to these destination IPs are made to use the specific WAN interface this connection goes through, in order for the traffic to correctly traverse the right WAN. You could adapt this rule to be more specific with UDP and port 53, however for easy debugging, this would also work for traceroute, ping etc. The disadvantage to having such a rule however is this traffic can not be load balanced or failover, given a specific WAN policy has been set. Using public DNS resolvers would be the best solution to maintain this, unless you really need to have your ISPs DNS working for your configuration.
In a similar fashion to DNS. An ISP mail server will typically only accept POP3/IMAP/SMTP traffic from IP addresses within their network and block any attempt of sending mail from unknown IP addresses. You will have to ensure mail traffic goes through the right interface as well:
config rule 'example_mail_wan' option dest_ip '22.214.171.124' option family 'ipv4' option use_policy 'wan_only'
This is the IP of smtp.virginmedia.com, you may need to add more IP addresses in order to cover IMAP, POP3 and other SMTP servers if used. You could also add use the
dest_port on rules to limit it to mail related ports.
opkg update && opkg install mwan3 luci-app-mwan3
luci-app-mwan3 is optional, if you don't wish to manage rules through LuCI.
To ensure the new menu item for mwan3 appears, logout of your existing session and restart the service hosting the LuCI interface i.e. uhttpd or just reboot the router.
A new menu entry “Network > Load Balancing” should now be present.
If there is a newer version of mwan3 available, you can upgrade mwan3 through either opkg or LuCI.
Your existing configuration will not be modified and instead if there any changes from the default, these will be able to be viewed in a
mwan3-opkg file alongside your mwan3 configuration file in
/etc/config. Occasionally there may be changes to the configuration options so it is a good idea to inspect the default configuration on upgrades to ensure your configuration has the latest changes in various sections.
A mwan3 configuration consists of five section elements, namely:
The globals configuration provides the following options.
| ||string||yes|| ||Firewall mask value|
| ||boolean||no|| ||Global firewall logging. This must be enabled for any rule specific logging to occur.|
| || ||No|| ||Firewall loglevel|
| ||number||No|| || How often should mwan3rtmon update the interface routing table (in seconds)
Deprecated since v2.9.0
| ||number||No||(none)||Specify an additional routing table to be scanned for connected networks|
Since version 2.9.0
rtmon_interval has been deprecated and will no longer have any effect in configurations, as the way routing table changes are monitored was refactored and no longer requires this.
For each WAN interface configure an interface section and define how each WAN interface is tested for up/down status. Each interface section must have a name that corresponds with the interface name in your network config. The settings are described below.
Certain interface configuration option defaults are mismatched between
mwan3track and LuCI. This means LuCI will display different values if they are not directly set than the values mentioned below. A fix is pending in LuCI to resolve this.
|Interface name||string||yes||(none)|| The OpenWrt interface name as defined in
| || ||no|| ||Tracking method for mwan3track|
| ||boolean||no|| ||Should mwan3 run on/track this interface?|
| ||list of ip addresses||yes||(none)||The host(s) to test if interface is still alive. If this value is missing the interface is always considered up|
| ||number||no|| || Number of track_ip hosts that must reply for the test to be considered as successful. Ensure there are at least this many
| ||number||no|| ||Number of checks to send to each host with each test|
| ||seconds||no|| ||Number of seconds to wait for an echo-reply after an echo-request|
| ||seconds||no|| ||Number of seconds between each test|
| ||seconds||no|| ||Number of seconds between each test during teardown on failure detection|
| ||seconds||no|| ||Number of seconds between each test during tearup on recovery detection|
| ||boolean||no|| ||In the event of an error, keep the number of seconds between each test during teardown (failure detection)|
| ||boolean||no|| || In addition to the interface being up, the
| ||number||no|| || Maximum packet latency milliseconds when
| ||number||no|| || Minimum packet latency in milliseconds when
| ||number||no|| || Maximum packet loss as a percentage when
| ||number||no|| || Minimum packet loss as a percentage when
| || ||no|| ||If the value is offline, then traffic goes via this interface only if mwan3track checked the connection first. If the value is online, then the mwan3track test is not waited for. The Interface is marked as online immediately|
| || ||no|| ||The specific protocol family this interface handles|
| ||number||no|| || Time to live (TTL) or hop limit. Only valid if
| ||number||no|| || Size of ping packets to use in bytes. Only valid if
| ||number||no|| ||Number of successful tests to considered link as alive|
| ||number||no|| ||Number of failed tests to considered link as dead|
In most cases the default values should work for most configurations. The primary reason to change the default settings is to shorten the time before an interface is failed-over (by reducing the ping interval and number of pings before the interface is down) or lengthen the time to avoid a false link failure report. Please note that if you change the timeout value on low bandwidth interfaces (e.g. 3G) or busy interfaces, that false positives of marking a WAN down can occur. A timeout value of less then 2 seconds is not recommended.
A typical interface section looks like this, mostly using the default values of all options described above:
config interface 'wan' option enabled '1' list track_ip '126.96.36.199' list track_ip '188.8.131.52' list track_ip '184.108.40.206' list track_ip '220.127.116.11' option reliability '1' option family 'ipv4'
reliability although the same as the default value is to avoid a misleading error being shown in LuCI.
Below are a collection of public IPv4 and IPv6 endpoints that accept ICMP and can be used with mwan3track for tracking the connection state of interfaces. These are open DNS resolvers with high availability and generally reliable to use as endpoints to confirm network connectivity. Alternatively you can also use your ISPs DNS resolvers, but these are often limited to source networks originating from the ISP and on average can be less reliable.
|DNS service||IPv4 resolvers||IPv6 resolvers|
|Level 3 communications|| 18.104.22.168 |
|Google DNS|| 22.214.171.124 |
|OpenDNS|| 126.96.36.199 |
|Cloudflare|| 188.8.131.52 |
|Hurricane Electric (HE.net)||184.108.40.206||2001:470:20::2|
|Quad9|| 220.127.116.11 |
Each member represents an interface with a metric and a weight value. Members are referenced in policies to define a pool of interfaces with corresponding metric and load-balancing weight. Members can't be used for rules directly. The default settings are described below:
|Member name||string||yes||(none)||The name of this member configuration, which is then referenced in policies|
| ||string||yes||(none)||Member applies to this interface (use the same interface name as used in the mwan3 interface section, above)|
| ||number||no|| ||Members within one policy with a lower metric have precedence over higher metric members|
| ||number||no|| ||Members with same metric will distribute load based on this weight value|
A typical member section looks like this:
config member 'wan_m1_w3' option interface 'wan' option metric '1' option weight '3'
Policies define how traffic is routed through the different WAN interface(s). Every policy has to have one or more members assigned to it, which defines the policy's traffic behavior. If a policy has a single member, traffic will only go out that member. If a policy has more than one member, it will either load-balance among members or use one member but fail-over to another, depending on how the members are configured.
If there is more than one member assigned to a policy, members within the policy with a lower metric have precedence over higher metric members. Members with the same metric will load-balance. Load-balancing members (with same metric) will distribute load based on assigned weights values.
|Policy name||string||yes||(none)||Unique name for the policy. Must be no more than 15 characters|
|Members assigned||string||yes||(none)||One or more members assigned to this policy|
| || ||no|| ||Determine the fallback routing behaviour if all WAN members in the policy are down|
A typical policy section looks like this:
config policy 'balanced' list use_member 'wan_m1_w3' list use_member 'wanb_m1_w2' list use_member 'wan6_m1_w3' list use_member 'wanb6_m1_w2' option last_resort 'unreachable'
A rule describes what traffic to match and what policy to assign for that traffic.
When creating rules from LuCI, the family option is currently not available to configure. This will mean rules created through LuCI default to the family value of
all, applying the rule for both IPv4 and IPv6 which may or may not be valid and can cause iptables related errors. You will need to manually edit the
/etc/config/mwan3 to explicitly define a valid family value. A pending pull request is currently in development to fix this in luci-app-mwan3.
These are the available options:
|Rule name||string||yes||(none)||The name of the rule. Must be no more than 15 characters|
| ||string||yes||(none)|| Use this policy for traffic that matches or set to
| ||IP address||no||any||Match traffic from the specified source IP address|
| ||port or range||no||any|| Match traffic from the specified source port or port range, if relevant
| || ||no|| ||Match traffic using the given protocol.|
| ||IP address||no||any||Match traffic directed to the specified destination IP address|
| ||port or range||no||any|| Match traffic directed at the given destination port or port range, if relevant
| ||string||no||(none)||Match traffic directed at the given destination IP address to an ipset set|
| ||boolean||no|| ||Allow traffic from the same source IP address within the timeout limit to use same wan interface as prior session|
| ||number||no|| ||Stickiness timeout value in seconds|
| || ||no|| ||Address family for which to apply the rule.|
| ||boolean||no|| ||Enables firewall rule logging (global mwan3 logging setting must also be enabled)|
A typical rule section looks like this:
config rule 'default_rule_v4' option dest_ip '0.0.0.0/0' option family 'ipv4' option use_policy 'wan_wanb'
It is also possible to group multiple ports or source/destination IP addresses under a single rule using a comma.
config rule 'multi_ip_rule' option dest_ip '18.104.22.168,22.214.171.124,126.96.36.199,188.8.131.52' option family 'ipv4' option use_policy 'wan_only'
The comma will be translated by
iptables and correctly create the required entries from a single rule.
For rules that require a large amount of destination IP addresses, it is recommended to use ipset as this more optimised to group large amounts of IP addresses, or CIDR ranges.
ipset and sticky support is available in mwan3 1.6 or higher
Stickiness lets you route a new session over the same WAN interface as the previous session, as long as the time between the new and the previous session is shorter then the timeout value (default 600 seconds). This can solve some problems with HTTPS sites, which don't allow a new source address within the same cookie/HTTPS session. Using ipset lets you route traffic over WAN interfaces based on set of IP addresses. A set can be created by hand, by dnsmasq based on domain names, or your own script. Rules with ipset option will try to match destination IP address to the configured ipset.
config rule 'youtube' option sticky '1' option timeout '300' option ipset 'youtube' option dest_port '80,443' option proto 'tcp' option use_policy 'balanced'
With sticky set to 1, this rule has now sticky enabled. When a packet for a new session matches this rule, its source IP address and interface mark are stored in an ipmark set with a timeout of 300 seconds. When a packet for a second new session from the same LAN host within the timeout period matches this rule, it will use the same WAN interface as the first packet and the timeout counter is reset back to 300 again.
Stickiness is on a per rule basis. With this example, all traffic from LAN hosts will use the same WAN interface for all hosts in the ipset, even if the source or destination IP address differs.
The option ipset matches only destination IP addresses. This example will only work if your LAN clients use the dnsmasq server as their one and only DNS server or have your configured existing upstream DNS resolvers use the dnsmasq server as their forwarder.
If the ipset chain does not already exist, mwan3 will create the ipset set for you. For this to work you need to configure a rule for dnsmasq in your
config dnsmasq .... list ipset '/youtube.com/youtube'
Or add directly to
You will then need to restart dnsmasq for the ipset change to be applied.
This is an example configuration based off the version provided in the master branch of mwan3. By default only a single WAN interface is enabled, but it provides the necessary configuration for having two WAN interfaces that are dual stack. You can adapt this configuration to your specific needs.
config globals 'globals' option mmx_mask '0x3F00' config interface 'wan' option enabled '1' list track_ip '184.108.40.206' list track_ip '220.127.116.11' list track_ip '18.104.22.168' list track_ip '22.214.171.124' option reliability '2' option family 'ipv4' config interface 'wan6' option enabled '0' list track_ip '2001:4860:4860::8844' list track_ip '2001:4860:4860::8888' list track_ip '2620:0:ccd::2' list track_ip '2620:0:ccc::2' option reliability '2' option family 'ipv6' config interface 'wanb' option enabled '0' list track_ip '126.96.36.199' list track_ip '188.8.131.52' list track_ip '184.108.40.206' list track_ip '220.127.116.11' option reliability '1' option family 'ipv4' config interface 'wanb6' option enabled '0' list track_ip '2001:4860:4860::8844' list track_ip '2001:4860:4860::8888' list track_ip '2620:0:ccd::2' list track_ip '2620:0:ccc::2' option reliability '1' option family 'ipv6' config member 'wan_m1_w3' option interface 'wan' option metric '1' option weight '3' config member 'wan_m2_w3' option interface 'wan' option metric '2' option weight '3' config member 'wanb_m1_w2' option interface 'wanb' option metric '1' option weight '2' config member 'wanb_m2_w2' option interface 'wanb' option metric '2' option weight '2' config member 'wan6_m1_w3' option interface 'wan6' option metric '1' option weight '3' config member 'wan6_m2_w3' option interface 'wan6' option metric '2' option weight '3' config member 'wanb6_m1_w2' option interface 'wanb6' option metric '1' option weight '2' config member 'wanb6_m2_w2' option interface 'wanb6' option metric '2' option weight '2' config policy 'wan_only' list use_member 'wan_m1_w3' list use_member 'wan6_m1_w3' config policy 'wanb_only' list use_member 'wanb_m1_w2' list use_member 'wanb6_m1_w2' config policy 'balanced' list use_member 'wan_m1_w3' list use_member 'wanb_m1_w2' list use_member 'wan6_m1_w3' list use_member 'wanb6_m1_w2' config policy 'wan_wanb' list use_member 'wan_m1_w3' list use_member 'wanb_m2_w2' list use_member 'wan6_m1_w3' list use_member 'wanb6_m2_w2' config policy 'wanb_wan' list use_member 'wan_m2_w3' list use_member 'wanb_m1_w2' list use_member 'wan6_m2_w3' list use_member 'wanb6_m1_w2' config rule 'https' option sticky '1' option dest_port '443' option proto 'tcp' option use_policy 'balanced' config rule 'default_rule_v4' option dest_ip '0.0.0.0/0' option family 'ipv4' option use_policy 'balanced' config rule 'default_rule_v6' option dest_ip '::/0' option family 'ipv6' option use_policy 'balanced'
# mwan3 status Interface status: interface wan is online 00h:00m:54s, uptime 71h:53m:14s and tracking is active interface wan6 is online 00h:00m:54s, uptime 71h:51m:35s and tracking is active interface wanb is online 00h:00m:54s, uptime 71h:53m:15s and tracking is active interface wanb6 is online 00h:00m:54s, uptime 71h:53m:11s and tracking is active Current ipv4 policies: wan_only: wan (100%) wan_wanb: wan (100%) wanb_only: wanb (100%) wanb_wan: wanb (100%) Current ipv6 policies: wan_only: wan6 (100%) wan_wanb: wan6 (100%) wanb_only: wanb6 (100%) wanb_wan: wanb6 (100%) Directly connected ipv4 networks: 192.168.1.0/24 192.168.225.0/24 192.168.100.0/24 192.168.2.0/24 172.16.0.0/12 18.104.22.168/3 127.0.0.0/8 192.168.0.0/16 10.0.0.0/8 192.168.10.0/24 Directly connected ipv6 networks: fd77:550d:5fb8::/64 fd77:550d:5fb8:10::/64 fc00:bbbb:bbbb:bb01::3:d260 fe80::/64 fc00:bbbb:bbbb:bb01::1:611c Active ipv4 user rules: 11 851 S https tcp -- * * 0.0.0.0/0 0.0.0.0/0 multiport dports 443 72 12381 - wan_wanb all -- * * 0.0.0.0/0 0.0.0.0/0 Active ipv6 user rules: 14 1120 S https tcp * * ::/0 ::/0 multiport dports 443 2 184 - wan_wanb all * * ::/0 ::/0
It would be good to be able to automatically monitor interface status through mwan3 using a monitoring system (for example, OpenNMS or monit).
It would be good for mwan3 to send some kind of alert (e.g. e-mail, SNMP trap) if it detects a failed interface and performs a failover, or if it performs a fail-back.
If you install the mwan3 package it comes with
/etc/mwan3.user hotplug shell script which may be modified to perform custom actions on interface ifup/ifdown events. It is already filled with a basic template for gathering information on the current state of WAN connections and just needs your changes in the send_alert() function as well as uncommenting the case statement at the bottom. The mwan3 LuCI package provides a page for editing this hotplug shell script or you may edit via command line.
When using multiple WAN connections, there will be multiple external IPs which can be used as the external IP for outgoing NATed traffic. In particular, an external interface might have a block of external IPs that should be mapped in a particular way to specified internal servers. For example, the internal mail server should send out traffic on the same external IP identified in its MX record. This is the procedure to do this.
Add an mwan3 traffic rule that directs the specific desired source IP out the correct WAN interface. Rules are processed in top-down order, so be sure this specific rule is higher in the list (thus higher priority) than more general rules below that implement load-balancing or failover in the default case.
config policy 'wanb_only' list use_member 'wanb_m1_w2' list use_member 'wanb6_m1_w2'
config rule 'mailserver_uses_wanb_only' option src_ip '172.16.1.20' option family 'ipv4' option use_policy 'wanb_only'
If the external WAN interface only has a single external IP, this is all that is needed. If the interface has multiple external IPs, both the next two steps are also needed.
This step is only needed if the desired external interface needs to have multiple external IP addresses assigned to it.
The specified external interface may have multiple IPs assigned to it. For OpenWrt 12.09, the preferred way to do this is using multiple interface definitions – see reference.
This step is only needed if the desired external interface has multiple external IP addresses assigned to it.
With multiple external IP addresses, we want to control which address is used when sending out traffic from particular servers. This is configured using a source NAT rule in OpenWrt.
Note that both a mwan3 rule to select the interface and an SNAT rule to select the specific IP on that interface are needed to correctly send traffic out a specific external IP.
config redirect option target 'SNAT' option name 'Mail server goes out 22.214.171.124' option src 'lan' option dest 'wan' option src_ip '172.16.1.20' option src_dip '126.96.36.199' option proto 'all'
This is the case where you want external clients using a DDNS name to automatically reconnect to the alternate WAN interface if the primary WAN interface fails.
This is the case where you want each specific WAN interface to register its own DDNS name and the WAN interface in question has an external IP directly assigned to it.
This is the case where you want each specific WAN interface to register its own DDNS name but the WAN interface in question is behind a NAT device and so does not directly have the correct external IP.
This is tricky when the WAN interface is not the default WAN interface, as ddns-scripts cannot be configured to use a specific interface to check its IP.
If the openwrt system is an openvpn client and a zone 'vpn' is defined on the vpn interface and this zone has the masquerading active, for reasons (yet) unknown the traffic from the internal lan to the vpn will be able to reach the destination and go back to the router but then will be not dispatched back to the lan clients. Disabling mwan3, instead, let the traffic be dispatched properly.
It could be a misconfiguration, more testing is needed.
If load-balancing between multiple WAN interfaces, it is desirable to have OpenVPN clients be able to connect through all active WAN interfaces.
In a multiple WAN interface failover scenario, OpenVPN will not accept client connections on the secondary WAN interface after a failover, as it started listening on the primary WAN interface when it was started.
The following configuration will allow multiple WAN interface to be used with OpenVPN Server.
... # Which local IP address should OpenVPN # listen on? (optional) ;local a.b.c.d ## Customization: have OpenVPN listen on the internal LAN interface IP only to allow client re-connections after a WAN interface failover local 192.168.1.1 ...
ifconfig, the redirect will transform in a DNAT+input accept rule, and the vpn server would be reachable. If the router has more than one ip address on the LAN interface, using one of them not mentioned in the
ifconfigwill cause the firewall application to transform it in a DNAT+forward rule and this means that the packet will be not routed on the router itself, therefore showing that then vpn port is unreachable.
OpenWrt 15.05.x (Chaos Calmer) note: Unfortunately, the above approach doesn't work for UDP port-forwards to the router's LAN interface fail to work. TCP port-forwards are fine. This bug report talks about the issue: https://dev.openwrt.org/ticket/18057. Apparently the change in the firewall3 package that broke this functionality has been reverted but the fix happened after the 15.05.x CC release.
If you want to use your OpenVPN client tunnels as virtual wan interfaces in mwan3, you have to make sure that you set a default route with different metric for each tunnel interface. Also most commercial VPN solutions push two static routes to override the standard default gateway. In most cases you don't want this override when using OpenVPN client tunnels in conjunction with mwan3.
As a solution you can add the following lines to your OpenVPN client config:
route-nopull route 0.0.0.0 0.0.0.0 vpn_gateway 20
This example will ignore the routes pushed from the OpenVPN server and will add a default route with metric 20 over the OpenVPN tunnel interface.
Transparent HTTP proxying relies on using iptables rules to transparently redirect outgoing traffic to port 80 first through the local proxy at another port number.
For example, here is a OpenWrt redirect rule to redirect outgoing traffic to TCP 80 port and re-send it to the local proxy listening on TCP port 8118. This will go into iptables NAT table rules.
config redirect option target 'DNAT' option dest 'lan' option proto 'tcp' option src 'lan' option src_dip '!10.0.2.1' option src_dport '80' option dest_ip '10.0.2.1' option dest_port '8118' option name 'Transparent Proxy [privoxy]' option enabled '1'
The problem is that mwan3 adds rules to the iptables's MANGLE table, and this is handled before the NAT table. So when a client makes a request to fetch a web page, it is first marked by mwan3. Mwan3 decides based on your mwan3 rules which wan interface to exit and marks the session accordingly.
Next, iptable nat rule handling takes place and diverts the web page request to privoxy. The reply from privoxy however is part of the same session and is already marked to leave a wan interface. The reply from privoxy is then send over the internet, which is obviously incorrect.
To fix this add the following rules to your mwan3 config:
config rule 'rule1' option proto 'tcp' option dest_port '80' option src_ip '10.0.2.1' option dest_ip '0.0.0.0/0' option family 'ipv4' option use_policy 'wan_wanb_loadbalanced' config rule 'rule2' option proto 'tcp' option dest_port '80' option src_ip '10.0.2.0/24' option dest_ip '0.0.0.0/0' option family 'ipv4' option use_policy 'default' config rule 'rule3' option dest_ip '0.0.0.0/0' option family 'ipv4' option use_policy 'wan_wanb_loadbalanced'
The policy “wan_wanb_loadbalanced” is just an example. Change it to whatever policy you like.
Since NoDogSplash v3.1.0, mwan3 and NoDogSplash work fine together without any configuration changes. Both use iptables mark bits but NoDogSplash defaults to using bits carefully chosen for compatibility with other packages. A common symptom of any incompatibility would be the NoDogSplash splash page appearing for every page even as an authenticated client.
It is possible to change the settings for NoDogSplash in the UCI config file (/etc/config/nodogsplash).
The default settings are shown in this section from the config file:
# Nodogsplash uses specific HEXADECIMAL values to mark packets used by iptables as a bitwise mask. # This mask can conflict with the requirements of other packages such as mwan3, sqm etc # Any values set here are interpreted as in hex format. # # Option: fw_mark_authenticated # Default: 30000 (0011|0000|0000|0000|0000 binary) # # Option: fw_mark_trusted # Default: 20000 (0010|0000|0000|0000|0000 binary) # # Option: fw_mark_blocked # Default: 10000 (0001|0000|0000|0000|0000 binary) # #option fw_mark_authenticated '30000' #option fw_mark_trusted '20000' #option fw_mark_blocked '10000'
These values let NoDogSplash work with mwan3, SQM etc. but can be changed if necessary.
Premise: very nice piece of work given to the internet community, congrats to the contributors. We, users, can only give back a bit of experience and documentation, still somehow useful.
In a simulated test environment described as follows:
external network <---> (ip a.b.c.118) router1 (ip 192.0.2.1) <---> (ip 192.0.2.166 - wan) tplink (ip 192.168.1.1) <---> (ip 192.168.1.50) pcA <---> (ip a.b.c.224) router2 (ip 188.8.131.52) <---> (ip 184.108.40.206 - wanb) (ip 192.168.10.1) <---> (ip 192.168.10.101)pcB
The mwan3 was installed on the tplink with the following configuration (apart from wan and wanb):
# /etc/config/mwan3 ... config member 'wan_m10' option interface 'wan' option metric '10' config member 'wan_m20' option interface 'wan' option metric '20' config member 'wanb_m10' option interface 'wanb' option metric '10' config member 'wanb_m20' option interface 'wanb' option metric '20' config policy 'wan_wanb' list use_member 'wan_m10' list use_member 'wanb_m20' config policy 'wanb_wan' list use_member 'wan_m20' list use_member 'wanb_m10' config rule 'rule1' list comment 'from 192.168.1.50 to wan_wanb' option src_ip '192.168.1.0/24' option dest_ip '0.0.0.0/0' option family 'ipv4' option use_policy 'wan_wanb' config rule 'rule2' list comment 'from 192.168.10.101 to wanb_wan' option src_ip '192.168.10.0/24' option dest_ip '0.0.0.0/0' option family 'ipv4' option use_policy 'wanb_wan'
Then with the pcA and pcB two different download were started, and every pc used a different connection. Great.
When one wan connection was physically disconnected, one pc lost the tcp active connections, but after 'restarting' them no problem, the pc was using the other wan connection configured as line backup. Great.
If a wan line recovers (let's say A), then the pc that was using the other wan line (configured as backup, let'S say B) is switched back to the wan line A, and this cause another disruption of tcp connections.
Edit: On recovery, connections already established over backup links will not be terminated and continue to traverse over backup wan. Only new connections will be routed over preferred wan.
Rdp connections to the pc behind the tplink are stable, this means that a service behind the tplink with mwan3 is reachable in a reliable way.
Therefore does not happen that an external request coming on one wan connection gets the replies through the other wan connection (at least for failover policies)
At least testing with ssh, does not happen that one connection through a wan line is router on the other wan line in case of line failover. Therefore ssh is stable.
mwan3 status shows often 'hits' only for the last rule if this one is generic (source 0.0.0.0/24 dest 0.0.0.0/24 )
like for web traffic. This is a bit misleading if in reality the traffic was
split by other rules. I do not know why iptables reports this (mwan3 status just reports what iptables
says) but analyzing it with iftop and bmon there is better to asses if the traffic is
directed properly using the wan connections.
Currently the primary maintainer of mwan3 is Florian Eckert an OpenWrt developer/maintainer, supported by other OpenWrt users and contributors.