unetd

unetd is a WireGuard based VPN daemon that simplifies creating and managing fully-meshed VPN connections between OpenWrt routers.

Source: project/unetd.git

• Splits network setup into network config (shared across all participating nodes) and local config (limited mostly to local private key, public signing device key, optionally tunnel device names)
• Supports automatic replication of peer IP addresses
• Supports automatic replication of network config updates
  • network config data is secure and protected by a Ed25519 signature
  • network config data replication is encrypted and only available to members of the network, even during bootstrap
• Fully meshed (all nodes connect to each other directly unless configured otherwise)
• Supports automatic VXLAN setup with multiple peers
  • automatically installs eBPF program to deal with MTU limitations and avoid fragmentation
• Automatic setup of routes / IP addresses based on network config data
• Supports direct connection through double-NAT, as long as the network has one node with a public IP address
• Simple CLI for creating and updating networks between OpenWrt hosts with very few commands
• Builds and runs on regular Linux distributions as well, and also macOS (with some limitations)
• Automatically assigns an IPv6 address for each host, which is generated from the host public key
• writes a host file containing entries for all configured hosts
  • can be used with dnsmasq for local lookup
  • configurable domain suffix
• allows creating freeform service definitions, which allows services to query member IP addresses using ubus
• Supports peer discovery via BitTorrent 'Mainline' DHT, which works even through double-NAT

TBC

The following build dependencies are required:

• cmake, pkg-config
• libelf-dev, zlib1g-dev, libjson-c-dev

To build:

git clone https://git.openwrt.org/project/unetd.git
cd unetd
./build.sh

This set of example commands assumes two OpenWrt routers with the IP addresses 192.168.1.13 and 192.168.1.15 which have not been configured for unetd yet.

This creates a new JSON file test.json and also generates a signing key as test.json.key (if it doesn't exist already):

# unet-cli test.json create

This creates a VXLAN tunnel definition and adds all hosts that are members of the ap group:

# unet-cli test.json add-service l2-tunnel type=vxlan members=@ap

This connects to 192.168.1.13 over SSH, generates an unetd interface named unet, along with new host keys, puts in the signing key and also tells it to create the vx0 vxlan device connected to the l2-tunnel service description we created in the last command, storing the public keys in test.json, along with the endpoint address 192.168.1.13:

# unet-cli test.json add-ssh-host ap1 root@192.168.1.13 endpoint=192.168.1.13 tunnels=vx0:l2-tunnel groups=ap

This does the same for the other host:

# unet-cli test.json add-ssh-host ap2 root@192.168.1.15 endpoint=192.168.1.15 tunnels=vx0:l2-tunnel groups=ap

This signs the network data and uploads it to unetd running on 192.168.1.13:

# unet-cli test.json sign upload=192.168.1.13

Please note that in this case, uploading the data to one of the two hosts is enough, because once it has processed the update, it will find the endpoint address of the other host and sync the network data with it automatically. After that last command, the unetd network should be up on both sides and the VXLAN tunnel created as well.

Configuration of a interface section in /etc/config/network for unetd.

The connect option only needs to be used for bootstrapping the setup in case you're not uploading the network data to the node directly. Once unetd has a working peer connection, it will always replicate updates over the tunnel.

Network config is written as a JSON file.

{
        "config": {
                "port": 51830,
                "keepalive": 10,
                "peer-exchange-port": 51831,
                "stun-servers": [
                        "stun.l.google.com:19302",
                        "stun1.l.google.com:19302"
                ]
        },
        "hosts": {
                "ap1": {
                        "key": "yB3V0Wz37qheZoZiG0KNFAqfuI2TetO4sfXgqO/Gd0c=",
                        "endpoint": "192.168.1.13",
                        "groups": [
                                "ap"
                        ]
                },
                "ap2": {
                        "key": "aGNcyMLrN+C4WW0nJBUGwqw8ifIleUVefv/9vh+d1Fw=",
                        "endpoint": "192.168.1.15",
                        "groups": [
                                "ap"
                        ]
                }
        },
        "services": {
                "l2-tunnel": {
                        "config": {
                        },
                        "members": [
                                "@ap"
                        ],
                        "type": "vxlan"
                }
        }
}

Usage: unet-cli [<flags>] <file> <command> [<args>] [<option>=<value> ...]

     Commands:
      - create:                                 Create a new network file
      - set-config:                             Change network config parameters
      - add-host <name>:                        Add a host
      - add-ssh-host <name> <host>:             Add a remote OpenWrt host via SSH
                                                (<host> can contain SSH options as well)
      - set-host <name>:                        Change host settings
      - set-ssh-host <name> <host>:             Update local and remote host settings
      - add-service <name>:                     Add a service
      - set-service <name>:                     Change service settings
      - sign                                    Sign network data

     Flags:
      -p:                                       Print modified JSON instead of updating file

     Options:
      - config options (create, set-config):
        port=<val>                              set tunnel port (default: 51830)
        pex_port=<val>                          set peer-exchange port (default: 51831, 0: disabled)
        keepalive=<val>                         set keepalive interval (seconds, 0: off, default: 10)
        stun=[+|-]<host:port>[,<host:port>...]  set/add/remove STUN servers
      - host options (add-host, add-ssh-host, set-host):
        key=<val>                               set host public key (required for add-host)
        port=<val>                              set host tunnel port number
        pex_port=<val>                          set host peer-exchange port (default: network pex_port, 0: disabled)
        groups=[+|-]<val>[,<val>...]            set/add/remove groups that the host is a member of
        ipaddr=[+|-]<val>[,<val>...]            set/add/remove host ip addresses
        subnet=[+|-]<val>[,<val>...]            set/add/remove host announced subnets
        endpoint=<val>                          set host endpoint address
        gateway=<name>                          set host gateway (using name of other host)
     - ssh host options (add-ssh-host, set-ssh-host)
        auth_key=<key>                          use <key> as public auth key on the remote host
        priv_key=<key>                          use <key> as private host key on the remote host (default: generate a new key)
        interface=<name>                        use <name> as interface in /etc/config/network on the remote host
        domain=<name>                           use <name> as hosts file domain on the remote host (default: unet)
        connect=<val>[,<val>...]                set IP addresses that the host will contact for network updates
        tunnels=<ifname>:<service>[,...]        set active tunnel devices
     - service options (add-service, set-service):
        type=<val>                              set service type (required for add-service)
        members=[+|-]<val>[,<val>...]           set/add/remove service member hosts/groups
     - vxlan service options (add-service, set-service):
        id=<val>                                set VXLAN ID
        port=<val>                              set VXLAN port
        mtu=<val>                               set VXLAN device MTU
        forward_ports=[+|-]<val>[,<val>...]     set members allowed to receive broadcast/multicast/unknown-unicast
     - sign options:
        upload=<ip>[,<ip>...]                   upload signed file to hosts

For DHT peer discovery, the unet-dht package needs to be installed, and dht enabled in the interface on the nodes. For NAT support, you also need to configure at least one working STUN server in the network data. While peers can find each other through DHT directly, STUN is needed for figuring out the external wireguard port and establishing a network connection over it. Please note that DHT based discovery needs some time for peers to actually discover each other, sometimes 1-3 minutes.