Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
Next revisionBoth sides next revision
lxc_openwrt_host [2021/09/19 08:22] – begin drafting page based on https://forum.openwrt.org/t/openwrt-arm64-quick-lxc-howto-guide-lms-in-debian-system-in-lxc-container/99835 darksky2lxc_openwrt_host [2021/09/22 08:05] – [Setup on the OpenWRT host] darksky2
Line 1: Line 1:
 ====== Running LXC on OpenWRT Host ====== ====== Running LXC on OpenWRT Host ======
  
-In principal, OpenWRT host can run any compatible guest distro via [[https://linuxcontainers.org/lxc/introduction/|LXC]].  In practice, the architecture of the guest OS must match that of the OpenWRT host, therefore only a few architectures are supported including:+In principal, an OpenWRT host can run any compatible guest distro via [[https://linuxcontainers.org/lxc/introduction/|LXC]].  In practice, the architecture of the guest OS must match that of the OpenWRT host, therefore only a few architectures are supported including:
  
 ^ OpenWRT host arch ^ LXC arch ^ ^ OpenWRT host arch ^ LXC arch ^
Line 12: Line 12:
 | tbd | s390x| | tbd | s390x|
  
 +===== Example  =====
 +This example was using OpenWRT ARM64 MVEBU ESPRESSOBIN et ESPRESSOBIN ULTRA.  It has also been verified on a OpenWRT aarch64 Raspberry Pi4 B.
  
 +[[https://forum.openwrt.org/t/openwrt-arm64-quick-lxc-howto-guide-lms-in-debian-system-in-lxc-container/99835|Original Forum Topic]]
 +==== Setup on the OpenWRT host ====
 +Install some necessary tools and prerequisites:
 +<code>opkg install xz tar gnupg cgroupfs-mount cgroup-tools</code>
 +
 +Install the needed kernel modules:
 +<code>opkg install kmod-ikconfig kmod-veth</code>
 +
 +Install the core lxc packages:
 +<code>opkg install lxc-start lxc-stop lxc-create lxc-attach lxc-destroy lxc-config lxc-ls getopt</code>
 +
 +FIXME: Note that getopt should be a package dependency, see: [[https://github.com/openwrt/packages/issues/16684|#16684]] is fixed.
 +
 +Additional packages exist that can add functionality but that aren't strictly required.  Find them with:
 +<code>opkg list | grep lxc</code>
 +
 +FIXME: Note that until [[https://github.com/openwrt/packages/pull/16660|PR#16660]] is merged, users of 21.02 will have to complete this extra step.  Users of snapshot builds newer than 20-Sep-2021 can skip this next step.
 +
 +Edit ''/usr/share/lxc/config/common.conf'' and comment out all lines relating to legacy cgroup configuration:
 +<code>sed -i s/lxc.cgroup.devices/#lxc.cgroup.devices/ /usr/share/lxc/config/common.conf</code>
 +
 +Optionally check the kernel config to see if anything required is missing:
 +<code>root@ultra:~# opkg install lxc-checkconfig
 +# lxc-checkconfig 
 +LXC version 4.0.5
 +--- Namespaces ---
 +Namespaces: enabled
 +Utsname namespace: enabled
 +Ipc namespace: enabled
 +Pid namespace: enabled
 +User namespace: enabled
 +Network namespace: enabled
 +
 +--- Control groups ---
 +Cgroups: enabled
 +
 +Cgroup v1 mount points: 
 +/sys/fs/cgroup/cpuset
 +/sys/fs/cgroup/cpu
 +/sys/fs/cgroup/cpuacct
 +/sys/fs/cgroup/blkio
 +/sys/fs/cgroup/memory
 +/sys/fs/cgroup/pids
 +/sys/fs/cgroup/rdma
 +/sys/fs/cgroup/systemd
 +
 +Cgroup v2 mount points: 
 +
 +
 +Cgroup v1 freezer controller: missing
 +Cgroup v1 clone_children flag: enabled
 +Cgroup device: missing
 +Cgroup sched: enabled
 +Cgroup cpu account: enabled
 +Cgroup memory controller: enabled
 +Cgroup cpuset: enabled
 +
 +--- Misc ---
 +Veth pair device: enabled, loaded
 +Macvlan: enabled, not loaded
 +Vlan: enabled, not loaded
 +Bridges: enabled, not loaded
 +Advanced netfilter: enabled, not loaded
 +CONFIG_NF_NAT_IPV4: missing
 +CONFIG_NF_NAT_IPV6: missing
 +CONFIG_IP_NF_TARGET_MASQUERADE: missing
 +CONFIG_IP6_NF_TARGET_MASQUERADE: missing
 +CONFIG_NETFILTER_XT_TARGET_CHECKSUM: enabled, not loaded
 +CONFIG_NETFILTER_XT_MATCH_COMMENT: enabled, loaded
 +FUSE (for use with lxcfs): enabled, not loaded
 +
 +--- Checkpoint/Restore ---
 +checkpoint restore: missing
 +CONFIG_FHANDLE: enabled
 +CONFIG_EVENTFD: enabled
 +CONFIG_EPOLL: enabled
 +CONFIG_UNIX_DIAG: missing
 +CONFIG_INET_DIAG: missing
 +CONFIG_PACKET_DIAG: missing
 +CONFIG_NETLINK_DIAG: enabled
 +File capabilities: 
 +
 +Note : Before booting a new kernel, you can check its configuration
 +usage : CONFIG=/path/to/config /usr/bin/lxc-checkconfig</code>
 +
 +In order to download distro images for the guest, we need to either:
 +  - Use a keyserver on the host which requires additional setup, or
 +  - Disable validation (not recommended)
 +
 +To verify signature of the images, we need to install some additional packages which can be remove after the guest is setup:
 +<code>opkg install gnupg2-utils gnupg2-dirmngr</code>
 +
 +Alternatively, just use the **--no-validate** switch in the command when setting up the container.  This is potentially dangerous and insecure.
 +
 +Example:
 +<code>root@ultra:~# lxc-create --name myLMS --template download -- --no-validate</code>
 +
 +==== Create a LXC container ====
 +There are many different distros available for installation.  Search for your favorite distro from the supported ones with this command:
 +<code>lxc-create --name myLMS --template download -- --list --no-validate</code>
 +
 +This guide will use Debian Buster selected by pre-specifying the distro, release, and architecture via switches, but it is possible to simply omit these three and select them interactively as well.
 +
 +<code>root@ultra:~# lxc-create --name myLMS --template download -- --dist debian --release buster --arch arm64
 +Setting up the GPG keyring
 +ERROR: Unable to fetch GPG key from keyserver
 +lxc-create: myLMS: lxccontainer.c: create_run_template: 1616 Failed to create container from template
 +lxc-create: myLMS: tools/lxc_create.c: main: 319 Failed to create container myLMS</code>
 +
 +==== Container management ====
 +
 +To list the installed containers and query their status, use **lxc-ls**:
 +<code>root@ultra:~# lxc-ls -f
 +NAME  STATE   AUTOSTART GROUPS IPV4 IPV6 UNPRIVILEGED 
 +myLMS STOPPED 0              -    -    false</code>
 +
 +Start and stop containers with **lxc-start** and **lxc-stop** respectively:
 +
 +<code>root@ultra:~# lxc-start -n myLMS
 +root@ultra:~# lxc-ls -f
 +NAME  STATE   AUTOSTART GROUPS IPV4 IPV6 UNPRIVILEGED 
 +myLMS RUNNING 0              -    -    false</code>
 +
 +<code>root@ultra:~# lxc-stop -n myLMS
 +root@ultra:~# lxc-ls -f
 +NAME  STATE   AUTOSTART GROUPS IPV4 IPV6 UNPRIVILEGED 
 +myLMS STOPPED 0              -    -    false</code>
 +
 +Setup networking in the container:
 +<code>root@ultra:~# nano /srv/lxc/myLMS/config
 +...
 +# Network configuration
 +#lxc.net.0.type = empty
 +lxc.net.0.type = veth
 +lxc.net.0.link = br-lan
 +lxc.net.0.flags = up
 +lxc.net.0.hwaddr = 00:FF:DD:BB:CC:01</code>
 +
 +Optionally mount a share from the OpenWRT host inside the guest
 +Make sure to create the path to the share in the container, then edit the container config adding the following line:
 +<code>lxc.mount.entry = /mnt/SHARE /srv/lxc/myLMS/rootfs/mnt/SHARE none bind,create=d</code>
 +
 +
 +==== Setup the containerized guest distro ====
 +Attach to the guest which will drop you in as root user.  Example below is enabling ssh and fixing sudo within Debian.
 +<code>root@ultra:~# lxc-attach -n myLMS
 +root@myLMS:~#
 +$ adduser admin
 +$ apt install sudo
 +$ addgroup admin sudo
 +$ apt install ssh -y
 +$ ip a
 +root@myLMS:~# ip a
 +1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
 +    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
 +    inet 127.0.0.1/8 scope host lo
 +       valid_lft forever preferred_lft forever
 +    inet6 ::1/128 scope host 
 +       valid_lft forever preferred_lft forever
 +2: eth0@if12: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
 +    link/ether 00:ff:dd:bb:cc:01 brd ff:ff:ff:ff:ff:ff link-netnsid 0
 +    inet 192.168.1.188/24 brd 192.168.1.255 scope global dynamic eth0
 +       valid_lft 42908sec preferred_lft 42908sec
 +    inet6 fdc5:f7f:d0b5:0:2ff:ddff:febb:cc01/64 scope global dynamic mngtmpaddr 
 +       valid_lft forever preferred_lft forever
 +    inet6 fe80::2ff:ddff:febb:cc01/64 scope link 
 +       valid_lft forever preferred_lft forever
 +$ exit
 +</code>
 +
 +==== Auto start the container on OpenWRT host ====
 +<code>opkg install lxc-auto lxc-autostart
 +uci show lxc-auto
 +uci add lxc-auto container
 +uci set lxc-auto.@container[-1].name=myLMS
 +uci set lxc-auto.@container[-1].timeout=30
 +uci show lxc-auto
 +uci commit lxc-auto</code>
  • Last modified: 2024/11/22 12:04
  • by levitatingbusinessman