Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
docs:guide-user:virtualization:virtualbox-advanced [2020/01/11 08:54] – [squashfs vs ext4] someothertimedocs:guide-user:virtualization:virtualbox-advanced [2020/08/15 16:11] (current) – [Resources] add flink re: 80211 bridging issues someothertime
Line 5: Line 5:
 This guide extends the basic VirtualBox HowTo with broader setup recommendations, samples and common "gotcha-s". This guide extends the basic VirtualBox HowTo with broader setup recommendations, samples and common "gotcha-s".
  
 +NOTE: Use a wired virtualbox host (unless you are really confident with virtualbox i.e. 80211 bridging issues ).
 ===== Prerequisite Concepts ===== ===== Prerequisite Concepts =====
 ==== x86_64 Basics ==== ==== x86_64 Basics ====
-As the operating system within VirtualBox is basically an x86_64 host, having a general grasp of the hard drive partition setup, bootloader operation etc. will assist. +As the operating system within VirtualBox is basically an x86_64 host, having a general grasp of the hard drive partition setup, bootloader operation et. al. is beneficial. Particularly, when you need to undertake more advanced network interface setup within the VM and then translate that to VBox nics. In other words, lack of confidence performing bare metal config will compound troubleshooting if you are unable to clearly differentiate what is physical and what is virtual. In this case perhaps using an old pc or spare usb disk to test on real hardware first is a good first step.
  
 ==== squashfs vs ext4 ==== ==== squashfs vs ext4 ====
Line 15: Line 16:
 {{:media:netdiag1c.png?direct&600|}} {{:media:netdiag1c.png?direct&600|}}
  
-===Using the squashfs:=== +===Using squashfs:=== 
-  * With squashfs you can sysupgrade at any time to update the kernel, which is useful if you are tracking snapshot to have the latest and greatest  +  * sysupgrade at any time to update the kernel, which is useful if you are tracking snapshot to have the latest and greatest  
-  * With squashfs you can factory reset +  * factory reset 
-  * While the defualt size of read-write partition in a squashfs image is 128MB it's a bit uncommon to actually install so much packages to fill it up +  * the default read-write partition in a squashfs image is 128MB, it's uncommon to actually fill it up with common package additions
   * If you need more space for data you can always increase the virtual disk size and add a data partition (which will be preserved on sysupgrade), or even better add more virtual drives you would use just for storage.   * If you need more space for data you can always increase the virtual disk size and add a data partition (which will be preserved on sysupgrade), or even better add more virtual drives you would use just for storage.
  
-For most other devices, the squashfs image is actually neat when you are using the [[docs:guide-user:additional-software:imagebuilder|Imagebuilder]]  to integrate all packages you want in a single image you then use with sysupgrade, so you are not losing any of your "installed packages" with a sysupgrade as you have integrated them in the base firmware image.+Squashfs images are neat when you are using the [[docs:guide-user:additional-software:imagebuilder|Imagebuilder]]  to integrate all packages you want in a single re-settable image. You can install with sysupgrade, and not lose any of your "installed packages" as you have integrated them in the base firmware image.
  
-Since you have Virtualbox anyway, why not have a Debian or Ubuntu VM and use the Imagebuilder too. It's a very convenient way of tracking snapshot to have the latest and greatest, and the only practical way to do so for most other devices supported by OpenWrt, so this is more or less the best "choice" for OpenWrt veterans that have deployed many devices and are using the imagebuilder already.\\ +Since you have Virtualbox anyway, why not have a Debian or Ubuntu VM and use the Imagebuilder too. It's a very convenient way of tracking snapshot(master) to have the latest and greatest, and the only practical way to do so for most other devices supported by OpenWrt, so this is more or less the best "choice" for OpenWrt veterans that have deployed many devices and are using the imagebuilder already.\\ 
-Note, if you use the Imagebuilder, the size of the read-write partition is irrelevant as all packages go in the squashfs read-only partition that is enlarged as needed, so 128MB is for config only.+NB: With Imagebuilder, the size of the read-write partition is irrelevant as all packages are embedded in the squashfs read-only partition that is enlarged as needed, so 128MB is available for config only.
  
-===Using the ext4:=== +===Using ext4:=== 
-  * With ext4 you lose the ability to sysupgrade but you can keep updating packages over and over.+  * With ext4 you lose the ability to sysupgrade but you can keep updating packages over and over. ( todo: clarify see final sentence ) 
 +  * Depending on your hdd durability, there is more wear but most users have zero issues. Kindof mute for vm's.
   * Updating the kernel must be done manually by mounting the first partition (**/dev/sda1** if you have only one disk) and replacing the "**vmlinuz**" file you find in the **/boot** folder with the file with the same name you find in the download folder https://downloads.openwrt.org/releases/18.06.4/targets/x86/64/ , then installing the "**kernel_xxxx.ipk**" package you find here https://downloads.openwrt.org/releases/18.06.4/targets/x86/64/packages/ for example if you just updated to 18.06.04 stable release, and then reboot. \\That's a virtual package, it exists only to tell the system what kernel version is installed but does not actually install anything. The reason this is not done is that the kernel installation in 99% of OpenWrt targets happens through Sysupgrade process, not through packages, so none took the time and effort to actually have kernel upgrades done properly in this fashion. But anyway, it's easy enough to script around this issue I guess.   * Updating the kernel must be done manually by mounting the first partition (**/dev/sda1** if you have only one disk) and replacing the "**vmlinuz**" file you find in the **/boot** folder with the file with the same name you find in the download folder https://downloads.openwrt.org/releases/18.06.4/targets/x86/64/ , then installing the "**kernel_xxxx.ipk**" package you find here https://downloads.openwrt.org/releases/18.06.4/targets/x86/64/packages/ for example if you just updated to 18.06.04 stable release, and then reboot. \\That's a virtual package, it exists only to tell the system what kernel version is installed but does not actually install anything. The reason this is not done is that the kernel installation in 99% of OpenWrt targets happens through Sysupgrade process, not through packages, so none took the time and effort to actually have kernel upgrades done properly in this fashion. But anyway, it's easy enough to script around this issue I guess.
   * With ext4 you can increase the partition size to your heart's content, but you will need to do it "offline", i.e. by booting a Gparted liveCD in your Virtualbox VM, as the default ext4 image is too small (technical reason: //the filesystem does not contain enough inodes at this size//) to allow online resizing (//ext4 can be enlarged without using a liveCD if it is big enough at the start//)   * With ext4 you can increase the partition size to your heart's content, but you will need to do it "offline", i.e. by booting a Gparted liveCD in your Virtualbox VM, as the default ext4 image is too small (technical reason: //the filesystem does not contain enough inodes at this size//) to allow online resizing (//ext4 can be enlarged without using a liveCD if it is big enough at the start//)
  
-This type of image mimics a more conventional Linux distro like Debian or Ubuntu, where you have packages and you keep updating or installing them as needed. But afaik this is a bit of a pain to do if you try to follow snapshot, as OpenWrt does not implement the ability to have multiple kernels with kernel modules (the Linux "device drivers") installed at the same time, and select what kernel to boot when you reboot.+This type of image mimics a more conventional Linux distro like Debian or Ubuntu, where you have packages and you keep updating or installing them as needed. But afaik this is a bit of a pain to do if you try to follow snapshot, as OpenWrt does not implement the ability to have multiple ( non-identical ) kernels with kernel modules (the Linux "device drivers") installed at the same time, and select what kernel to boot when you reboot. Although it is possible to manually edit grub.cfg to support this manually using differing rootfs partitions for each kernel ( without sysupgrade support )
  
 So when you go and update the kernel or the kernel modules (packages that start with **kmod-xxxx.ipk**) you will probably experience breakage or will have to fight against opkg that refuses to install them as they are not compatible with the kernel in use. I //think// that you could actually do this if you install the kernel first, which is a virtual package, and then install the kmod packages and then reboot immediately, but I never tried so YMMV. So when you go and update the kernel or the kernel modules (packages that start with **kmod-xxxx.ipk**) you will probably experience breakage or will have to fight against opkg that refuses to install them as they are not compatible with the kernel in use. I //think// that you could actually do this if you install the kernel first, which is a virtual package, and then install the kmod packages and then reboot immediately, but I never tried so YMMV.
  
 +Therefore, using sysupgrade to flash a new combined-ext.bin will typically zap the "boot" and "rootfs" partitions, config is migratable, packages are not.
 ==== VirtualBox Networking Basics ==== ==== VirtualBox Networking Basics ====
 Knowledge of the basic network types offered by VirtualBox will help you immensely...  Knowledge of the basic network types offered by VirtualBox will help you immensely... 
Line 99: Line 102:
     --nictype1 82540EM \     --nictype1 82540EM \
     --nic2 "bridged" \     --nic2 "bridged" \
-    --nictype2 82540EM \+    --nictype2 82540EM
  
 $VBOXMANAGE storagectl $VMNAME \ $VBOXMANAGE storagectl $VMNAME \
Line 126: Line 129:
 exit 0 exit 0
 </code> </code>
- 
  
  
 === Add some shaping === === Add some shaping ===
  
-As virtualbox built in shaping only works in one direction... example 1 is the perfect network architecture to perform shaping using the built in virtualbox bandwidth limits. +As virtualbox built in shaping only works in one direction... example 1 is the perfect network architecture to perform shaping using the built in virtualbox bandwidth limits. It is possible to change the speed on the fly while the vm is running.
- +
-It is possible to change the speed on the fly while the vm is running+
  
 ( NOTE: "wan" in the script below refers to the vm-nic1(second) ) ( NOTE: "wan" in the script below refers to the vm-nic1(second) )
Line 141: Line 141:
  
 VMNAME="openwrtx64-20191002-2352" VMNAME="openwrtx64-20191002-2352"
-lannicspeed="350k" #.35MBps use m for mbps +lannicspeed="350k" # .35mb/s use m for mbps 
-wannicspeed="360k"+wannicspeed="360k" #
  
 VBoxManage bandwidthctl "$VMNAME" add Limitlan --type network --limit $lannicspeed VBoxManage bandwidthctl "$VMNAME" add Limitlan --type network --limit $lannicspeed
Line 148: Line 148:
 VBoxManage modifyvm "$VMNAME" --nicbandwidthgroup1 Limitlan #assign to nic1 VBoxManage modifyvm "$VMNAME" --nicbandwidthgroup1 Limitlan #assign to nic1
 VBoxManage modifyvm "$VMNAME" --nicbandwidthgroup2 Limitwan #assign to nic2 VBoxManage modifyvm "$VMNAME" --nicbandwidthgroup2 Limitwan #assign to nic2
 +</code>
 +
 +
 +=== Tagging for VM's  ===
 +
 +Note: example ip link is non-permanent... see your distro for appropriate place to configure your host nic for a vlan.
 +
 +You can use a single cable as a "trunk". This essentially turns your wired link into a managed switch as you will tag VLAN per vm upstream on the wired link.
 +
 +This can be used for clients... with openwrt on a "real router" running several vlans over a single LAN port. Or you could use this method for an OpenWrtVM... tag several of it's interfaces outside of the guest software... "mimicking" a managed switch.
 +
 +
 +This example shows clientVM's to an upstream OpenWrt router.
 +<code bash>
 +
 +######################################################### Step1 ONHOST
 +NIC="enp0s25"
 +ip link add link $NIC name $NIC.50 type vlan id 50
 +ip link set dev $NIC.50 up
 +
 +######################################################### Step2
 +In VirtualBox change bridge for a VM to interface NIC.50
 +
 +######################################################### Step3 In Openwrt
 +#-Go to switch... add vlan 50 tagged on same port as pc/server + cpu1
 +#-Go to interfaces... add interface... NAME + INTERFACE:ethNcpu.50 + IP:192.168.50.1 etc. etc.
 +
 +
 +################################## debugging on host
 +#ip -d link show $NIC.50
 +#ip link delete $NIC.50
 +#tcpdump -nnei $NIC -vvv
 +
 </code> </code>
  
Line 164: Line 197:
  
 {{:media:netdiag1b.png?direct&600|}} {{:media:netdiag1b.png?direct&600|}}
 +
  
 As show in the diagram; As show in the diagram;
Line 172: Line 206:
  
 You now have a powerful internal router for VPN and any other service you wish. Leaving the edge router to do simple routing, nat and firewalling tasks. Give some clients your normal LAN gateway and some clients the OpenWrt VM, and presto! poor mans PBR ;). You now have a powerful internal router for VPN and any other service you wish. Leaving the edge router to do simple routing, nat and firewalling tasks. Give some clients your normal LAN gateway and some clients the OpenWrt VM, and presto! poor mans PBR ;).
 +
 +=== CLI Basic Network Access Setup (vi dhcp) ===
 +* use gateway and dns options for static ip
 +
 +{{:media:vbox-simple-opkg-as-client.webm?direct&600|}}
 +
 +
 +=== CLI Install LUCI on snapshot ===
 +{{:media:vbox-simple-opkg-add-luci-to-snapshot.webm?direct&600|}}
 +
 +
 +
  
 ==== Buildroot Export ==== ==== Buildroot Export ====
Line 221: Line 267:
 Then change the correct VM>bridged-to interface to br201 in VirtualBox. Then change the correct VM>bridged-to interface to br201 in VirtualBox.
  
 +NOTE: You may need to set the vbox-nic promiscous mode to Allow if bridging to a host bridge
 +https://forum.openwrt.org/t/only-the-first-interface-in-br-lan-is-working-properly-with-a-virtualbox-x86-64-openwrt/62539/5
 +
 +
 +==== ToDo Serial and SSH console examples ====
 +
 +==== Video of Boot ====
 +For boot-time debugging purposes... you would generally want to attach a vbox-console. One nice and easy trick is to use the capture facility of VirtualBox. Not so handy for copy paste but this method has it's own advantages.
 +
 +Settings => Display => Recording [x] Enable Recording
 +
 +( NB: you must repeat this every boot, you can find the default .webm file in the VirtualBox machines directory )
 ===== Tips beyond the Basic ===== ===== Tips beyond the Basic =====
   * vnic0 ( LAN ) is key. Get access to it... Know how it works at each level of abstraction.   * vnic0 ( LAN ) is key. Get access to it... Know how it works at each level of abstraction.
Line 264: Line 322:
 [5] Image filesystems and alteration in detail [5] Image filesystems and alteration in detail
 [[https://quantumwarp.com/kb/articles/25-dsl-broadband/899-run-lede-as-a-virtualbox-virtual-machine]] [[https://quantumwarp.com/kb/articles/25-dsl-broadband/899-run-lede-as-a-virtualbox-virtual-machine]]
 +
 +[7] forum re: macos 80211 bridging issues
 +[[https://forum.openwrt.org/t/virtualbox-openwrt-works-fine-but-clients-connecting-to-vm-router-have-no-internet/71398]]
 +
  
  • Last modified: 2020/01/11 08:54
  • by someothertime