ESXi on Raspberry Pi

ESXi on Raspberry Pi

Chris Evans Composable Infrastructure, Containers, Edge Computing, IoT, Lab Work, Virtualisation, VMware Leave a Comment

As part of VMworld 2020, VMware announced the availability of the ESXi hypervisor on a range of ARM devices, including the Raspberry Pi. Many home lab fans will be looking to get hold of this software and see how virtualising on ARM might work. Are there any practical benefits to virtualising the Raspberry Pi?

Evaluation

I currently have two “spare” Pi devices, both 4B models with 8GB of RAM. So I took the plunge and followed the instructions posted here (ESXi Arm Edition).

Raspberry Pi 4 hardware

Getting ESXi up and running on one of the Pis was relatively easy but there are a few gotchas.

  • ESXi 7 requires UEFI boot and so there’s a few configuration changes needed. This includes booting from an SD card into the UEFI management screen and configuring to boot from a connected USB device (more on this in a moment). The result is that the micro-SD card slot is unusable for the O/S. Great if you have some really small micro-SD cards lying around. No so great if you expected to run a datastore on the local device.
  • Installation boots from a USB device to install onto another USB device. I used two memory sticks, with the running O/S on a 4GB device. VMware recommends using an externally powered USB drive or simple memory sticks as the Pi runs hot with ESXi (CPU heatsink/fan is also recommended).
  • Datastores can be connected via iSCSI or NFS, however I couldn’t get a USB drive to be recognised and the power draw of a USB3 SSD seemed to lock the Pi up.

Aside from the above issues (and there are more caveats in the documentation), my installation is a pretty faithful reproduction of ESXi on a server or home lab. However, ESXi itself uses about 1.3GB of RAM, leaving only around 6.5GB usable. This could support perhaps 4-5 small VMs, but would be much more useful for running containerised applications and eliminating the individual VM overhead.

Proof of Concept

Remember that Flings are just proofs of concept and not always aimed at final production. VMware suggests a range of hardware options for ARM-based workloads and this is more likely where we will see the initial development of ESXi on ARM. The challenge for end users here is to determine whether the price/performance/power/cooling ratio works better on ARM than x86. Of course applications also need to be available, but it’s not hard to find ARM-compiled versions of most open source software and operating systems.

Next Steps

Over the coming weeks I’ll be trying out some application deployments with ARM and looking at the various configuration options. One scenario I’m keen to try is iSCSI boot. It looks possible to boot ESXi on ARM from shared iSCSI storage and to use shared NFS for datastores. Performance will be terrible, but the capability offers the scenario of stateless hardware and even templated deployments by cloning the iSCSI boot LUN.

The Architect’s View

We already know that ESXi runs on ARM within Project Monterey SmartNICs. With offloaded storage, networking & security, ARM for core applications could be a more compelling option, if the cost model stacks up. Although the ESXi on ARM implementation is currently a Fling, I would expect to see a fully production-baked offering released for the right combination of hardware. At this point I don’t think that will ever be the Raspberry Pi, but of course hardware advances move on rapidly, so you never know. If you can spare around $100 and some time, why not give ESXi on ARM a go and feed back your experiences?

One final thought. Using ESXi as the onboard hypervisor is great to offer a similar look and feel to traditional hardware installations. However, edge solutions don’t necessarily need the same degree of management and functionality. So I wonder if a better solution could be a custom or lightweight O/S purely for running containers. VMware could choose to do this, cutting the overhead of ESXi to just run container workloads. Now that would be interesting to see.


Copyright (c) 2007-2020 – Post #21b3 – Brookend Ltd, first published on https://www.architecting.it/blog, do not reproduce without permission.