Alternatives to VMware: Proxmox VE

This is one of a series of posts looking at alternatives to VMware following the completion of the acquisition of the company by Broadcom. In this post, we review Proxmox VE, an open-source virtualisation platform.

Background

Proxmox VE is a virtualisation platform based on Debian Linux using the KVM and QEMU hypervisor technologies. The software is developed and published under the GNU Affero General Public Licence v3 (AGPLv3) by Proxmox Server Solutions GmbH, based in Austria.

Proxmox (the company) was founded in 2005 by brothers Martin and Dietmar Maurer, both Linux developers. The first product released by the company was Proxmox Mail Gateway, an anti-spam and anti-virus filter. The first version of Proxmox VE was released in 2008, initially developed as a GUI for OpenVZ (a container platform). Proxmox Server Solutions GmbH operates independently and claims to be profitable.

Proxmox VE

Proxmox VE is built on Debian Linux using KVM and QEMU device emulation. Successive versions of the software have followed the releases of Debian, from Debian 5 (Lenny) onwards to the current version 8 release, which is based on Debian 12 (Bookworm). Typically, new versions are released every 2-3 years and supported for approximately three years from launch.

As an open-source platform, Proxmox VE is distributed under the GNU AGPLv3. This means the software is free to use but is copyleft protected. Derivative works must also be made available under the same licence. However, commercial support is provided by Proxmox Server Solutions (the details of which we will discuss in a moment).

Installation

We’ll cover a detailed hands-on review of Proxmox VE in a separate post. However, for now, we will provide some high-level details.

The Proxmox VE software is installed as a bare-metal ISO, or can be deployed as Linux software packages on an existing Debian 12 installation. The ISO deployment is the preferred option. We have been running Proxmox VE in the Architecting IT lab for about five months across a mix of form factors, including standard Dell 1U servers with shared storage and on Intel NUC.

Using our seven comparison points to VMware, how does Proxmox VE shape up?

Hypervisor

In common with other virtualisation platforms, Proxmox VE uses KVM and QEMU built into Linux as standard. Both features are solid technologies with over 15 years of development history. KVM uses Intel VT-x or AMD-V virtualisation extensions with full support for VirtIO drivers.

The Proxmox VE GUI exposes a much greater range of configuration options compared to (for example) Scale Computing’s SC//HyperCore. The administrator has control over BIOS settings, chipset emulation (for legacy or older O/S support), and virtual I/O and storage devices. Storage settings for asynchronous I/O need to be reviewed, as the default io_uring setting doesn’t support shared storage.

Platform Support

As Proxmox VE is essentially a set of software packages built to run on Debian Linux, the solution should run on most hardware configurations. We have Proxmox VE running in the lab on two Dell PowerEdge R640 servers, with another single setup running on Intel NUC11 hardware.

Storage and networking features are dependent on Debian support. We’ve configured iSCSI shared storage on the Dell cluster, including multi-pathing. Although we have it available in the lab, we haven’t tested Fibre Channel connectivity, but we believe it should be possible to implement with the correct Debian drivers for the FC HBAs.

The Proxmox VE architecture provides the flexibility to use components that would possibly be outside the scope of VMware. However, as always, we would recommend validating supported configurations with the vendor.

Ecosystem

Proxmox VE is a fully integrated platform running on Debian. The software implements a GUI environment which runs in multi-master mode across all nodes (servers) in a cluster. This means there is no separate management software or tools compared to (for example) VMware with vCenter.

The multi-master design is implemented using a cluster file system (pmxcfs), which is distributed across all nodes in real time using corosync. Configuration data for the cluster would, therefore, be protected in the event of any node being lost.

The infrastructure can be driven by GUI, CLI or API. There is a complete set of RBAC controls that enable the administrator to delegate permissions over specific features. Pooling provides the capability to group storage and virtual instance resources. However, Proxmox VE doesn’t implement a fully abstracted software-defined data centre (more on that later when we discuss networking).

One other aspect to consider is the scalability of Proxmox VE compared to VMware. With an integrated management GUI, Proxmox VE doesn’t need a separate management plane. However, with large environments comprising many clusters, visualisation of an estate would be an issue. This problem represents an opportunity for platform vendors that provide the next layer up in terms of management, or for Proxmox to offer additional tools to meet this rerquirement.

Commercials

Proxmox Server Solutions GmbH offers paid support for Proxmox VE with four levels of subscription. Each is based on CPU socket count, requiring each socket to be licensed. There is no charge based on the cores running on each processor or the memory deployed in a server. Therefore, it makes sense to deploy with the largest hardware configuration available. The four subscription levels are as follows:

Community – access to the Proxmox enterprise repository for software upgrades.
Basic – Community entitlements plus portal-based support, three support tickets and one business day response time.
Standard – Basic entitlements, upgraded to ten support tickets, 4-hour response time and remote support via SSH.
Premium – Standard entitlements, upgraded to unlimited support tickets, 2-hour response time.

Large enterprise customers may want further support in addition to these options. Proxmox Server Solutions GmbH has a range of partners that provide support, training, and hosting (more details here), so it should be possible to find a solution provider that can complement the standard subscription offerings.

Deployment Model

We’ve highlighted some of the deployment options already. Customers can run on-premises, work with co-location partners or run in environments such as Equinix. We’ve looked at this option (including a test deployment), which isn’t directly supported but will work. Although Proxmox Servers Solutions GmbH doesn’t quote any support options for public cloud environments, we believe the solution will work on certain bare-metal instances and will be evaluating this separately, with a write-up as a separate blog post.

Features

Proxmox VE supports a wide range of storage options that include both local and shared storage types. We found the storage descriptions and choices a little quirky, and certainly, they need review with respect to thin provisioning and data efficiency. Storage support also includes a Ceph deployment, which is distributed across all nodes in a cluster. This option provides an HCI configuration that doesn’t require external shared storage (although this option is also available).

Networking includes the ability to implement limited software-defined networking with zones, virtual networks, and subnets. Proxmox VE also implements firewall capabilities per VM and per node. We will cover more on the storage and networking capabilities in a separate post.

High availability is supported through node-to-node replication, while VMs can be protected through an integrated backup capability based on snapshots.

Migration and Mobility

As Proxmox VE is based on KVM and QEMU, virtual machines are stored in the qcow2 format by default. It is possible to import virtual instances from VMware vSphere (in vmdk format). The migration and mobility capabilities of Proxmox VE are limited and require manual command line work. This aspect is probably the weakest part of the solution.

The Architect’s View®

There are significantly more aspects to the Proxmox VE platform than we can cover in a single blog post. However, our experience running the software across multiple hardware platforms over the last five months has provided enough data to have an informed opinion on the solution compared to VMware vSphere.

As a solution for SMB or mid-range deployments, Proxmox VE is a platform worthy of consideration. The GUI and general management features are well thought out (although storage configurations were initially a little confusing).

Proxmox VE offers a broader range of configuration options compared to (for example) Scale Computing, which is both a good and bad thing. Too many “nerd knobs” can introduce the risk of misconfigurations or other issues (such as performance problems or even data loss). However, existing VMware customers with mature virtualisation skills will be comfortable with the degree of flexibility Proxmox VE offers.

Proxmox VE is a definite candidate as a VMware replacement. As ever, requirements and use cases need to be considered. However, as a technical solution, Proxmox VE should meet most server virtualisation needs.