Storage Predictions for 2023 and Beyond (Part IV - CNS)

This is the fourth post in a series looking at predictions for the storage industry in 2023. Previous posts:

Container-native storage (CNS) provides a way to integrate persistent storage within a Kubernetes cluster. With vendor acquisitions and increasing use of Kubernetes for structured databases, what’s next for CNS?

Background

The use of containerisation, and in particular Kubernetes, has grown significantly during the start of this decade. It’s incredible to see how quickly this technology has evolved, as Docker popularised the infrastructure framework and Kubernetes ran with it. Today, we expect persistent applications to run seamlessly within a Kubernetes cluster, and that means providing resilient storage.

The way in which persistent storage is mapped to containerised applications can present a problem. A single container application could be started (or restarted) on one of many nodes, with physical storage delivered from local NVMe drives, shared storage, or networked storage. If a container moves location (to another node), the mapping to existing NVMe storage (for example) is lost.

Shared storage could possibly manage this change (although the device mapping process represents a challenge), while networked storage gives the best flexibility at the risk of lower performance.

One solution to the problem was the now-defunct Flocker, which was superseded by the Container Storage Interface (CSI).

CNS

With many similarities to hyper-converged storage deployments, container-native storage implements a scale-out storage solution across the nodes of a Kubernetes cluster. The CNS layer provides data resiliency (through replication), application performance (by enabling the use of node-local storage) and other data management features such as snapshots and clones.

Most CNS implementations use a similar architecture, running container processes on each node that expose storage resources and consumable APIs to applications, usually through CSI.

Container-native storage is an elegant solution that solves many issues of data persistence. Physical storage is abstracted from the application into storage classes, which define characteristics such as performance and resiliency. If the physical infrastructure changes, the storage class remains the same to the application. If an application needs a different set of characteristics, simply use an alternative storage class.

Side Note: Interesting fact, the concept of storage classes was invented as part of System Managed Storage for the IBM mainframe in 1989. It’s not a new concept!

CNS can scale up quickly as new nodes are added to a cluster. This process can happen in seconds or minutes, compared to adding a new node that uses shared storage (such as a SAN), which could take many minutes or hours to provision.

Landscape

We mentioned that acquisitions have already taken place in the CNS market. Pure Storage acquired Portworx in September 2020, while DataCore acquired MayaData (the original developer of OpenEBS) in November 2021. For more information on the CNS marketplace and a review of vendors, you can buy and download our Market Perspective eBook. We also have a free performance benchmarking guide for CNS (free download here) and two posts that cover Kubernetes performance benchmarking.

These reports highlight the leaders in this market, where application performance is a key requirement.

Buy Report Now

We also worked with Ondat to develop a next-generation benchmarking report that looks at containerised databases, which is available for download here (registration required) with our accompanying blog here.

Predictions

So, where is CNS headed in the coming decade?

Maturity – CNS solutions have matured significantly in the last two years since we discussed predictions. Previously we suggested that data protection and new media adoption were areas of opportunity. We still believe there is a way to go (across the landscape) to deliver efficient data protection. Some vendors have implemented solutions into existing products (Portworx with PX-Backup, Longhorn with snapshots). Others rely on 3^rd party plugins and associated products. The protection of Kubernetes clusters is an in-depth discussion for another day.
Data Mobility – this area is improving. We’d like to see more features that make it possible to reuse data already stored in snapshots without having to recreate an entire duplicate cluster. This might mean enhancements to CSI, as well as vendor-enhanced solutions. One reason for this requirement is the deployment of CNS into the public cloud.
Cloud Adoption – we will see further adoption of CNS solutions in the public cloud as virtual instances continue to offer local storage (NVMe devices). These implementations need to be “cloud-aware” and capable of using cloud storage efficiently. It may seem counter-intuitive to build distributed storage in the public cloud that already has cloud-native solutions, but there are benefits in abstracting the storage layer, especially across availability zones and regions.
Observability – We hope to see increased observability capabilities introduced into CNS. As the use of Kubernetes for persistent data applications continues to grow, we will see deployments become ever more complex and need the ability for detailed examination. Cluster optimisation solutions (such as StormForge, for example) need to work in conjunction with infrastructure applications in a cluster like CNS.
Integration – One area we hope to see some traction in is the integration between CNS and traditional storage. SANs have a long legacy in the data centre and are unlikely to be superseded anytime soon. In the meantime, CNS solutions and shared storage could work together more effectively, perhaps to offload capabilities that already exist at the appliance layer, such as encryption and deduplication. CNS solutions should be capable of scaling physical storage on demand, including the public cloud. We know that Portworx already offers this feature for FlashArray.

The Architect’s View®

Container-native storage is here to stay but will morph in the coming years to be an abstraction and features layer for physical storage, including appliances. Pure Storage has clearly made that commitment with the Portworx acquisition, while DataCore has signalled its intention with the MayaData acquisition. Robin Systems has disappeared into Rakuten to serve 5G requirements, leaving Ondat as the remaining commercial start-up in this market. It seems logical that Ondat will also be acquired at some point.

An alternative view is that the block-based CNS market lives a few years longer, to eventually be replaced by file or object storage connectivity. I guess we will need to wait and see.