Storage Predictions for 2024 and Beyond (Part IV - CNS)

This is the fourth post of a multi-part series looking at technology infrastructure predictions for 2024 and beyond. It builds on discussions from previous posts to predict what we might expect in the container-native storage segment from both a short and long-term perspective.

Background

Two or three years ago, container-native storage or CNS, looked to be a promising new area of enterprise data storage. The concept is relatively simple. In a world of Kubernetes clusters, some technique is needed to provide persistent storage to containers that themselves could be long or short-lived. Storage needs to meet the demands of rapid scale-up and scale-down deployments while also addressing all the expected features of modern data storage, such as resiliency, availability, and performance.

CNS runs as a set of containers within a Kubernetes cluster. It takes advantage of the resilient nature of Kubernetes while operating as a set of locally adjacent processes to applications in the same cluster. CNS generally uses local storage resources available to each cluster node. The process of starting CNS in a cluster is relatively straightforward, depending on the solution and can be easily scripted or deployed using vendor tools.

CNS and CSI

Container-native storage has coalesced around the Container Storage Interface (CSI), a specification that defines how persistent storage should be consumed by containers. CSI implements APIs for logical volume creation, deletion, and data protection. Storage can be provided through block and file protocols. In September 2022, the Container Object Storage Interface (COSI) was introduced as an alpha feature of Kubernetes 1.25.

The aim of COSI was to bring object storage to Kubernetes in the same way CSI introduces storage volumes. The solution was put forward by developers at MinIO, an open-source object storage platform. Since the first alpha release was announced, it seems little has been done to progress the standard based on the information available on the project website.

Landscape

In our July 2022 report “Essential Features of Container-Native Storage”, we reviewed six vendors, three of which were commercial solutions and three released using an open-source licence.

Of the commercial solutions, Portworx (acquired by Pure Storage) has expanded to add data services and data protection to the platform. Portworx Enterprise is now at release 3.1. Ondat (formerly StorageOS) was acquired by Akamai Technologies in May 2023 to be used for the Akamai Connected Cloud and is no longer sold as a separate product. Robin Systems (robin.io) was acquired by Rakuten Symphony in February 2022. The storage solution from the company (Robin CNS) appears to now be used within the Rakuten platform and as a standalone solution under the brand name Symcloud Storage. However, a download link on the Rakuten site simply points back to the old Robin.io website, making it difficult to determine if the product continues to be developed for external customers.

Of the open-source solutions, Longhorn has now reached release 1.6, which looks to implement a new data engine and other significant enhancements. Rook/Ceph has reached release 1.13.3. OpenEBS is now at release 3.10. We will cover the whole landscape of products more fully in our next release of the Container-Native Storage report later this year.

Additions

In our last report, we highlighted Astra Data Store (NetApp), Ionir and Red Hat OpenShift Container Storage as possible future additions to the CNS list. Although the Ionir website is still active, there doesn’t appear to have been any company activity for almost two years. NetApp still promotes Astra Data Store, while Red Hat OpenShift Container Storage is an overarching term for Red Hat Ceph, Rook and NooBaa.

DataCore (which acquired OpenEBS developer MayaData) launched Perifery in April 2023, a new division to address edge requirements. Bolt, the repackaging of OpenEBS, is now sold under the Perifery brand as OpenEBS PRO.

Predictions

How will container-native storage evolve over the next few years?

Maturity – CNS solutions have been quick to evolve but have now slowed to a reasonable level of maturity. Where the aim of the open-source solutions was to quickly develop viable products, work in recent years has improved performance with the rewrite of data engines as an example. In the future, we see a constant rate of product improvements and enhancements, with few or no new “blockbuster” features.
Cloud Competition – while CNS solutions have cloud awareness, new storage software solutions are delivering a “SAN in the Cloud” experience that can be exported to a Kubernetes cluster. This model of deployment represents a challenge for CNS, as public cloud environments increase the performance capabilities of individual virtual instances (and offer fast bare-metal solutions). The separation of storage from a Kubernetes cluster, in a SAN-like experience, will mirror what exists today in the public cloud for managed clusters.

The Architect’s View®

Looking back at previous predictions, the promise that may have existed for CNS hasn’t materialised in the way we thought it might. Vendors have been acquired, and feature sets have settled on improving resiliency and reliability. As more Kubernetes workloads move to the public cloud and use managed Kubernetes distributions, the use and relevance of CNS will diminish. For vendors like Pure Storage, Portworx represents the ability to abstract storage and offer CNS as another protocol for its existing hardware products rather than just as a software solution. Think of it as the platform for on-premises, compared to the acquisition of Robin for the Rakuten Symphony platform and Ondat for Akamai Connected Cloud.

This leaves the market for open-source solutions to be used in vanilla or self-managed Kubernetes deployments.

CNS still has a future, but perhaps with less of a critical role in providing persistent Kubernetes storage compared to a few years ago. Going forward, we will consolidate CNS as part of our SDS coverage rather than as a dedicated storage segment.