NetApp Delivers Cloud-Native Storage to AWS with FSx for NetApp ONTAP

AWS has announced the general availability of Amazon FSx for NetApp ONTAP, a native implementation of the ONTAP file system within AWS. This milestone represents the completion of a goal to deliver native NetApp file services into the three major public cloud service provider platforms.

Background

Storage, they say, “is hard”. This truism has pervaded the storage industry for decades and represents one of the greatest challenges when building private or public cloud infrastructure. By comparison, networking (and to a degree compute) is more like “pass the parcel” (or hot potato elsewhere). In the network world, if you drop the ball, it’s not a big deal – just wait for the retry that’s built into the protocol. With storage, losing data results in a bad day for everyone.

So, it should come as no surprise that persistent storage has generally been the last piece of the puzzle built with new architectures. EC2 (AWS Elastic Compute Cloud) just passed the 15-year anniversary, but persistent storage in the form of EBS took another two years to arrive after the initial launch in 2006. OpenShift developed with the same model (ephemeral then persistent storage), while Docker and Kubernetes also went through the same evolution.

File Services

Looking at other storage solutions, we can see that file-based storage in the public cloud has struggled to compete with on-premises offerings. AWS introduced EFS (Elastic File System) in June 2016 (with previews before that, but ten years after the launch of EC2). The solution was somewhat underwhelming and lacked the features, functionality and performance expected of enterprise storage solutions.

At the time, I was guest-writing blog posts for Zadara Storage, a start-up that offered co-located storage services adjacent to the major public cloud providers. One of my articles compared EFS and traditional cloud storage, in this instance, what I considered the “gold standard” of enterprise storage, NetApp ONTAP.

Unfortunately, AWS were less than impressed, accusing me of cherry-picking features to make EFS look bad. The post was taken down, as Zadara was an AWS partner and didn’t want to rock the boat. That event pretty much ended my guest blogging on their website.

Features & Functionality

Although I don’t have a copy of the original article, the premise of the discussion was to show that EFS had a long journey to meet the existing capabilities of on-premises file storage. I expect the first versions of EFS simply used an NFS server on Linux. In comparison, ONTAP has had decades of development across thousands of demanding customers, with petabytes of deployment.

As we said at the top of this article, storage is hard, takes many years to develop and reach a level of maturity acceptable to enterprise customers. This challenge has represented an opportunity for the likes of NetApp and other start-ups. The public cloud providers have chosen to either partner or acquire file storage solutions rather than persist in developing in-house technology.

There are some good reasons for taking this approach.

Enterprise file servers are mature and reliable platforms that are trusted by customers.
Most modern file server solutions are available as virtual instances or virtual appliances, translating well to the cloud model.
The internal architecture of file server software uses proprietary technology for features such as replication. This makes it impossible to build a generic (and efficient) “replication engine” for file storage.

This last point is possibly the most important in this discussion.

While many other features and functionality can be obfuscated to the level where any solution could offer those capabilities, moving data efficiently between deployments is a proprietary feature that is generally heavily embedded into the platform architecture.

This design constraint makes it hard to move file data in and out of the public cloud without resorting to basic copying and risking data inconsistencies.

Partner or Acquire

As we’ve said already, the strategy so far has been for public cloud platforms to partner or acquire.

Google Cloud acquired Elastifile and made CFS generally available on the platform in December 2018. NetApp Cloud Volumes were implemented on GCP in May 2018.
Microsoft Azure implemented NetApp ONTAP (as Azure NetApp Files) in 2017, with general availability from May 2019.
AWS built file solutions based on Window File Server and Lustre using the FSx marketing tag. ONTAP has been available in AWS for some time via the marketplace and implemented as a virtual instance.

Until now, the NetApp implementations on GCP and Azure are the only ones to be “cloud-native”, meaning they directly integrate with other cloud services like billing, networking, and security.

Amazon FSx for NetApp ONTAP

On 2^nd September 2021, AWS announced the general availability of Amazon FSx for NetApp ONTAP. This is a fully managed implementation of ONTAP and makes NetApp the only vendor to have cloud-native integration of its storage platform in the public cloud.

The implications of this announcement shouldn’t be understated. AWS customers can now gain access to NetApp storage features with no minimum fees or set-up charges. FSx for NetApp ONTAP supports Linux, Windows and macOS clients with NFS, SMB and iSCSI protocols. Customers can replicate data between regions or from on-premises ONTAP into AWS using SnapMirror. This capability extends the concept of the hybrid multi-cloud. Note however, that file systems cannot be exposed directly to the public internet, which may represent challenges moving data directly between public clouds.

The Architect’s View™

NetApp has finally completed the native deployment of ONTAP into the three major public cloud platforms used widely across Europe and North America. The process has been a long journey, as NetApp continues its re-invention into a cloud company.

The implementation of native ONTAP by AWS is a tacit acceptance that EFS didn’t meet enterprise customer needs, especially around data mobility. More interesting is the acceptance that hybrid cloud is a deployment model that’s here to stay.

Where does NetApp go next in this cloud storage journey? The most obvious development would be to link clouds with native replication, making data available seamlessly across clouds with minimal overhead and cost. This step may not be acceptable to Google, Microsoft or Amazon. There’s also some exciting scope for extending the ONTAP file system to include native archival into S3 and other object stores, as well as industry vertical requirements. We could also see some of the automation features of EFS being back-ported to ONTAP, which could be useful for on-premises customers.

NetApp’s cloud strategy continues to be one of the most interesting in the industry and one that seems to have no appearance of slowing down, which should make for an interesting 2022.