Why is HPE 3PAR not moving to NVMe SSDs (yet)?

During our coffee talks at HPE Discover 2018, statements were made that claimed HPE 3PAR storage platforms wouldn’t benefit much from going to NVMe at the back end. Instead, the statement of direction is to use storage-class memory as a platform accelerator. I’ve been trying to understand why HPE isn’t making the move at this point and I’m seeing only contradicting messages. What’s going on?

HPE 3PAR Architecture

Let’s talk about architectures. The 3PAR storage platform has been one of the most successful storage appliances in the industry. The first products shipped in 2002, with HPE acquiring the company some eight years later in September 2010. The architecture is highly distributed, with data spread across multiple drives that are in turn divided into 1GB “chunklets”. This allows the system to build dynamic RAID groups from chunklets, which in turn become common provisioning groups (CPGs) and virtual volumes. Over time, HPE has added new features like compression, de-duplication and advanced caching.

Richard Arnold has some good background blog posts available here that cover the design if you want more information.

I’ve written many papers (example below) for HPE that cover the new feature announcements. At each stage of development I’ve been pleasantly surprised at how the architecture has been extended. However, at the same time, it’s clear to see that each addition is a reworking of the existing design, changing or extending metadata structures, for example.

HPE 3PAR Adaptive Data Reduction (white paper)

Incremental Improvement

As a (former) enterprise storage user, I can say that the constant improvement in performance and release of new features is a positive move. As applications become more demanding in their storage needs, successive generations of 3PAR have delivered, while maintaining a degree of stability that enterprises need. Businesses that run thousands of applications are rarely in a position where every single system needs to move to ultra-fast storage. This means, in general, the incremental approach is how most IT organisations move forward. There will be pockets of demand for ultra-high performance and that’s also to be expected. Those applications can be dealt with as separate use-cases.

3PAR 2020

So what’s the direction for 3PAR as we move into the third decade of evolution? The addition of SCM into the platform was announced and displayed in preview at the end of 2016 (two years ago) and finally started shipping in December 2018. In a Coffee Talk at HPE Discover 2018 in Madrid, Vish Mulchand indicated that SCM is being primarily used as a read cache, both reducing average latencies from 350µs to around 200µs, as well as shortening the “long tail” of I/O. Improvements have also been seen in traditional workloads such as Oracle. You can listen to some of the details in this recording from the event.

HPE has also been working to improve platform reliability, with further developments in InfoSight. We recently chatted with Ivan Iannaccone about Performance Insights (Storage Unpacked episode #77), which provides ML/AI performance management for 3PAR platforms.

3PAR, Nimble and NVMe

What about replacing the existing SAS back-end with NVMe? In an HPE Storage Tech Day event in October 2018, Phill Gilbert (3PAR Product Manager) indicated that simply replacing SAS with NVMe had been done in the lab and didn’t provide much benefit. Vish Mulchand expands on this in the Coffee Talk above (shortcut to the start at 17:42 here). He indicates that HPE views NVMe as being useful only when it can be exploited end-to-end in the storage architecture. Omer Asad extended the analogy to Nimble, highlighting that as a dual-controller architecture, Nimble is constrained at the front end, so going full NVMe would not give significant benefits. Introduction of NVMe SSDs on Nimble will be in line with the developments from vendors that allow introduction of the technology without significant hardware developments.

Nimble Strategy

Picture of a Toshiba U.3 Drive with BiCS Flash — Toshiba U.3 Drive with BiCS Flash

The Nimble stance surprises me somewhat, because commodity hardware is already available today to deploy NVMe in servers. In fact, with U.3, system builders can deploy NVMe, SAS and SATA in the same server and interchange drives. This technology was being showcased on the floor at HPE Discover. However, the dual-controller architecture is more of an issue. This is difficult (or impossible) to resolve and so it may mean that Nimble will only adopt NVMe when the cost profile is insignificant over SAS. Where will the rest of the market be by then?

3PAR Strategy

Surely with a highly distributed architecture like 3PAR, moving to NVMe should see significant performance improvements? However, the implication is that this isn’t the case. I wonder then if there are other issues involved here, like internal conflicts in the way metadata is managed that cause excessive locking and prevent highly parallel operation. Remember that Optane technology highlighted weaknesses in the SAS/SATA protocols, so perhaps the 3PAR O/S needs some significant rewrites to fully benefit from the move to NVMe. Echoing what Omar stated about Nimble, then we should remember that the hardware solutions on the market today provide perhaps 24 NVMe U.2 SSDs in a 2U chassis. Extending past this requires some hardware design. So bringing NVMe into the 3PAR platform is probably both a significant hardware and software change.

Note – this is pure speculation on my part and I have no direct insight, other than my industry experience.

The Architect’s View

Technology companies have to continually make decisions about how to introduce new features and functionality to their products. The variables involved and subsequent decision paths can become complex in terms of cost, benefits and market demand. I can only second-guess HPE’s decision making. However, what’s clear is that the move to NVMe requires significant engineering change to implement, if vendors want to get the full value from the transition. The alternative of course, is to have either built in the NVMe transition in from day one, or to develop a new platform that is built specifically to exploit the capabilities of NVMe.

Customers need to ask; “do I need this new feature, or can performance improvements be gained in other ways?”. While at the same time, asking “is pain and impact of moving to another vendor worth the gain”? These are questions that only the customer can answer themselves as only they know the variables that affect their business.