Pure Storage - Seeding the NVMe Market

I’m only just digesting the news from Pure Accelerate 2018, including some of the product updates we saw. One aspect of the announcements was the positioning of FlashArray with NVMe. Certainly a hot topic for the moment, as vendors jostle for position in the end-to-end NVMe stakes. What might not be obvious is that FlashArray has included NVMe support for nearly 3 years, providing Pure with the ability to offer migration to NVMe at no cost premium.

Native NVMe

NVMe support was introduced with the FlashArray//M in May 2015 (see this post). The architecture changed slightly in that the persistence of storage on disk shelves was moved to the controller chassis, although as pointed out to me, the NVRAM is PCIe connected and not directly associated with a specific controller. This means each controller can be stateless and access all the NVRAM in the event of a failure. It also means the controllers can easily be replaced. Older storage designs used replicated, not shared persistent memory for writes and so significantly dropped in performance with a controller failure or downtime.

The biggest change though was the introduction of flash modules for storage. Initially, these were a 2-drive caddy that could be SAS or NVMe connected. Each caddy held two 2.5″ drives. I remember thinking at the time that this seemed like a strange design because the failure of any individual drive would fail the caddy. This was less of a problem than expected. I remember asking John Colgrove (“Coz”) at the time about failure rates. He indicated they were low and in many cases were blips caused by controller firmware. In most cases, the drives weren’t faulty and simply needed reformatting.

DirectFlash

Although the caddy may have seemed like an odd design, it was there for a reason. We started to see the thinking with the introduction of FlashBlade at Pure Accelerate in 2016 (see this post). FlashBlade does away with SSDs and talks directly to the NAND using a separate custom daughter board with integrated NVRAM, FPGAs and ARM cores.

As the post explains, moving to custom NAND was always part of the plan, however, getting a foothold in the market (and some revenue) necessitated the use of SSDs in the early products.

Skip forward to April 2017 and Pure Storage announces the FlashArray//X. This new array uses DirectFlash modules that replace the previous caddies and are 100% NVMe enabled. DirectFlash removes the wrapper of the SSD and allows the Purity O/S to talk directly to the NAND. In effect, the Flash Translation Layer (FTL) that we see in a traditional drive, is moved to the O/S software.

FTL

Removing the constraints of the SSD form factor has some significant benefits. DirectFlash software can now talk to each NAND component directly, eliminating the “black box” nature of a traditional SSD that usually has direct control over garbage collection and wear levelling. SSD vendors build NAND management into their products at the drive level. At a system level, the Purity O/S can do a better (and more efficient) job at managing performance, over provisioning and wear levelling than allowing each drive to make those decisions independently.

We talked about some of the SSD management problems on Storage Unpacked episode #55 and covered some of the issues in a blog too. Hyperscalers want to have direct access to the FTL in order to implement some of the features we’re talking about here. The idea of a build or buy strategy is discussed further by me in this post. It includes further background on FlashBlade design.

Non-Disruptive

What makes this story most interesting is that the introduction of FlashArray//M in 2015 provided customers a non-disruptive upgrade path to FlashArray//X in 2018. Both //M and //X use the same chassis and support SAS and NVMe. The upgrade from //M to //X is essentially dual controller replacement, then a replacement of SAS/SATA modules to NVMe. Remember this is in place, not wiring in new controllers and moving data.

Why is this important? As a storage administrator, the biggest issues to manage in the data centre are risk and availability. Reducing the impact of risk and increasing availability by providing in-place upgrades is a key operational consideration. The classic Forklift Upgrade is a massive pain, with implications to both cost and risk.

NVMe over Fabrics

What next? For NVMe, the next opportunity is to replace existing Fibre Channel capability with NVMe over Fabrics (NVMe-oF). However, the simple view would be to assume that this simply means replacing existing SAN technology. The reality is a bit more complex than that. First, NVMe-oF can be used at the back-end of storage systems to expand capacity. Pure Storage introduced this capability with DirectFlash shelves, announced at Pure Accelerate 2018. A FlashArray//X appliance can talk directly to up to 28 DirectFlash modules in an expansion shelf using NVMe-oF (in this case 50Gb/s Ethernet and RoCE v2).

What is the point of this? First, all the storage within a system can be NVMe accessible at low latency and high throughput. This is a good thing because it continues to eliminate design bottlenecks. Second, there’s no logical reason that a shelf has to be owned by a single appliance. In fact, Pure could implement a rack-scale architecture of multiple controllers and shelves and allow any-to-any communication. Although this specific architecture is not being offered today, the implication is there within the “Data-Centric Architecture” messaging that Pure used at Pure Accelerate 2018.

The Architect’s View®

Everything we’ve talked about sounds great from a technical perspective. The roadmap to end-to-end NVMe exists and that will extend to support host-connected fabrics in the future. But what’s in it for the customer? Operationally, the most obvious benefit is simplification – being able to move to new technology and consolidate islands of storage over time into a single logical view. Financially, Pure claims to be able to offer //X systems with no price premium over //M. This represents a huge benefit to customers, plus great positioning for Pure compared to competitors. Of course, these days I’m not a customer, so I would recommend testing the water on this offering as part of any sensible RFP process.

NVMe is here and offering benefits to customers across the board. The most interesting technical advancement for me will be in seeing how NVMe-oF solutions play out, and not just those replacing SANs. As a strategy, Pure were in early with NVMe, so existing customers can choose when to take advantage of performance improvements. This early start has led Pure to be able to claim (in their recent earnings call) that more than 50% of systems shipped with all-NVMe in 2Q2019 and almost all shipped systems will be all-NVMe by the end of the year.

Pure Storage – Seeding the NVMe Market

Native NVMe

DirectFlash

FTL

Non-Disruptive

NVMe over Fabrics

The Architect’s View®

Further Reading