Build or Buy - A Tale of Two All-Flash Strategies

Update: Andy Warfield of Coho Data has written a post on the FlashBlade architecture and decision choices to build/buy. Find it here (link).

Ever since flash was first added to traditional storage arrays back in 2008, there has been vigorous debate on the right approach to storage architectures for all-flash systems. There are a number of opposing camps, each with their own perception of the right way forward. Some vendors looked to use existing hardware and amend it to work with all-flash devices. This approach has been taken by 3PAR and NetApp (with AFF, All-Flash FAS).

Startups have developed “built from the ground up” solutions that are designed specially around the foibles of solid state disks (SSDs) and look to deliver consistent performance and low latency by second-guessing how the SSD controller manages the NAND chips embedded in the device. Examples of these models are Kaminario, XtremIO and obviously the most recent hybrid to all-flash converts (Tintri and Nimble). Finally we have the fully custom solutions, like those from Violin Memory, HDS (in the VSP) and now FlashBlade from Pure Storage.

Building SSDs are Hard – True or False?

For the longest time, SSD manufacturers have been telling us how hard it is to build a solid state disk device and efficiently manage all that pesky NAND inside. As an example, I recommend going back and looking at SanDisk’s presentation on Guardian Technology at Storage Field Day 5, almost 2 years ago in April 2014. Guardian is the underlying IP that manages the lifetime of the NAND in SanDisk’s products and has some pretty intelligent algorithms to detect and optimise the wear of NAND cells. As we should all know, NAND memory degrades slightly every time it is written to, so we want to optimise/reduce the number of physical writes as much as possible.

So far in the market we’ve seen only a few companies develop bespoke all-flash systems based on NAND, most notably Violin Memory (with their VIMM technology), HDS (with FMDs, Flash Module Devices) and now Pure Storage with the NAND inside FlashBlade. So developing custom NAND flash hardware should be hard – or is it? The implications from discussions with Pure Storage at their recent //Accelerate 2016 event implies that the process isn’t as hard as you would think. FlashBlade implements a custom NAND design in a scale-out architecture based on multiple compute/storage blades in a 4U appliance. The persistent memory component of each blade accommodates either 8TB or 52TB of flash, with integrated NVRAM (9GB or 3GB) and a custom FPGA and dual ARM processor cores (to manage the NAND).

Pure Route

Why have Pure taken this approach? I was fortunate to be able to spend time with the best minds in Pure Storage, including John Colgrove, John Hayes and Brian Pawlowski, a number of product engineers and a room full of fellow bloggers. In our discussion (which lasted a couple of hours) we dug into some of the design strategies the company has chosen in bringing FlashBlade to market. Firstly, the design of FlashBlade was done to reduce costs. Building a custom NAND device allowed Pure to eliminate other expensive components like interconnects, SAS expanders and so on.

The ARM cores and FPGA allows things to be programmable for the future, implying the ability to use future NAND and solid state products when they are available. Development of FlashBlade was done as a separate team within the company – a startup within a startup – in order to continue to get the benefits of being fully focused on one platform. New people were brought in, including experts with 20 years knowledge of signal processing to work on the error correction algorithms.

So custom all-flash designs are not necessarily a bad thing. In fact, building a custom “SSD” is much easier than developing a hard disk drive from scratch. For companies like Pure, the profile of I/O written and read from each blade is well understood and so the algorithms managing the NAND don’t have to be capable of dealing with every possible workload scenario, as is the case with generic SSDs. This implies the FPGA logic can be much simpler and easier to develop.

Reinforcement and Contradiction

By coincidence, Pure’s //Accelerate 2016 conference was immediately followed by another Storage Field Day, with presentations from NetApp and Violin Memory.

Violin have been developing custom solutions for many years, starting with the 1010 DRAM array launched back in 2007. During their presentation at SFD9, Violin founder Jon Bennett showed us their “museum” copy of the 1010, based entirely on DRAM, with no flash and no processors. The current technology uses VIMMs (Violin Intelligent Memory Module), a PCIe based memory card that combines sixteen Toshiba flash NAND chips into an intelligent AIC – 64 of which make up a Violin appliance. Previously I’ve been critical of the custom approach Violin have taken, mainly because other startups have managed to innovate faster than Violin appear to have done. I personally believe that position to hold true, as Violin still focus on very high performance and low latency as their core values. Other vendors are moving to use cheaper and more cost effective TLC technologies that don’t appear to be on Violin’s roadmap.

Netapp Route

NetApp recently acquired SolidFire, an all-flash startup and has also been pushing the flash message with their AFF – All-Flash FAS systems. To be fair to NetApp, the company has had some kind of flash strategy for years, initially based on flash as a caching tier (PAM and Flashcache products). More recently, the strategy has evolved to have flash as a tier of storage in the FAS series and now as all-flash devices (EF and AFF). I asked Dave Hitz (NetApp founder) why the strategy had changed (you can watch the video here; my question is about 8 minutes in). He candidly pointed out that NetApp’s strategy was initially right when flash was expensive, but didn’t evolve quick enough when flash become more readily available and cost effective.

So NetApp provides the contradiction point that all-flash systems have to be bespoke. Their platforms (AFF, EF, SolidFire) all use commodity SSDs, but the architecture dictates the specific requirements to the customer. AFF accelerates traditional workloads; EF provides the super-fast no-frills low latency experience and SolidFire offers a scale-out solution with QoS and tight service-provider type management integration.

The Architect’s View

Having presentations from Pure, Violin and NetApp/SolidFire all in the same week helped to highlight the different architectural design decisions vendors are choosing to take. If you have the skills to do it, custom designs can apparently be done cost effectively, although using SSDs perhaps provides a faster route to market when you’re developing your first product. Of course focusing on the technology implementations of flash doesn’t address the needs of the application and ultimately the business.

The bigger picture is how those architectures will be used to deliver what the business needs. NetApp has acquired SolidFire to provide a broad portfolio of products to meet traditional, performance and scale-out needs. Pure Storage has brought out FlashBlade to attack the high performance file and object market. The nuances of the implementations are nice to know, but ultimately we care about features and price.

I don’t think we’ll ever get to an answer on buy or build for flash. Each of the platforms discussed here have specific benefits that will resonate with potential customers in different ways. However, I would suggest that the philosophy of these vendors tells us more about their future intentions and likely success in the market. The videos from last week are well worth watching. You can find my links to Pure //Accelerate 2016 and Storage Field Day 9 below in further reading. What’s your opinion – build or buy?

Build or Buy – A Tale of Two All-Flash Strategies

Building SSDs are Hard – True or False?

Pure Route

Reinforcement and Contradiction

Netapp Route

The Architect’s View

Further Reading