StorONE S1: AFAn - Architecting IT

StorONE has introduced the S1:AFAn or All-Flash Array.next which uses Intel Optane and QLC media. In this article, we look at the technology in a bit more detail to see how new media is driving new architectures like AFAn and the efficiencies they deliver.

Background

Intel Optane is a new persistent memory technology that is available in two form factors, either as a DIMM or SSD. The SSD option provides low latency and high performance with high endurance and capacities up to 1.5TB per device.

QLC NAND is a persistent memory technology in an SSD form factor providing low latency and high performance. Endurance is not as high as Optane, so QLC media works better for read-biased workloads.

You can find out more about persistent memory, SSDs and new technologies like NVMe in these suggested articles.

All-flash Arrays

All-flash storage arrays have been taking an increasing part of the storage platform market since their introduction 8-10 years ago. All-flash solutions dispense with hard drives and place all data on flash media. The result should be predictable performance with excellent throughput and low latency.

Flash arrays are more expensive relative to hybrid and HDD-based systems. Performance costs more to implement than capacity at both the media and systems level. So, customers have to accept that all-flash solutions attract a price premium. Even today, SSDs are around ten times more expensive than large capacity HDDs (although TCO has an impact on this ratio).

Naturally, the storage industry has attempted to resolve the cost issue through data optimisation techniques (compression and deduplication) and through the use of cheaper, higher capacity media that includes TLC and QLC NAND. Unfortunately, the overhead of managing multiple bits per cell results in increased latency for TLC and QLC drives, which could approach many hundreds of microseconds.

So even with all-flash platforms, we’re back to a scenario where systems need to be designed with multiple media to get the best price/performance/capacity ratios.

AFAn

StorONE has developed a range of storage solutions that include hybrid and all-flash offerings. The latest platform, dubbed “All-Flash Array.next” uses a combination of Intel QLC and Optane media to address a balance between performance, capacity and cost.

The architecture of AFAn operates slightly differently to traditional storage arrays and exploits the characteristics of the two media types in use. A typical system will consist mostly of QLC NAND, supplemented by a smaller amount of Intel Optane. This ratio is usually three 750GB Optane drives as a starting point, providing around 1.4TB of usable Optane capacity.

The QLC and Optane media operate as tiers of storage, rather than using Optane as a cache. This configuration means data exists only once in either the QLC or Optane layers. There’s no need for cache management or other techniques to improve performance and throughput (see this recent article on Caching vs Tiering).

When processing write I/O, all data is initially written directly to the Optane layer – there is no intermediate caching in DRAM. Read I/O is subsequently delivered from the Optane tier until that tier reaches a high watermark of utilisation. This process ensures that the best level of performance is given to the active working set of data.

Once Optane occupancy has reached a pre-determined threshold, the most inactive data transitions down to QLC from where it is available for future read operations. The process of cascading data down to QLC ensures that the profile of writes to QLC is sequential and reduces write amplification as much as possible. If data on the QLC tier needs to be updated, then the updates are made back into Optane, without the need to “uptier” the data by moving it from QLC to Optane.

Endurance

The design of AFAn works because:

Optane has a high endurance compared to NAND flash and is well-suited to active write I/O.
QLC media provides cost-effective, scalable capacity with good read I/O latency and throughput.
The tiering process of moving data from Optane to QLC optimises for the best write characteristics of the QLC media, ensuring that the effects of media wearing are minimised.

The most obvious “what if” scenario with this design is the situation where there is more active data than the Optane tier can handle. In this situation, the Optane layer can be extended to meet the right balance between the more expensive and cost-effective storage.

The second question to ask is how does performance change if data has to be read from QLC? With the right balance of QLC and Optane, data will only be read from QLC if it is inactive. After that, if the data is updated, it will get placed back into the Optane layer. This design makes it essential to get the right balance between Optane and QLC to avoid “thrashing” or the undesirable moving of data continually between tiers.

You can learn more about AFAn in a recent Storage Unpacked podcast.

Compromises

There are two schools of thought when it comes to storage system design, that is to tier or not. Some vendors prefer to develop platforms with a single tier, using the justification of delivering guaranteed performance for all data. There’s nothing wrong with this approach, as long as the customer knows the cost of I/O will be the same across both active and inactive data.

The alternative is to tier data across multiple media types. This approach makes a compromise that data may be on the wrong tier for the level of activity required. Most tiering solutions are reactive, so inevitably result in data sitting in the wrong place. The cost of the compromise is dictated iby how quickly tiers can be rebalanced. In the StorONE architecture, tiering is dynamic and implicit based on write I/O activity. The result is better performance than could be achieved using traditional tiering methods.

The Architect’s View

In a previous post, I talked about using caching and tiering techniques more effectively with new media. The specific characteristics of Intel Optane make the technology suitable for more than just a persistent read cache. The use of Optane as a single static tier also has limited benefits, due to the relatively low scalability of today’s product offerings.

Using Optane efficiently demands new solutions. While vendors can use Optane as a cache or tier, the capabilities of the media won’t be fully exploited, and that’s a competitive disadvantage for storage systems vendors that need to build efficient solutions for a competitive market.

StorONE AFAn is one example of new thinking that exploits the specific characteristics of new media. Ultimately though, customers don’t focus on the minutiae of how technology works. They’re driven by cost, performance and reliability. As always, vendors that deliver solutions in the most efficient way will be those winning the business of their customers.