HDD Capacity Threshold Reaches 20TB

Less than 12 months ago, we were talking about the release of 15TB drives from Western Digital (WDC). Today 18TB (CMR) and 20TB (SMR) drives are about to be released for sampling to customers. The relentless momentum of the HDD industry is pushing us closer to the mythical 100TB HDD. Will flash ever catch up?

Ultrastar

The latest WDC announcements extend the Ultrastar DC HC5xx series to 18TB with the DC HC550 in a standard 3.5″ form factor using 9 platters. There are now no single-digit drives in this family – capacities start at 10TB with 12TB, 14TB and now 18TB models. No performance figures have been made available (yet), but looking across the range, we can expect a slight increase in throughput compared to the 14TB generation.

The HC550 is a CMR (conventional magnetic recording) drive whereas the HC650 is SMR (shingled magnetic recording). SMR drives use a magnetic bit overlapping technique which means new data has to be written to zones of multiple tracks, slowing down the overall write speed. Zone management can be done either on the drive or by the host.

HC650 is expected to arrive in 2020. The current drives in the same family (HC620) have capacities of 14TB and 15TB.

Capacity, Performance & Cost

The classic triumvirate of metrics for choosing storage media is capacity, performance and cost. Hard drives offer capacity, acceptable throughput performance (not good latency or random I/O) at a low $/GB cost. Flash drives offer high capacity, great throughput but at a high cost.

As flash has evolved, technologies like TLC and QLC trade off some performance for cost/capacity. The performance gap is starting to be filled by new products like XL-FLASH and Z-NAND.

In the HDD market, we’re moving almost exclusively to capacity drives as SSDs replace the high end, because the cost/performance profile works compared to HDDs (and vendors have already dropped out of this market).

Multimedia

It’s easy to think that HDDs will eventually be phased out by flash. However, the cost profile between the two media types is still huge. In retail terms, a 14TB HDD is about the same price as a 4TB SSD. This difference may seem small (“only” 3x) but look at the cost for building a multi-petabyte archive and SSDs are still unsustainable.

Hybrid Rules

As I discussed with Erik Kaulberg on a recent Storage Unpacked episode (below), simple tiering isn’t a solution to exploiting multiple media options. Instead we need more intelligent systems to move data (or parts of data) around. Dare I say it, perhaps even using ML/AI techniques to optimise large archives with a mix of HDD and flash.

The Architect’s View

The idea of mixed media may not seem new, but the capacities we’re seeing in the latest products (and the use of SMR) means we need a higher degree of intelligence to get the best out of our devices. As an illustration, at best, a 20TB drive could be read in its entirety in just over 21 hours. This sounds OK, but with any I/O interruption from write activity, the elapsed time will increase hugely.

The original question here was “would SSDs ever catch up?”. Of course, in capacity terms they already have, exceeding HDDs by a factor of two, three, four or even five, depending on which product comparison you make. But this isn’t purely a capacity discussion. The cost cap is still significant and really stands out at scale.

We will continue to need storage systems that can make use of these devices, because the complexity of getting it right (optimising for cost/performance/capacity) still remains hard. The object storage/capacity market seems to have been quiet for a while. Perhaps we need a review to see how vendors are coping with super-high capacity devices.