Solidigm introduces the “world’s highest capacity” PCIe SSD

Solidigm has announced the D5-P5336, a PCIe SSD with capacities of up to 61.44TB. The company claims this is the highest device available on the market today. Is this new SSD a simple shoo-in to increase the capacity of SSD-based storage and servers, or is there something a bit deeper here that we need to understand?

Background

Solidigm is a wholly owned entity of storage and memory maker SK Hynix. The company was formed from the acquisition of Intel’s SSD business back in 2020/2021. We last covered the Solidigm with the announcement of the P41 consumer device, which gained performance improvements from a host-based driver.

This year, Solidigm has announced the D5-P5430 and D5-P5336, new products within the D5 family of QLC NAND capacity-optimised SSDs first introduced by Intel. The “P” designator is, of course, there to represent PCIe as the interface; however, the remainder of the naming standard isn’t particularly clear.

D5-P5430

The P5430 was introduced in May 2023, with capacities up to 30.72TB in U.2, E3.S and E1.S form factors. The E1.S device only supports capacities up to 15.36TB, which is a reflection of the relative dimensions of these three form factors.

Form Factor	Length	Height	Thickness	Volume
U.2	100mm	69.85mm	15mm	104.8cm³
E3.S	112.75mm	76mm	7.5mm	64.27cm³
E1.S	111.49mm	31.5mm	5.9mm	20.72cm³

In relative terms, the E1.S P5430 offers the greatest density, but that capability is dependent on server designs and options offered by manufacturers. The P5430 delivers up to 971,000 read and 120,000 write IOPS or sequential throughput of 7000MB/s (read) and 3000MB/s (write).

#84 – Discussing New NVMe Form Factors with Jonathan Hinkle

The read/write ratios of this device are important; the IOPS ratio is (approximately) 8:1, while the throughput is 2.3:1. The P5430 has a DWPD of 0.58 or up to 32PBW (depending on model) over a 5-year warranty lifetime.

D5-P5336

Let’s look at the new P5336, a high-capacity, read-optimised PCIe SSD. This new device will support storage capacities up to 61.44TB (30.72TB at launch), using 192-layer QLC NAND (like the P5430). The supported form factors are U.2, U.3, E3.S and E1.L, with E1.L (30.72TB) available at launch, U.2 and E1.L (61.44TB) available later in 2023 and E3.S (30.72TB) available in early 2024. Endurance is rated between 0.42 (7.68TB drive) to 0.58 (61.44TB drive) or 5.9PBW to 65.2PBW, respectively.

If we examine the performance characteristics of the P5336, we start to see some differences with the P5430. Maximum IOPS are 1,005,000 (read) and 38,000 (write), with 7000MB/s read and 3300MB/s write, respectively. We believe from the product brief that these figures are for the 61.44TB devices. Compared to the P5430, the IOPS ratio is now 26.4:1, while the bandwidth ratio is about the same. From an IOPS perspective, the ratio is now skewed much more towards the read I/O capability.

QLC Challenges

As we can see, the D5-P5336 sacrifices IOPS write performance for the benefit of additional capacity. It’s also interesting to note that the indirection unit (IU) size has increased from 4KB on the P5430 to 16KB on the P5336. An increased IU size creates additional overhead for backend clean-up, especially with random writes, so this explains the performance discrepancy. The increased IU size is probably needed to keep controller memory at a minimum and, in turn, the cost of the SSDs down.

System Design

The interesting aspect of these two new products from Solidigm is how increases in drive capacity can negatively impact write performance. It’s not possible, for example, to replace two 30TB P5430 drives with a 61TB P5336 and still have the same bandwidth available. What are the implications of this?

Well, back in the old days of spinning media, the IOPS and bandwidth capability of a single drive (or spindle) were important in calculating the right ratio of expected performance to drive numbers. I/O density decreased over time, leading to capacity over-provisioning. The same scenario is appearing in all-flash systems.

We can see a few ways in which the storage systems market will evolve to take advantage of the diverse range of SSD performance and capacity options. The first is the existing move to build “capacity optimised” versions of storage appliances, as we’ve seen from Pure Storage and NetApp. The second will be to tier within a platform, similar to the way StorONE operates.

What about scale-out architectures like VAST Data and Weka? Here we can expect both vendors to work with customers to build out configurations that meet application requirements based on performance, capacity, and cost. But let’s not forget that both VAST and Weka are software-based solutions, with the customer on the hook to purchase or lease the hardware. What happens if the performance characteristics change after 6-12 months into a 3-5 year expected hardware purchase?

The Architect’s View®

We can expect every SSD vendor in the market to push towards 60TB drives. However, these devices will only be used with larger form factors and for scale-out solutions with specific requirements (like heavy read-intensive traffic).

More choice is great, but these new solutions aren’t symmetric in their read/write capabilities. The highest capacity Solidigm “performance” drive, for example, is the D7-P5520, which only offers a maximum of 15.35TB.

Going forward, storage system vendors will need to demonstrate how heterogeneous drive configurations can operate efficiently without capacity or performance wastage. Yet again, this begs the question as to whether custom hardware (like Pure DFMs) is a better route when building storage systems compared to the increasingly diverse range of SSDs available on the market today.