Measuring the Size of the Storage Market

Last week I posted a small comment on the latest IDC Storage Tracker figures. The data shows ODM Direct as the largest category and the one with the most growth (73% year on year). For some time I’ve wondered if the reporting of hardware really represents the state of the industry. After all, it’s a big assumption to make that end users and hyperscalers only buy storage when they buy a server. In addition, the external array figures don’t take into consideration SDS software licence sales. Is it time for a re-evaluation of how we measure the storage market?

IDC Figures

If you’re not aware, IDC has been tracking storage sales for many years. The company tracks servers sold with storage and external storage arrays, by revenue. Each quarter the figures are announced for the top 5 or so vendors in each of the two categories. Obviously some vendors (like NetApp) only have external array sales, so their revenue is the same in both categories. Whereas HPE and Dell have larger storage server figures because they incorporate external arrays and servers. Just over a year ago I did some maths around the IDC figures which discussed whether storage as a market was reaching a point of stagnation. Some of the data in this post extends those figures with the latest numbers.

One final comment on the IDC figures – the data isn’t consistent or static. For example, there were no publicly published 4Q16 figures for Total Enterprise Storage (see this link). Instead IDC seemed to attempt to combine the two tables of data into one, so the External Enterprise Storage figures (which the combined tables seem to represent) look anomalous. Also, data is obviously revised after first publishing. Look at these two links (here and here), from one year apart and you can see that the initial estimated figures are significantly revised for some vendors in the following year (compare the two 1Q14 values). This means the figures presented here are estimates of trends rather than hard facts.

Revised 2017 Figures

Graph 1 – Enterprise External and Systems Storage 2007-2017

Graph 1 shows a revised Q1 total figure to include 2017. As the IDC data has some gaps, using either annual or all results comes out a little anomalous. Instead I’ve chosen Q1 for each period as the data is consistent by total across each year. The result is we see a continual decline in External Enterprise Storage and an increase in Storage Systems.

Graph 2 – IDC External Storage Systems Tracker Q1 2007-2017

What does this tell us? I guess it seems obvious that fewer bespoke storage arrays are being purchased and more data is moving to SDS, HCI and/or the public cloud. Graph 2 shows the figures for External Storage systems by vendor, with Dell and EMC now combined. IBM, HPE and NetApp are relatively flat, while Dell/EMC is declining and Others are increasing. Compare this to the overall Storage Tracker (graph 3) and we see HPE increasing, Others increasing, while Dell/EMC and IBM decline and NetApp sees some modest recovery.

Is Hardware Enough?

From these numbers it would be easy to assume that Dell/EMC’s storage business is in free fall. But is that really a fair reflection? Dell owns most of VMware, which has reported significant take-up of HCI solutions using vSAN, so there’s a lot of unreported storage licensing revenue not included in the hardware figures.

What about other vendors? HPE claims many StorVirtual licences out in the wild. There are lots of other software-defined storage solutions in the market too, such as the object storage vendors (Scality, Cloudian, Caringo).

There are also secondary storage vendors like Rubrik and Cohesity who sell appliances. Presumably these figures are included in the other section, rather than being counted as backup solutions.

Public Cloud

Finally there’s public cloud. The main reason for this post is to highlight the increase in ODM Direct figures, which have leaped 73.5% since last year and are now the largest category at 23.3%. I suggested on my Linkedin post that this rise was indicative of the huge migration to public cloud. However Rob Peglar (formerly of Micron, now Symbolic IO), who is probably in the know more than me, sees this as a move from SATA to M.2 NVMe as part of server retooling. Either way, this is a positive affirmation for public cloud. BTW, here’s some thoughts on the M.2 format (link).

The Architect’s View

The traditional storage market is shrinking, yet we’re told that we create data at an exponential rate. The market for storage hardware isn’t going away, just evolving. Having broad-brush categories like array or server-based storage doesn’t give us any real indication of the winners and losers in the market. We need a better benchmark metric that shows public cloud growth, sales in SDS licences and the data going into HCI. It would also be useful to break down flash and spinning media to something more useful than just all-flash and hybrid arrays. I suspect this is close to the “too hard” category, which is why we ended up with convoluted all-flash definitions in the first place. Hopefully IDC is working on new metrics that can reflect the industry more usefully. I’m certainly interested to see what results they show.

IDC Figures

Revised 2017 Figures

Is Hardware Enough?

Public Cloud

The Architect’s View

Related Links