IBM FlashSystem Review – Part 1 – Hardware

IBM FlashSystem Review – Part 1 – Hardware

Chris EvansAll-Flash Storage, IBM, NVMe, Storage Hardware

This post is the first of a series of three blog posts looking at the IBM FlashSystem shared storage platform.  The three posts cover hardware, software and the operations and integration of the solution.  IBM has sponsored this work and provided evaluation hardware for the project.

Update: 26 November 2021 to refresh details on product models and additional 5200 details.

Update: on 9 February 2021, IBM replaced the existing 5010 with the 5015, the 5030 with 5035 and the 5100 with the 5200 models. The specifications of the old and new models are retained in this post for comparison purposes.

Modern storage arrays have mostly commoditised onto standard server-based form factors from the early monolithic solutions of the late 1990s onwards.  IBM FlashSystem is no exception, with an evolutionary transition towards standardised hardware and media. Here we look at the FlashSystem hardware options and unique features including FlashCore Modules.

FlashSystem Family

The FlashSystem family is now divided into three main groups:

  • Entry – Models 5000 and 5200
  • Midrange – 7200 and 9200
  • High-end – 9000 and 9200R

Hardware is consistent across all models and built on a dual-controller/node (or canister) architecture that scales from a 2U server form factor to a single 19” rack solution.  The following table summarises aspects of the hardware configuration:

ModelForm FactorProcessorMemoryConnectivityDrives
5010 (H) (deprecated)2U, Dual Canister2.2Ghz, 2 core Intel Broadwell16-64GB1 GbE (iSCSI) & 12Gb SAS, optional 4x 16Gb FC, 10/25GbE iSCSI, 12Gb SAS12x 3.5” or 24x 2.5” SAS, max 392 with expansion
50152U, Dual Canister2x 2.2Ghz, 2 core Intel Broadwell32-64GB1 GbE (iSCSI) & 12Gb SAS, optional 4x 16Gb FC, 10/25GbE iSCSI, 12Gb SASUp to 392 drives (12PB)
5030 (H) (deprecated)2U, Dual Canister1.9Ghz, 6 core Intel Broadwell16-64GB10 GbE (iSCSI) & 12Gb SAS, optional 4x 16Gb FC, 10/25GbE iSCSI, 12Gb SAS12x 3.5” or 24x 2.5” SAS, max 760 with expansion
50352U, Dual Canister2x 2.2Ghz, 6 core Intel Broadwell32-64GB10 GbE (iSCSI) & 12Gb SAS, optional 4x 16Gb FC, 10/25GbE iSCSI, 12Gb SASUp to 504 drives (15PB)
5100 (H) (deprecated)2U, Dual Canister8-core64-576GB10GbE (iSCSI), optional 16/32Gb FC, 25GbE iSCSI/iSER24x 2.5” NVMe SSD, optional 4x PM drives
52001U, Dual Canister2x Skylake 2.3Ghz, 9 cores64-512GB16/32Gb FC/NVMe-oF, 25Gb iSER/iSCSI, 10Gb iSCSI, 12Gb SASUp to 748 Drives (23PB)
7200 (H)2U, Dual canister4x 8-core 2.1GHz Intel Cascade Lake Processors with hardware compression assist.128GB – 1.5TB10GbE iSCSI onboard, with optional 16/32Gb FC and FC-NVMe, 25GbE iSCSI/iWARP/NVMe-oF12x 2.5” NVMe SSDs, 12x 3.5” SAS HDD or 24x 2.5” SAS HDD/SSD, plus expansion
9100
(deprecated)
2U, dual canisterUp to 4x 14-core 2.2Ghz Skylake Processors with compression assist128Gb – 1.5TB10GbE onboard (iSCSI), with 25GbE and 16Gb FC optional.24x 2.5” NVMe Drives
92002U, dual canister4x 16-core 2.3Ghz Cascade Lake Processors with compression assist128GB – 1.5TB10GbE iSCSI onboard, with optional 16/32Gb FC and FC-NVMe, 25GbE iSCSI/iWARP/NVMe-oF24x 2.5” NVMe or FCM drives (760 per system), 12Gb SAS expansion
IBM FlashSystem Hardware Options

The current models replace the previously branded Storwize products and despite the name, are offered as hybrid and all-flash solutions as indicated with the “H” suffix.  The 5010/5 and 5030/5 models have onboard iSCSI (either 1/10GbE) and the option to add SAS expansion shelves.  Host connectivity can also be extended with 16Gb Fibre Channel, 10/25GbE iSCSI or SAS. 

FlashSystem 5100 Rear View

The 5100/5200 offers connectivity with 10GbE onboards and expansion cards for 16/32Gb Fibre Channel, 25GbE iSCSI or iSER. 

Hard drives are offered across five categories:

  • 2.5” Persistent Memory (SCM) drives
  • 2.5” Tier 1 Flash – 1.9TB – 30.72TB capacity
  • 2.5” High performance HDD (10,000RPM) – 900GB – 2.4TB capacity
  • 2.5” Capacity HDD (7,200RPM) – 2TB only
  • 3.5” HDD (7,200RPM) – 4TB – 16TB capacity

The 5100/5200 models upwards offer IBM FlashCore Modules (FCM), a custom drive that uses IBM MicroLatency technology from the acquisition of Texas Memory Systems (more on this in a moment). 

Where the 5000 series systems use dual controllers that are mounted side-by-side, the 5100 and 7200 models use a top/bottom design that provides for greater network port expansion. 

The FlashSystem 9200R combines two, three or four FlashSystem 9200 systems in a single rack acting as a cluster, with pre-cabled configuration and Fibre Channel networking. 

The features of each FlashSystem option are summarised in the following table (older models not included).

FlashSystem 5015FlashSystem 5035FlashSystem 5200FlashSystem 7200FlashSystem 9200
IBM Spectrum Virtualise Software
IBM Storage Insights
VMware & Red Hat Openshift Container Integration
3-site replication
Local & remote replication
IBM Easy Tier
Transparent Data Migration
Data Reduction Pools
Scale-out Clustering
HyperSwap high availability
Encryption
NVMe flash and FC-NVMe host connections
High -performance compression & encryption in FCM
External storage virtualisation
Storage-class Memory
FlashSystem Capabilities by model

References

Scalability

IBM offers a standard set of features and software functionality across the entire FlashSystem range, through the use of a single storage operating system (which will be discussed more in part 2).

The range of offerings from 5010 upwards provides an entry-point for SMB/SME customers, right through to the largest enterprise requirements.  Customers can choose to cluster FlashSystem enclosures together to create increased levels of availability, which is offered as a pre-packaged solution in the form of the 9200R.

Having a single product range based on a consistent hardware and software architecture is important because:

  • Customers can choose their entry point and scale up or down on the basis of requirements.
  • Customers can pick the right model and solution to expand existing infrastructure, either increasing capacity (more disk shelves) or throughput (more controllers).  FlashSystem aids that strategy by providing clustering and array-based replication.
  • A single look and feel provides consistent operations, management and reporting.  Scripting and automation that works on one platform will be guaranteed to work on another.
  • A consistent hardware model provides more predictability in terms of performance and availability. 
  • A single platform type reduces the amount of training and skills development needed for IT teams. 

FlashCore Modules

One exciting innovation that has developed from IBM’s TMS acquisition is the FlashCore Module (FCM).  Most storage vendors use off-the-shelf commodity SSDs or have built systems that are entirely bespoke in design.  FCM bridges the two options by offering a custom-designed solution which fits into a standard FlashSystem appliance using the 2.5” form factor and NVMe interface. 

FlashCore Module

FlashCore Modules have gone through a number of evolutions since the technology was acquired from TMS.  Today, the FCM 2 hardware uses custom FPGAs, Micron QLC NAND (96, layer, previously TLC in the FCM 1) and Everspin STT-MRAM to eliminate the need for super-capacitors.  This allows FCM to support capacities up to 38.4TB per device. 

SSD vendors are notoriously secretive about the algorithms used to manage the NAND within their devices, so using a custom design allows IBM to improve the capabilities of FCM, which includes built-in encryption and compression.  The encryption feature ensures no data can be read from a FlashCore Module removed from a FlashSystem appliance.  Encryption keys are managed internally by the system. 

Data compression within FlashCore modules produces a typical 2:1 reduction, with no performance impact. Where data reduction results in higher compression ratios, the following maximum effective capacities are achievable:

  • 4.8TB FCM – 21.99TB maximum effective capacity
  • 9.6TB FCM – 21.99TB maximum effective capacity
  • 19.2TB FCM – 43.98TB maximum effective capacity
  • 38.4TB FCM – 87.96TB maximum effective capacity

The maximum effective capacity ratio is determined by the size of metadata available on the drive used to store compressed data.  At a typical 2:1 ratio, a single 24-drive 2U system supports an effective capacity (before RAID overhead) of over 1.8PB and maximum effective capacity of over 2PB.

FlashCore Modules provide IBM with a significant differentiation over many competitors using traditional NVMe SSDs, including improved performance while using cheaper, higher-capacity NAND.  FCM 2 delivers 2 DWPD (device writes per day), around double the equivalent TLC-based drive.  Features including heat segregation (placing active data on healthy flash cells) are an enabler to meeting the DPWD capability. 

(Side note: FlashSystem does have restrictions on combining FCM and non-FCM drives in the same chassis, including the layout of RAID groups, so it’s worth checking the configuration documentation for the full details).

FlashSystem 5200

The FlashSystem 5200 model is a single 1U system that supports up to 12 SCM, Flash or FlashCore modules. We highlight this system specifically as we believe it offers differentiation in the market. The 5200 uses two redundant canisters, which are effectively two controllers within the physical server, each with power supply, CPU DRAM and PCIe adaptors. The canisters are hot-swappable, so offer redundancy and no single point of failure with uninterrupted replacement.

FlashSystem 5200 internal schematics

We think this hardware is unique in the entry-level part of the market, as most similar solutions are built around a 1U server with no redundancy. These solutions generally have a single motherboard, CPUs and DRAM with no component isolation.

The benefit of the 5200 design is that it provides a solution for edge and remote office locations where space and cost are a premium but redundancy is still required. A 5200 solution could be deployed, for example, with redundant application servers in a configuration similar to HCI but with the benefits of dedicated storage and consistent features/functionality across edge and core data centre locations.

The Architect’s View™

Shared storage appliances have evolved significantly over the past two decades.  The storage monoliths of the last decade have given way to high-density solutions that pack an enormous amount of value into a 2U chassis.  Today’s enterprise customers want simplicity, reliability, efficiency and value for money. 

FlashSystem has evolved into a unified solution that scales from small business requirements to the high-end enterprise. As vendors standardise on the 2U server form factor, features such as FlashCore provide IBM with differentiation in an increasingly competitive market.

The use of a 1U enclosure is interesting as the design still provides for two redundant canisters and the same level of redundancy as the 2U family members.

In part 2, we will look at the standardisation of the storage operating system on Spectrum Virtualise. Part 3 will look at the operational characteristics of FlashSystem and the cost economics including transparent pricing.


Copyright (c) 2007-2021 – Post #441a – Brookend Ltd, first published on https://www.architecting.it/blog, do not reproduce without permission.