APIs Are Essential for Delivering Storage in Enterprise Cloud Infrastructures

It’s pretty easy to pick holes in the current legacy storage products, especially when it comes to integration within both public and private cloud deployments. However it’s worth discussing exactly what is required when implementing cloud frameworks, as the way in which storage is deployed is radically different from the traditional model of storage operations. In this post we will look at why traditional methods of storage management need to change and how that affects the way in which the hardware itself is used. This leads to a discussion on APIs and how they are essential to drive cloud deployments effectively.

The Legacy View

For the last 10 years or so, the traditional view of storage management has consisted of a number of Storage Administrators using a GUI, CLIs and/or scripts to process storage requests as they are generated by the business user. The process is highly manual, with lots of interactions between the requestor, the storage admin delivering the work and other intermediaries to cover things like billing, change control, capacity management and workload scheduling. This made the overall process pretty people intensive and not surprisingly elongated the delivery time.

Many end users will also have recollections of asking for their specific requirement to be told they can only have something “off the shelf” – i.e. storage to a standard LUN size and with a specific RAID protection. This was done for obvious reasons; firstly the configuration of large arrays was predicated on pre-planning and a fixed design, usually created at hardware installation time. Once defined and in use, it couldn’t be changed (or at least couldn’t be changed without significant impact and cost). Second, it makes sense to reduce requirements into a smaller subset to make the provisioning process easier.

As well as being rigid in configuration, many legacy arrays assume the creation and provisioning of LUNs is an infrequent task. Many require requests to be packaged and executed in batch and certainly can’t cope easily with concurrent requests. Although it is possible to automate some provisioning processes using CLIs and scripts, this doesn’t address the real requirements in creating an on-demand model.

The New World

As we scale up to ever large IT deployments and especially within service-based or “cloud” configurations, the idea of having large amounts of human intervention in the provisioning process simply doesn’t work. Instead, we need to move to a model of “storage on demand” where an external agent – user or orchestration software – can request storage as part of a portal and see the request actioned in real-time or at least within a matter of minutes or hours. This kind of operation can only be delivered where the hardware has been designed for the purpose. Where previously storage administrators were involved in every provisioning request, those requests will be actioned within a provisioning framework, defined by the administrator or a storage architect.

Framework

What do we mean by framework? Well, it’s all about setting a set of parameters around which allocations take place. This could include:

LUN Size
Resiliency/Availability
Performance
Security credentials
Snapshot policy
Capacity on demand LUN

The architect chooses which specific hardware components are used to meet the requirements. There are also operational limitations:

Maximum number of concurrent requests
Maximum number of provisioning requests per hour
Ability to suspend or reject provisioning requests by array
Restrict requests by array capacity
Restrict requests by user based on capacity guidelines

The provisioning framework also needs the ability to work asynchronously and autonomously; that is, to accept, process and acknowledge provisioning requests without the requestor having to maintain a permanent session to the array. Once requests are completed, the requestor is alerted via a callback mechanism or by manually checking whether a request has completed. Obviously there is a need for integration into monitoring frameworks, in order to track hardware and performance issues.

Designed for API

Of course there’s a big question around whether APIs can be retro-fitted to existing storage. In classic IT tradition the answer is “it depends”. Without a doubt no new storage array should be released without native API functionality. However fitting an API to existing technology will depend on how flexible the existing configuration process is. It’s possible to create an API wrapper and build automation into a middleware layer. This is how products such as iWave’s Storage Automator work. However adding these features to existing storage products could be costly and still be an imperfect solution.

The Architect’s View™

As new storage platforms evolve, native API support is a must. The Storage Administrator will simply be required to deploy the infrastructure and plug it into a higher framework from where provisioning will be entirely automated. Vendors offering this level of functionality will be the most attractive to service providers, looking to make the cost of acquiring and managing storage as cheap as possible. We’re about to see a paradigm shift in the way in which storage is managed and possibly an end to the storage administrator.