As public cloud adoption matures, the generally accepted consensus of the long-term trend for technology will be the development of hybrid and multi-cloud operations.  In some scenarios, for example, the public cloud offers greater flexibility and cost management, whereas on-premises infrastructure provides greater customisation.  In this article, we discuss how businesses should develop a hybrid cloud storage strategy that uses the best of on-premises and cloud in a strategic and flexible manner.  By way of example, we will compare our recommendations to the leader in the hybrid cloud storage market, NetApp. 

Rationale

Before we get into the details, it’s worth first looking at why a hybrid storage strategy makes sense in modern IT and how this approach delivers more substantial benefit than either an on-premises or cloud strategy alone.  We should qualify that if a business has been “born in the cloud” and never used on-premises infrastructure, then this discussion will probably not be for you.  However, most mature companies already have a mix of technology solutions that will include both on-premises and the public cloud.

Data

At the heart of the discussion is data.  Every business on the planet uses data in some form as a driver of increased revenue and competitiveness.  In the early days of IT, that data probably took the form of structured tabular information showing sales, profit, and loss.  Today, businesses combine a mix of multiple data types and sources that include machine-generated, proprietary and public sources of information. 

All this data needs to be stored somewhere, and increasingly, businesses are adopting “delete nothing” policies or retaining data for extended periods.  When data gets used for business processes that include analytics, the processing may be done on-premises or could be executed in the public cloud, where the on-demand availability of complex compute products like GPUs can arguably be used more efficiently.  

Benefits

So, why build a hybrid storage and data strategy that spans both public and private clouds?  Here are some benefits of public cloud storage compared to on-premises deployments.

Cost Efficiency – the public cloud enables IT organisations to pay for usage rather than buy infrastructure.  Consumption is based on simple metrics like capacity in terabytes and performance (IOPS). 

Time to Value – accessibility of cloud storage is measured in minutes.  A typical file system with cloud volumes can be provisioned and active within 30 minutes through a GUI, CLI or API.  The process doesn’t require human intervention and, in the most advanced implementations, is integrated natively with other cloud services.

Risk Avoidance – a service-based consumption model pushes the infrastructure management tasks to the service provider.  Typical on-premises tasks that include design, planning, installation, and deployment all have a lead time associated with them.  Global supply chain challenges have introduced a degree of risk for on-premises purchases.  We’ll discuss the implications of this in a moment.

Innovation – the public cloud offers a breadth of technology that evolves on an almost daily basis.  Developers can try new solutions, build applications for testing, and tear them down with minimal investment and no commitment.  The result is much greater agility to develop and try out new applications and technologies without the financial obligation.  Innovation doesn’t just apply to the end-user side of the equation.  When services are delivered through a cloud model, the service provider can introduce new features and functionality at a much greater pace. 

Supply Chain Challenges

It’s worth taking a moment to talk about supply chain issues in the current market.  The COVID pandemic has impacted the global supply of microprocessors and directly affected many industries, including IT.  Whilst we can look at COVID as a once in a generation event (or even once a century), the supply chain will always be subject to fluctuation. 

Earlier this year, in two manufacturing plants in Japan, for example, the Western Digital/Kioxia partnership announced the loss of flash capacity and products due to materials contamination.  This issue resulted in the reduction of 8% of the world’s global supply of NAND flash, estimated at around 6.5 exabytes.  The result is an estimated 5-10% increase in NAND prices in mid-2022. 

In 2011, flooding in Thailand resulted in a global shortage of hard drives.  Prices almost doubled and took two years to recover.  For many IT organisations, the shortage introduced interesting conversations with their suppliers on the cost and availability of storage systems and upgrades.  Backblaze, one of the most visible cloud service providers in the market at the time, detailed the lengths the company had to take in order to maintain access to HDD supplies. 

The takeaway from these stories is clear.  Supply chain operations have a direct impact on the ability to deploy technology.  In on-premises deployments, the vendor may be able to mitigate some of those issues, affecting lead times and/or cost.  Typically, businesses internally hedge against supply chain issues in two ways, first, by ordering more equipment than necessary and so incurring increased costs; secondly, by constricting internal supply that results in longer lead times for internal projects.  The second option introduces the risk that business competitiveness suffers.

In the public cloud, all the responsibility for supply chain management lies with the cloud provider.  Cloud resources are generally pooled from both a capacity and price perspective unless businesses are big enough to negotiate special pricing or demand dedicated hardware.  As a result, supply chain issues are more likely to be mitigated across all customers, with economies of scale reducing the impact on individual businesses. 

Adoption

So, adopting a hybrid cloud strategy is clearly a good idea, but how can it be done?  There are three main groups to persuade that the benefits are worthwhile.  These are the business users, the end-users (including developers) and infrastructure teams (generally the storage administrators).  We’ll discuss each in turn in a moment, but let’s look at some of the operational, technical, and financial considerations towards which these teams will be sensitive.

• Data sovereignty – where is my data being stored?  Is it local, and can I comply with governance and compliance rules?
• Data security – is the cloud safe?  This eternal question is one that by now has been well and truly answered (yes, if you follow the correct process).
• Data protection – how do I protect my data in the cloud and across multiple locations?  Can I have a single unified data protection experience?
• Workflow integration – how do I introduce both cloud and hybrid storage into application workflow?
• Chargeback & billing – how do I align charges with the business, the vendor, and budgets?
• Performance – how do performance, latency and availability in the public cloud compare to on-premises?

It’s natural to expect questions from all aspects of the business.  Here’s how to frame the conversation.

Convincing the Business

Business owners want to ensure that their applications will run smoothly, with no risk and at a predictable cost.  A hybrid model enables the business to adopt billing at an operational expenditure level that can be more closely aligned with individual projects and applications.  Consumption in the public cloud is based on demand, which in turn is driven by the relative success of specific business applications.

Looking at the broader business cycle for the development of new applications, historically, the planning process to quantify the resources needed for the development and production deployment of new applications required a detailed analysis of storage capacity over time.  Frequently, these estimates would be incorrect, either wildly overestimated, resulting in unused resources or underestimated with an impact on project timelines.  With the use of a hybrid cloud, part or all of the development and production requirements can be diverted to the public cloud, mitigating the risk of miscalculating resources needed to deliver a project. 

Convincing End Users

In today’s modern development process, storage is a commodity to be consumed and used by applications.  A developer doesn’t want to be concerned with RAID groups, storage layout, managing capacity or availability.  Instead, storage represents an endpoint for the application, whether that exists on-premises or in the public cloud.  

From the DevOps perspective, storage needs to offer the same levels of security, performance, and availability as on-premises.  For the developer, the workflow process around the consumption of storage should be API or CLI-driven, making it easy to consume (and return) storage at any time. 

Implemented correctly, a hybrid storage strategy enables DevOps teams to consume storage from either on-premises or the public cloud without having to be concerned about which platform is being used.

However, we need to review what “hybrid” means in this context.  A multi-cloud strategy offers the capability to consume resources from multiple clouds, whereas a hybrid approach integrates those clouds together.  Imagine, for example, that a development team working with data on-premises needs to create a copy in the public cloud and have that copy kept periodically refreshed.  When storage is truly hybrid, that relationship between copies will be intrinsic to the platform, to the extent that refreshes can be scheduled and maintained incrementally with minimum overhead.  This capability is only available from a small subset of vendors.

Convincing the Infrastructure Teams

Infrastructure teams, specifically storage administrators, have spent decades honing skills focused on managing the availability and uptime of storage hardware.  In the past two decades, storage solutions have become increasingly more reliable, commoditised, and centred around features in software.  Some IT organisations don’t operate with separate storage disciplines because modern infrastructure removes many of the specialist skills that were previously needed.

Today’s administrators are much more concerned with maintaining access to storage, protecting data, preventing ransomware attacks and optimising costs.  As a result, the industry is experiencing a transition towards storage-as-a-service where vendors deliver on-premises, vendor-managed hardware that is ready for use by the customer.

A hybrid cloud strategy aligns with the ongoing direction of modern infrastructure management.  IT teams now offer a different set of capabilities to the business, acting as technology custodians and trusted partners to ensure the best mix of on-premises and cloud-based solutions.  The storage administrator now gets to deliver higher-value services to the business, focusing on the key metrics that matter the most rather than majoring in the specifics of the hardware.

Bringing it Together

The following table summarises the impact of a hybrid storage strategy on the business, end-users and storage teams.

Metric	Business	Developers	Admins
Data Sovereignty	Business users can specify policies that determine where data should be placed, geographically, or between on-premises and cloud.  Policies are implemented by the administrator.	Deferred to the business and administrators to assign by policy.  Developer simply chooses an appropriate endpoint.	Storage admins have control over deployment into regions and availability zones.  This can be built into automated policy definitions.
Data Security	Business users gain strong cloud-based identity management and security tools, with integration into enterprise IM solutions such as AD.	Developers continue to use existing credentials to access endpoints.  No change in operations.	Administrators integrate security with cloud and on-premises solutions, including audit and logging features.
Data Protection	Backup and restore can be integrated into existing processes, with file-share backup requirements specified at creation time or defaulted by policy.	Developers specify data protection requirements when creating file systems, leaving the specific implementation to administration teams.	Administrators implement data protection policies that can be aligned with on-premises implementations.
Workflow integration	The business should expect seamless integration between on-premises and public cloud.  This should include any requirements to authorise usage.	Cloud storage (and on-premises) offers deployment through API and CLI.  Developers consume and return resources on-demand.	Storage administrators step out of the path of provisioning and decommissioning, increasing business agility.
Chargeback & Billing	Businesses require and gain visibility of cloud usage, costs and utilisation with fine-grained granularity.	Developers can consume resources with much better visualisation on costs and with on-demand profiles that charge only while resources are being used.	Administrators gain much more detailed consumption information, making chargeback easier to implement and assign costs directly to the business user.
Performance	Cloud offers multiple performance options, with availability matching enterprise-class services.  The business doesn’t have to compromise with cloud-based deployments.	Developers select resources based on service levels rather than technology, deferring performance management to the cloud provider and/or admin teams.	Administrators offload responsibly for availability and uptime to cloud providers, with options to take StaaS for on-premises consumption.

A Practical Example – NetApp Cloud Storage

NetApp has evolved a cloud storage story that started eight years ago with NetApp Private Storage (NPS).  This initial offering expanded six years ago with the concept of the Data Fabric and Cloud Volumes, quickly followed by native cloud storage in all three leading cloud platforms (AWS, Azure and GCP). 

Today, customers have a choice of two public cloud deployment models.  CVS, or cloud volumes service, are native as-a-service offerings in AWS (FSx for NetApp ONTAP), Azure (Azure NetApp Files) and GCP (NetApp Cloud Volumes Service for Google Cloud).  CVO or Cloud Volumes ONTAP enables customers to run ONTAP in a virtual instance.  The CVS model provides greater integration into the cloud ecosystem, while CVO gives the customer more control over the storage environment. 

In addition to the public cloud, customers can choose to consume NetApp storage as a service through Keystone, either on-premises or through co-location partners such as Equinix. 

Looking back at our initial benefits, we can align the NetApp offerings against each of them.  Here are some examples. 

• Cost Efficiency – CVS solutions provide granular consumption-based billing using a simple tiering model.  CVO offers more flexibility to set service levels by choosing specific virtual instances and storage.  Keystone offers flexible models for as-a-service consumption on-premises and in co-located data centres.
• Time to Value – Both CVO and CVS services can be provisioned within minutes.  Using the FSx for ONTAP API, the entire provisioning process has the capability to be automated, including integration into AWS CloudFormation. 
• Risk Avoidance – For production operations, the speed of CVS/CVO deployments enables businesses to avoid risks associated with running low on resources or extended delivery timelines.  This is because the cloud versions of ONTAP offer the same features as on-premises technology, making them interchangeable to the user.    
• Innovation – For developers, the ability to provision storage without involving any admin team reduces friction and improves agility.  Development teams are free to spin up file systems for only as long as they are needed while retaining the ability to clone and replicate from existing cloud and on-premises sources. 

The Architect’s View®

All the characteristics of storage expected by business users and developers are available in the public cloud.  With minimal effort, all enterprises can transform their infrastructure strategy, with long-term benefits that include lower costs and increased business agility.  However, from a vendor perspective, some vendors, such as NetApp, are leading the way in innovation.  In this post, we’ve highlighted some specific benefits of NetApp technology.  We will be following this up with a more in-depth look at how to use public cloud storage from NetApp, validating the ability to build solutions such as FSx for ONTAP into a hybrid cloud storage strategy. 

Post #4e84. Copyright (c) 2007-2022 Brookend Ltd. No reproduction in whole or part without permission. The Architect’s View is a registered trademark of Brookend Ltd. NetApp is a client of Brookend Limited.