Torus – Because We Need Another Distributed Storage Software Solution

Torus – Because We Need Another Distributed Storage Software Solution

Chris EvansSoftware-Defined Storage

Update: Accourding to the Github page for Torus, CoreOS ceased working on the project in February 2017.  Clearly it was obvious that developing a new storage platform is in fact hard and needs developers with knowledge and experience, rather than spare time on their hands.

Last week the CoreOS project announced the release of Torus, a scale-out storage platform for container clusters built and orchestrated by Kubernetes.  At this point in the project’s development, the software isn’t production ready and still being heavily developed.  Over the last few weeks, scale-out storage solutions have been presented at Storage Field Day 10 and been part of the discussions at the recent SolidFire Analyst’s Day.  With so many solutions already on the market, do we really need another one?

Why?

The CoreOS blog post page justifies the need to develop a new scale-out storage platform, claiming “traditional” storage solutions don’t meet the needs of large clusters of “inexpensive” machines.  Their key tenet is to follow the self proclaimed “Google Infrastructure for Everyone Else” or GIFEE approach to application development.  It’s a fair point that older storage arrays and appliances can’t meet the needs of modern application deployment.  Storage provisioning was slow and cumbersome by modern standards and was done because the hardware wasn’t as commoditised as it is today.  Solutions were bespoke and managed in a structured manner to deliver reliability and high availability.

Re-Inventing the Wheel

However, we have plenty of solutions that can meet the requirements of today’s applications.  Platforms like SolidFire are already fully driven by API and will no doubt directly support containers (and can do via OpenStack and Cinder).  At SFD10, Hedvig demonstrated fully working Docker integration, as did Datera with Docker Swarm (link here).  There are numerous others out there too.

It’s somewhat disingenuous to dismiss the entire storage market and imply that none of the solutions available today can work successfully with container-based application deployment.  The principles behind Torus are; extensibility, ease of use, correctness and scalability (check out the article for a definition of what these terms mean in this context).  I would argue that almost every (scale-out) storage platform today can match those attributes as well as being further down the line in their development.

Key Requirements

What I can’t see in any of the details of Torus so far is a discussion on the way throughput and performance will be managed.  In fact the GitHub page for the project states specifically that “speed” is a secondary objective at this point in time.  Presumably “speed” refers to both throughput and performance, and in reality if it isn’t designed in from the beginning, retrofitting it isn’t really an option.

The Architect’s View

Torus to me looks like another “not invented here” project that wants to re-invent something that’s already out there and working.  Now, I’m all for competition, but claiming none of today’s solutions are suitable is just plain wrong.  The more worrying thing for CoreOS is that the project could start to head down the OpenStack model, where more and more side projects are created that divert time and attention away from fully developing the core products.  A better approach would have been to develop storage standards as part of the Open Container Initiative and allow existing storage providers to adapt or develop their solutions to an open and agreed standard.  I think CoreOS do want to control and manage the entire platform and become, piece by piece another OpenStack.  Over time Torus will probably be mostly deployed with CoreOS but I can’t see it getting widespread traction, with so many other competing solutions out there.

Further Reading

You can read more about CoreOS at their website, plus details of Datera and Hedvig, either on their own sites or through the videos at the Storage Field Day site (either directly or my landing page collating all of the event details).


Copyright (c) 2009-2021 – Post #A95F – Chris M Evans, first published on http://blog.architecting.it, do not reproduce without permission.