Building a modern data platform

This is a guest post by Paul Stringfellow from Gardner Systems and was originally posted at Building a modern data platform – what have we learned? where you can also find Paul’s “Tech Interviews” podcast.

As I reach the end of this series, it raises the question “what have we learned?”. If you’ve read through it all, you’ve learned you are patient and I’ve learned that writing a series of posts actually takes quite a bit of time. But I digress!

Let’s start at the beginning – what is a modern data platform?

I’ve used the term throughout, but what does it mean? In the introductory post I stated “In today’s modern world however, storing our data is no longer enough, we need to consider much more” and that’s true as organisations now want their data to provide competitive edge and insights, we also need to ensure we are “developing an appropriate data strategy and building a data platform that is fit for today’s business needs”. In essence those two areas neatly define a modern data platform, storing data is no longer enough and our platform needs to fit today’s rapidly changing demands, integrate with new technologies and give the scale and flexibility we need to turn our data into an asset, all of this while ensuring our data maintains its privacy, security and we maintain governance and control

It’s not storage

While storage plays an important part in any data strategy (our data has to live somewhere) it’s important to realise when we talk about a data platform, it’s not about storage, while the right storage partner plays a crucial part, the choice isn’t driven by media types, IOPS, or colour of bezel, it’s about a wider strategy and ensuring our technology choice enables us to provide the scale, flexibility and security a modern platform demands.

Break down walls

We have also learned that data cannot be stored in silo’s, be that an on-prem storage repository or its modern equivalent the “cloud silo” placing our data somewhere without consideration of how we move it so we can do what we need to with it quickly and easily, is not designing a modern data platform.

Data Insight is crucial

Where our data is held and on what, while important, pales when compared to the importance of insight into how our data is used. Our modern data platform must provide visibility into the who’s, where’s, when’s, what’s and why’s of data usage, who’s accessing it, where is it and when, if ever, are they accessing it, what are they accessing and why. Knowing this, is critical for a modern data platform, it allows us to build retention, security and compliance policies, it allows us to start to build effective data leak protections and be more efficient with our storage and control the costs and challenges that comes with our ever increasing reliance on data.

Without this insight you don’t have a modern data platform.

Data is everywhere

We have also learned that our data is everywhere, it no longer resides in the protected walls of our data centers, it’s living on a range of devices both sat inside and outside of those walls. That’s not just the data we have, it’s also the increasing range of devices creating data for us, our platform needs to be able to ingest, process and control all of it. Protecting data on the very edges of our network to the same degree that we protect, secure and govern that which sits inside our data centers is crucial.

Cloud, cloud and more cloud

Just a few years ago the prognosis for the data industry was that cloud was going to swallow it all and those who looked to use “traditional” thinking around data would be swept away by the cloud driven tide.

Now while cloud is unlikely to wipe out all data life as we know it, cloud should certainly play a part in your data strategy, it has many of the attributes that make it an ideal repository, its flexibility, scale, even commercial models make it an attractive proposition.

But it has limits, however ensuring our data platform can integrate cloud where appropriate and maintain all of the enterprise control we need is a core part of a modern platform, you can’t design a modern platform without considering cloud.

It’s a platform

The reason I used the word platform, is because that is what it is, it’s not one component, it is built up of multiple components, as I’ve shown here, it’s storage, data management, governance, control, be it in the datacentre, on the edges of your network or utilising the cloud.

The days of our data just been about one element are gone, we need a strategy that looks at how we use data in its entirety.

The point of this series has been to provide some practical examples of the tools and technologies I’ve used building modern data platforms. Not every platform uses all of these technologies all of the time and it doesn’t have to be these specific ones to build your platform. What is more important is the concept of a data platform and hopefully this series has introduced you to some areas you may not have considered previously and will help you design a platform to get the very best from your data assets.

If you have any questions, please leave a comment on the site, or contact me on twitter @techstringy or LinkedIn

If you’ve missed any of the series head back to the introduction where you’ll find links to all of the parts of the series.

Thanks for reading.

This is part 6 in a series of posts on building a modern data platform, the previous parts of the series can be found below.

This is a guest post by Paul Stringfellow and was originally posted at

Building a modern data platform – what have we learned?

reprinted with permission.