Databases are the next battleground for Public Cloud

This week, Amazon Web Services announced the availability of Amazon DocumentDB, a MongoDB-compatible document database. For AWS at least, the database market has been heating up recently. With so many offerings to choose from, are databases the next land-grab for the public cloud?

DocumentDB

Let’s look first at the announcement of Amazon DocumentDB. This new service is a fully-managed document database, based on version 3.6 of MongoDB. From what I’ve read, it looks like AWS is taking the open source edition of MongoDB and using the code as the basis of DocumentDB. Some of the feedback I’ve seen on news sites highlights that this isn’t the latest version of MongoDB and so doesn’t have all the current features. However, you do get the ability to create clusters of up to 64TB and distribute the data across up to 15 replicas and three availability zones, with six copies of data.

Naturally, you could choose to run MongoDB natively within EC2 and in fact, DocumentDB uses standard EC2 virtual instances. However, the difference with DocumentDB compared to self-build comes from all the integration points with other AWS services (like KMS) and the delegation of management functions to AWS. There’s also a MongoDB-compatible API, which we will come back to in a moment.

Latest and Greatest

A key criticism of DocumentDB is that it doesn’t use the latest version of MongoDB and so is deficient in some of the more recent features. Of course this is true, but in general, many IT organisations aren’t running “latest and greatest” versions, so being one or two generations back isn’t a problem. Let’s also not forget about the licence changes that were introduced in October 2018. MongoDB changed to a new licence model called Server Side Public License which requires companies wanting to modify and run MongoDB as a service to release their modified software as open source. For AWS, this would mean releasing the code that introduces new features or functionality, including that which integrates with other AWS features. I’m guessing that as a result, AWS has gone slightly back-level in order to avoid the licensing issues.

API

Going back to the API, AWS has stated that DocumentDB will be compatible with the MongoDB 3.6 API and so be simple to transfer over existing applications. The Database Migration Service helps here too. Why is API compatibility so important? Look at the wider AWS ecosystem and we can choose AWS S3 as an example of a de-facto API standard. Pretty much every object storage vendor on the planet supports S3 in some form or other. S3 compatibility is an issue I’ve covered many times before. The whole subject is important because the API acts as the endpoint to the data being stored and abstracts from the underlying infrastructure delivering the service. Write for one API and you can use any compatible platform.

This is great for customers/end users because developers can test on 100% compatible systems in public cloud, then choose later where the production application will run (or the other way around). There’s no dependency. So if it makes sense to run on-premises or in an edge-core design, then this is fine.

Catching Up

Even though DocumentDB is back-level to the current release of MongoDB, adding new features is a case of writing software and changing the API to support new functions. The API is freely available and not in itself copyright as part of open source. This means AWS can develop DocumentDB in parallel to MongoDB, as long as the API offers feature compatibility. Why develop in parallel? Because AWS doesn’t want to make its software available to a wider market. The intellectual property of AWS is in integrating products and solutions together to deliver a cloud service.

Database is a Service

AWS has been pushing hard on databases recently. At re:Invent in November/December 2018, Andy Jassy announced new database solutions in his keynote, including TimeStream and QLDB. RDS already offers all of the major database platforms and is coming on-premises soon. To be fair, Microsoft Azure hasn’t been standing still and has a number of solutions that mainly cover traditional databases, but aren’t as wide as AWS. Google perhaps follows up a close third, depending on your requirements.

Databases feel like a natural service to offer public cloud customers in a similar way to object stores. Data is simply stored and retrieved in a structured format without a need to understand the underlying infrastructure. Performance can be linked to service levels (although for DocumentDB at this time, the requirements are linked to the size of EC2 instances). As an IT department, why not get rid of all that DBA overhead that doesn’t add value and let database administrators focus on the content?

The Architect’s View

Like the push into every aspect of retail we saw from Amazon, AWS continues to deliver on all aspects of modern IT. Locking customers into a public cloud database is a smart move because data has inertia and moving it is hard. DocumentDB adds to the line up of existing database platforms (SQL and NoSQL) that AWS already supports. The risk for MongoDB is in AWS matching and exceeding the efficiency of new features in MongoDB Server as the software evolves. In addition, this new service competes directly with MongoDB Atlas, the managed public cloud offering of MongoDB Server in public clouds.

This battle, if there is one, will be about features, functionality and ease of use. Expect things to get very interesting in the database market in 2019.