In the bad old days, the first step when building a database, analytics or business intelligence solution would be to order or provision a number of servers for hosting.
Particularly in the data world, you would have to worry about capacity planning and over provisioning of infrastructure to account for future volumes of storage and processing. To unlock the budget for all of this, you would inevitably need a waterfall style project and a business case.
Nowadays, the go-to tools and platforms are evolving to be completely SaaS and serverless based solutions, where we simply start using the various data storage, middleware, and reporting solutions, and consume more resources in line with usage, often with consumption based pricing.
When designing a new platform, the first place most enterprises would usually look is your cloud provider such as AWS, GCP or Azure. There, you will find a range of relational and NoSQL datastores, some of which completely insulate you from the concept of a server. Often, you might look to store your data on some object store such as an S3 or Azure Blob Storage based data lake. The suite of serverless tools within the cloud providers arsenal is growing, spanning storage, ETL, analytics, streaming, business intelligence etc.
Outside of this, the leading vendors in the higher level tools are also increasingly serverless and SaaS. For instance, you might turn to Confluent Cloud for your streaming engine, Databricks for your analytics, Tableau Online for your business, Elastic for your search, or Timeflow for real time streaming analytics.
Leaving the infrastructure and middleware behind is great for total cost of ownership, in that you don’t have to stand up and then maintain all of this technology. When we don’t have to make a large up-front capital investment in infrastructure, licenses and project costs, you’ll be able to innovate and experiment with data much easier and democratise it across the business.
Your time to value is also massively improved, allowing you to move “up the stack” and straight to building the data products and analytics which move the needle for your business.
In short, serverless, SaaS and consumption based pricing will be transformative for how data and analytics are used within industry, completely changing the dynamic for how project are delivered and the business value that they create.