The New Big Picture for Data
Friday, October 31, 2014
It's a heady time for data. We've seen more change in the last few years than in the previous couple of decades. Because of this, we need to think about data in some new ways.
For example, the traditional big-picture view of data technologies looks like this:
In this world view, the operational data that applications use is stored in a
relational database. Over time, that relational data gets loaded into a
relational data warehouse, where it becomes analytical data. Business
intelligence (BI) applications then use that analytical data to help
organizations make better decisions .
But things are changing. Here’s a more accurate big-picture view of
the data world today:
Increasingly, applications are using relational and NoSQL
databases for operational data. Turning this operational data into analytical
data implies having both a relational data warehouse and an unstructured data
lake. BI applications are then able to access both kinds of
data to help their users.
And there’s another new piece: search data. As search
services become more available (Amazon Web Services and Microsoft Azure both
provide them today), building search into every application gets easier. Users love search, and with a managed search service in the cloud,
the barrier to entry is significantly lower. But search data is different from
either operational data or analytical data—it’s a new category. Accordingly,
it’s staking out a new position in the data world.
Data technologies have shaken off decades of relational
torpor; lots of new things are happening. It’s time to look at this world in a