Media Query Source: Part 41 - CIO (US digital magazine); Today's most notable data strategy trend
- CIO (US digital magazine)
- Today's most notable data strategy trend
- Data lakehouse adoption
- Combination of data lake & data warehouse
The responses I provided to a media outlet on October 17, 2022:
Media: What is today's most notable data strategy trend?
Gfesser: Data lakehouse adoption.
Media: What makes this trend so important?
Gfesser: Data lakehouses make use of open standards to combine the traditional data lake with the traditional data warehouse, which typically resides in a separate analytical database product apart from the data lake.
As such, architecture is simplified, and costs are reduced. Data does not first need to be moved from the data lake to a separate database product, and the performance of a data lakehouse can often either meet the performance of a data warehouse, or be "good enough" depending on use case. Additionally, data lakehouses enable data engineers and data scientists to work in the same collaborative environment, outside of traditional silos, enabling faster execution.
Media: Is there anything else you would like to add?
Gfesser: My current team has been working to implement a data lakehouse for Deloitte this past year, by re-architecting an existing data lake and consolidating data that was once stored in separate database products.
Implementation is being executed in phases, by addressing fundamentals first: reworking data pipelines, properly zoning data and designing data table models for this data, making use of open source Delta Lake, and sunsetting redundant cloud services.
In conjunction with data lakehouse adoption, we are also exploring related trends which build on this foundation, such as data sharing. While data sharing is a longer-running trend than data lakehouse adoption, which is relatively newer, we plan to continue to make use of open standards as much as possible.