The responses I provided to a media outlet on January 30, 2020:
Media: We hear a lot about data transformation these days and most companies are going through some stage of data transformation. Since data tends to be the domain of the IT group, it can be easy to assume that data transformation is a job for IT. In truth, to be successful, data transformation needs to be a team effort. Do you agree or disagree? Who should be part of the team? Who needs to initiative discussions? What should the data transformation process involve? I'm looking for input and insights from experts on data transformation and others that have been involved in the process and can share insights and best practices. Also interested in any reports/statistics on the prevalence and success of data transformation efforts.
Gfesser: Data transformation definitely needs to be a team effort that includes participants both inside and outside IT, keeping in mind that the line between the IT and the business continues to blur due to the importance of systems, software, and data to the enterprise.
In order to effectively perform a data transformation, sponsors should ideally be executive stakeholders who understand the importance of data to the business. Additionally, it is extremely beneficial for such stakeholders to have a technology background so that they can relate to the work to some extent, including the benefits that data can bring to the business, as well as the challenges that teams are likely to face. In addition to executive sponsors, participants from each business domain associated with the data need to be included with IT, as data transformation typically implies that the data being discussed already exists to some extent.
That said, in order to be effective, data transformation participants need to understand, as is the case with digital transformation efforts, that work performed in an evolutionary manner will likely bring the best results. The problem with doing too much at once is that communication between stakeholders typically becomes too complex, and it always takes time to built out the underlying technology stack used by the data, and unless this stack has reached some level of stability, at least for core components, rework will likely result.
While data transformation work is best executed in an agile manner like everything else, key stakeholders such as product owners and architects should be on the same page as to how the work should be approached. As with executive stakeholders, it helps for product owners to have technology backgrounds as well, otherwise significant time will be spent by architects to communicate why the work should be carried out in a given manner, and product backlog items arranged in an order that understands that any specific use cases of a given business domain depend on the underlying technology stack being in place first, or at least in a manner which enables both to be addressed in parallel within a given Sprint.
As someone who has been building data platforms with data transformation teams, I have seen clients want to move too quickly, leading to communication issues and improperly implemented data use cases when too many individuals are brought on board in a short period of time. Cost always seems to be the motivating factor in such scenarios, but executive stakeholders need to understand that savings in the short term typically do not translate to savings in the long term, due to likely rework involved, as well as team stress that may lead to failure that brings such efforts to a either a hold, or worse, to an end.
As with digital transformation efforts, data transformation is not cheap. The teams involved with such efforts need to be on the same page as to how the work is to be carried out, the importance of data as an asset, and that properly executed data transformation efforts will bring value to their firms. Teams consisting of varied stakeholders are assembled for a reason: data transformation efforts need to take the enterprise into account, with silos / factions within a given firm brought together so that they can work toward a unified goal.
In addition to assembling effective teams, the data transformation process should involve common understanding of its goals, reaching consensus on technology stack, determining an effective minimum viable product (MVP) that includes the data sources involved, the stages through which this data should progress, and the specific use cases to be implemented that may address a variety of business needs such as exploratory work, reporting, visualizations, and machine learning.
See all of my responses to media queries here.