Every so often I am asked what I do. I am a Data Transformationalist focusing on all aspects of Data Management, Data Governance, Information Security, BI & Analytics, and AI & Machine Learning. By the time I get to Data Governance, eyes start glazing over and one of the feet turns outward, ready to run.
I’d like to share why data transformations are so important today. The end-goal of most transformations today is the ability to leverage the capabilities of Artificial Intelligence, Machine Learning, Natural Language Processing, bots, etc. These can be thought of as little corporate monsters or little corporate Einsteins depending on what they are fed. Their ‘data diet’ is what determines what they will grow into. It wouldn’t be fair to expect peak performance from an all-star athlete if they are fed an unhealthy diet, not purpose-built for optimal nutrition! If we, as corporate stakeholders and data custodians, expect these technologies to operate at peak performance, we need to make sure their ‘data diet’ supports such expectations.
Data transformation is a concept that many organizations embark on with the hope of creating a perfect data world. Technology is truly a continuous improvement industry and the devil is in the details because we have a constantly moving target. Initially, the enterprise data warehouse (EDW) was the source for Decision Support Services. It became common practice for organizations to adopt strategies where the EDW forced required data manipulation addressing data issues downstream rather than fixing it at the source. This worked when the data was used only for reporting and many data transformations were done within the reporting layer – moving the fix from the data layer to the consumption layer. The most common reason provided for this strategy was that technical debt was created due to time and resource constraints with the thought that it would be fixed when time allowed. As reporting became more divergent, so did the calculations within each report. This decision sidelined best practices mainly leading to the future impact being sacrificed, not fully realized, nor truly considered for the delivery of the immediate need. The choice is between nutritious & balanced meals vs fast food. I equate technical debt to a donut. Delicious, satisfying, and filling but having to work much harder tomorrow to deal with the after-effects.
Today, we have initiatives to leverage EDW for AI, ML, and Deep Learning. There are significant issues caused by the fast-food approach. Leveraging the EDW for these advanced technologies cannot be fully realized because the root data layer has not been optimized with the business logic nor business-critical calculations. These critical elements exist in a variety of reports and, more importantly, may not share the same logic across the reports for the same calculation. With the advent of these new technologies that are more attainable, effective, intelligent, and pivotable – AI, machine learning, and deep learning are the end goal to stay competitive and viable in today’s data-driven world. Data transformations focus on creating a solid platform and structure to feed our corporate ‘children.’ With technical debt, there is a lot more work to be undertaken and discipline to be implemented to get into shape to achieve these goals.
A balanced ‘data diet’ consists of the right blend of data quality, data integration, data management, data security, and data governance mixed with industry best practices specific for each organization. Data transformations done correctly should allow organizations to develop corporate Einstein’s today and easily move to the next emerging technology tomorrow by being built on a solid foundation.
Contact us today to schedule a consultation on creating your corporate Einsteins on the proper ‘data diet’.
Comments are closed