The COVID-19 pandemic is changing much of what business and society takes for granted. It may also be the incentive needed to get digital transformation done.
Dr. Barry Devlin, Founder and Principal, 9sight Consulting
Barry will be presenting the courses via Live Streaming, ‘Essentials of Data Warehouses, Lakes and BI in Digital Business‘ 22-23 March 2021
and ‘From Analytics to AI: Transforming Decision Making in Digital Business‘ 24 March 2021
Digital transformation, or digital business, has been high on the must-haves list of strategic initiatives from consulting firms and information technology vendors for more than half a decade. Evolving from earlier suggestions that businesses must become “data-driven,” the entire set of strategic thinking can be traced back to the emergence of “big data” as far back as the mid-2000s.
An Economist special report[i] “Data, data everywhere” in February 2010 began with a quote: “Information has gone from scarce to superabundant. That brings huge new benefits…but also big headaches.” Rereading the report a decade later, it is surprising how little has changed, beyond an increase of a couple of orders of magnitude in data volumes and the emergence of a new and even larger source of data—the Internet of Things.
In the introduction to a 2012 big data survey report[ii] with Enterprise Management Associates, I listed five broad types of business applications enabled by big data (1) revenue generation and business model development, (2) cost containment in real-time, (3) real-time forecasting, (4) tracking of physical items, and (5) reinventing business processes. The outlines of digital transformation as a business strategy were already clearly visible. We might ask: Why does it still remain largely unimplemented?
There exist two main reasons, in my opinion. First, it’s the sort of over-arching project that affects the entire technology stack—the IT equivalent of simultaneously changing all four engines on a 747 … in flight—and requires substantial organisational change from boardroom to shop floor and back. Such initiatives are exceedingly difficult to get going—except, perhaps, in one specific circumstance, to which I’ll return in a moment. Second, many of its promoters hail from a big data background and equate data-driven with big-data-driven, underestimating the role and value of traditional data—that come from operational systems and loaded into the data warehouse—in a true digital business.
“Never let a good crisis go to waste[iii]”
At the time of writing (February 2021), the COVID-19 pandemic has been raging for more than a year and, despite extraordinary ups and downs in rates of infection and death, many observers feel the health crisis is far from over. Vaccines were developed at speed but the logistics of roll-out and distribution are challenging. Meanwhile the virus mutates and undoes at least some of the progress. Economic impacts are becoming clearer, with the short-term outlook gloomy and longer-term prospects ranging from depression to a rerun of the “Roaring Twenties”. While a few industries, such as home delivery and online retail, are benefiting, the majority are seeing enormous impacts in all areas, from supply chains and product mix to customer engagement and home working for employees.
The combined political, social, macro-economic, and business outcomes—often in addition to prior changes—has left businesses across many industries needing to reinvent themselves, from the ground up in many cases. For example, many airlines and, indeed, airplane manufacturers are being forced to reimagine their businesses, from the elimination of four-engine planes to a move from passenger-and-cargo to cargo-plus-maybe-passengers. Such tectonic shifts in business impinge dramatically on IT, beginning with operational analysis and ending with new processes and information systems.
The trend toward a no-touch, physically distant environment[iv] drives digital interaction and the need for information and insight to new levels. Adding just-in-case to just-in-time planning in supply chains demands revamping operational and informational systems. The Internet of Things (IoT) has suddenly made another leap in potential across manufacturing and distribution. All part of digital transformation, and all driven by immediate or imminent business needs. What better circumstances could be imagined for getting digital transformation done?
And IT (as well as business) would be well advised to consider the similarities between the impacts of this and future pandemics and those likely from the rapidly accelerating climate emergency. All provide additional incentive to act.
From big data to treasure data
Back in the 19th century, virgin wildernesses from California to New Zealand and Lapland to Witwatersrand in South Africa were overrun by gold rushes to the cry of “there’s gold in them thar hills!” The 21st century has seen its own version: “there’s insights in that thar data exhaust!” Just as very few gold diggers got rich two centuries ago, the benefits of mining big data seem to be increasingly restricted to a few large companies and government agencies. You know their names—or at least some of them.
At the more mundane level of small, medium, and even top-tier traditional businesses, it is becoming apparent that digital transformation is about more than using AI, analytics, and algorithms to detect rare, novel, and beneficial insights in endless volumes of digital dross. Although there remains an element of truth in that thinking, the benefits are often harder than expected to identify and difficult to apply in production. What digital business needs is to reinvent the underlying operational and management processes to create entirely new cross-business value propositions.
This suggests that we need to start digital transformation from data warehouses rather than data lakes. The fundamental basis for data warehousing is to create and enable cross-functional understanding and reconciliation of data for quality decision-making support. Meaningful and consistent data from every aspect of business operations and planning is a sine qua non for digital business transformation. If you cannot get a reliable view of how all the cogs and wheels of your business are turning together (or not), any insights you may gain from the digital landscape—both internal and external—will likely hang in the air over the business, highlighting failures rather than enabling improvements.
Conclusions
Digital transformation begins from revamping existing IT systems. The data warehouse is already where most businesses have previously striven to become data driven. The basis therefore exists to become information informed and, indeed, insight inspired. Of course, the changes are not confined to the informational environment. Their roots also reach deep into the operational processes and data. Such change is indeed challenging, even if 747s will no longer offer an analogy.
That challenge can be most easily addressed when the business must reinvent itself just to survive. Then, executives are unusually receptive to change. Then, the need for such change is at its broadest and deepest. Then, IT systems are being opened up, investigated, and revamped to meet entirely new circumstances and needs. In this environment, digital transformation becomes more natural, more likely, more possible. In an age of pandemic, business is “all changed, changed utterly: A terrible beauty is born.”[v]
Dr. Barry Devlin is among the foremost authorities on business insight and one of the founders of data warehousing, having published the first architectural paper in 1988. With over 30 years of IT experience, including 20 years with IBM as a Distinguished Engineer, he is a widely respected analyst, consultant, lecturer and author of the seminal book, “Data Warehouse—from Architecture to Implementation” and numerous White Papers. His 2013 book, “Business unIntelligence—Insight and Innovation beyond Analytics and Big Data” is available in both hardcopy and e-book formats. As founder and principal of 9sight Consulting (www.9sight.com), Barry provides strategic consulting and thought-leadership to buyers and vendors of BI solutions. He is continuously developing new architectural models for all aspects of decision-making and action-taking support. Now returned to Europe, Barry’s knowledge and expertise are in demand both locally and internationally.
[i] “Data, data everywhere – A special report on managing information”, The Economist, February 2010, https://econ.st/2VeAaMl
[ii] “Big Data Comes of Age”, EMA and 9sight Consulting, November 2012, http://bit.ly/Big_Data_Survey
[iii] Quote attributed to Winston Churchill
[iv] McKinsey, “The future is not what it used to be: Thoughts on the shape of the next normal”, April 2020, https://mck.co/2ZFelIf
[v] William Butler Yates, “Easter 1916”