Every technology, every architecture, and every design technique has an expiration date. And this is very true for the world of information technology. It would be inconceivable if assembler languages, hierarchical databases, and waterfall design techniques would still be used to develop the complex systems of today. No exception applies for business intelligence.
Rick F. van der Lans, R20/Consultancy, rick@r20.nl @rick_vanderlans
Rick will be teaching the following two courses for IRM UK in the next couple of weeks: Incorporating Big Data, Hadoop and NoSQL in BI Systems and Data Warehouses, 29-30 November 2016, London and The Logical Data Warehouse – Design, Architecture, and Technology, 1-2 December 2016, London
The heart of most current BI systems is formed by a traditional data warehouse architecture initially designed to support classic forms of reporting. For such systems, aspects such as improved governance, high‐quality data, and stability play a key role. To offer such qualities, BI systems are accompanied by a rigid development, operation, and management process.
This traditional architecture has had a great run for twenty‐five years and has served countless organizations well. But for many organizations it has passed its expiration date. Due to its rigid architecture, many new BI requirements are hard to implement with the existing BI systems. For example, BI systems have to support new forms of reporting and analytics, such as self‐service BI;
investigative analytics; data science; external users, such as online customers, partners, and suppliers; new storage technologies, such as Hadoop and NoSQL; external data sources, such as social media data and open data; and large quantities of data. In addition, reports must be developed faster.
Organizations know that their current BI system can’t be thrown away, because the existing reporting workload has to keep working. But how should they implement this new BI workload and integrate it somehow with the existing system? Many organizations struggle with this dilemma. Currently, organizations try to solve this problem by developing many analytical islands, and, because so few specifications are shared form island to island, business analysts are constantly reinventing the wheel. Plus, it’s close to impossible to guarantee report consistency across the classic reports and the new BI workload.
How can old and new forms of BI be supported by the same system? This big BI dilemma must be solved. New BI architectures are needed that support the traditional, somewhat rigid style of BI development with the new development style that is introduced by the new BI requirements.
The two styles of BI development correspond with the two development modes introduced by Gartner. Gartner’s Mode 1 and Mode 2 clearly match, respectively, the traditional BI development style and the new development style. Mode 1 is the classic style of development, in which every report must be
reliable, reproducible, and correct. These reports must be formally tested, governed, managed, they must be auditable, etc. Mode 2 corresponds to the more agile development styles that focus on speed and agility as well as experimentation, flexibility, and self‐service analytics.
The challenge, for most organizations, is how to transform their current BI system into a modern BI system that supports both development modes. In other words, they must transform their uni‐modal BI system into a bi‐modal one.
One new architecture used for developing bi‐modal BI systems is the logical data warehouse architecture. Systems with this architecture can exhibit the same robustness as the traditional data warehouse for the standard forms of reporting. In addition, they are more suitable to support new data sources, such as big data and open data; they can more easily handle new data‐storage technologies, such as Hadoop and NoSQL; they match better with the dynamic world of self‐service BI; they simplify support for investigative analytics and data science; and they speed up development and ease maintenance with fewer resources.
This article is derived by permission from the whitepaper by Rick van der Lans Developing a Bi‐Modal Logical Data Warehouse Architecture Using Data Virtualization, published by Denodo Technologies, October 2016. The full whitepaper can be downloaded here
Rick F. van der Lans is an independent analyst, consultant, author, and lecturer specialising in data warehousing, business intelligence, data virtualisation, and database technology. He is Managing Director of R20/Consultancy based in The Netherlands, and has advised many large companies worldwide on defining their business intelligence architectures. His popular IT books have been translated into many languages and have sold over 100,000 copies. Rick writes for TechTarget and B‐eye‐Network. For the last 25 years, he has been presenting professionally around the globe and at international events. In 2012, Rick published a new book entitled “Data Virtualization for Business Intelligence Systems”.
Copyright © 2016 R20/Consultancy BV. All Rights Reserved.