Data Warehouse Automation & Real-time Data – Reducing Time to Value in a Distributed Analytical Environment
Creating Data Products in a Data Mesh, Data Lake or Lakehouse for Use in Analytics (9-10 June 2022, Stockholm)
Smart Infrastructure & Smart Applications for the Smart Business – Infrastructure & Application Performance Monitoring
For those of you looking at Operational BI a very common question is how will this use of intelligence impact the classic BI system architecture? To answer this question requires that we first define operational BI. This is the use of intelligence in every-day business operations. We are not talking about running reports that access an operational system here. What we are talking about is much more comprehensive than that. Two major used of BI in operations are:
- Access to on-demand BI in the context of a process activity
- Event-driven automatic analysis and decision making
The on-demand use of BI in business operations is about right-time intelligence. This is where a user chooses to perform a specific activity as part of an every-day operational business process and is presented with only the BI that is relevant to help the user perform the task more effectively. It is sometimes referred to as a BI in context™ and is the most precise use of BI. To make that happen typically requires the deployment of BI systems in a service oriented architecture (SOA). In that sense it requires companies to bring together IT professionals responsible for business process management and SOA with IT professionals responsible for BI. Most companies are not yet organised in this way and so will need to mobilise to make this happen. However the impact on the BI system is relatively minimal in that the latest releases of many modern BI platforms today are already service enabled™ i.e. they have web service interfaces to allow on-demand invocation of reports and queries from composite applications, processes and portals. So in this case the BI system per se is really being extended to plug into a SOA and the emphasis is more on re-organising IT to make on-demand BI happen.
However event-driven automatic analysis is a different ball game. In this case we really are stepping outside the classic™ BI system architecture in the sense that this aspect of operational BI is about being able to detect operational events, automatically analyse data and take action well BEFORE the data reaches a BI system. This brings together new technologies such as data streaming, highly parallel in-memory data and rules engines with familiar technologies such as event driven data integration and scoring models built by power users using data mining tools. So if you have event-driven data integration tools already (most ETL tools support this capability) and you already have power users developing predictive models using data mining tools then there is nothing new in these areas. However it is the data streaming, in-memory database (both IBM and Oracle for example have extended their DBMS products by integrating them with in-memory databases) and rules engines that are new. Implementing operational BI in this way is about automating business optimization in every day operations. This kind of automation is operational performance management. So some BI system related technologies are used in this new form of operational performance management while other required technologies are brand new to us. Event-driven business monitoring therefore is a different architecture to a classic BI system because of the need to analyze stream data in memory before it ever reaches a data warehouse. Once analysis has taken place, decisions made and action has been taken it is only then than that event data may well find its way into a BI system via classic data integration processing. So we have to think differently about this. In that sense it would not surprise me if this form of real-time business event monitoring was initiated as a new project with a different architecture outside of a BI team.
In many cases it may well be that this kind of sponsored project is part of a business process management or an RFID initiative rather than being part of a BI program. If this happens it should in my opinion be quickly flagged up and brought under the management of the IT team responsible for BI and Performance Management otherwise the worlds of operational performance management and strategic performance management will never meet. This would be a very disappointing outcome. Frankly, they have to meet because business users need performance management to encompass both the classic strategic performance management (strategy management, scorecards, dashboards, budgeting, planning etc.) AND operational performance management. They need to see what is happening over time in their area of responsibility alongside what is happening right now. Therefore once again we need to keep our eye on the ball regarding project sponsorship, ownership and architecture so that the true benefits of on-demand and event driven operational BI becomes a major contributor to business performance and allows people to leverage intelligence to help them optimize every-day business operations.