Blog

Data Integration – Breaking Free of Line of Business Dominance

Still today in this ‘modern’ world I wonder when common sense will prevail so that enterprises that demand real guidance from their BI investments will realise that one fundamental corporate barrier has to give way if true business benefit is to come from enterprise wide performance management. That barrier is the refusal of line of business to relinquish control over budget on data integration. At present many line of business BI projects cause re-invention of the wheel as the same data is integrated again and again for different stand-alone BI systems. If this continues, BI systems will continue to run the risk of inconsistency, duplicate development and overspend on data integration and BI tool technologies.

Big deal you might say. Business has to move and be agile and so can’t wait for enterprise wide agreement. Most businesses have done OK so far even with line of business BI systems. Well I cannot necessarily argue against all of that but one thing is clear, and that is that the explosion in interest in data integration is most certainly upon us.

Evidence of this was in abundance this week as I attended the jammed out CDI-MDM conference in London to do a joint talk on MDM with one of my clients and found a lot of business managers with a massive thirst for understanding data integration. So I duly networked to find out the reasons for the interest. What materialised was a consistent message coming back from person after person. We are here so that we can understand how to get a single view of our customers, our employees, our suppliers…etc. Thinking to myself that this must be Groundhog Day it took me back to distant memories of me on a stage with Bill Inmon some 18 years back talking about the very same subject that would be solved by Data Warehousing.

So why, I asked did they not think that data warehousing gave them this? The answer was crystal clear. “Who said we just needed a single version of the truth in analytical systems? What we need is integration of master data independent of applications so that this common data can be shared among operational systems as well as BI systems”. As one person put it “data integration should not be confined just to BI systems – it is an enterprise wide problem”. Another said “we have a business need to see across the enterprise to manage our operations”.

So I got the message. People are screaming for shared operational data accessible through shared master data services, But what about BI – are there problems there I probed? Yes came back the answers. Line of business data warehouses have partial duplication of data and data overlap. The result is that they still suffer from various levels of disagreement because projects develop their own data integration jobs which can cause reconciliation problems. Does any of this resonate with you? It certainly does with me – goodness knows I have lost count of how often I have seen this over the years.

As one CEO put it to me recently, “In the ’90s we bought into client/server and open systems. We took our mainframe applications, re-wrote them and distributed them across lots of servers and budget was devolved to each of my line of businesses to get on with building systems to meet their business needs. Line of business reigned supreme. While distributing the applications may have been OK, whose idea was it to distribute the data! What a crazy idea that was” !

Well I certainly have sympathy with that. If there is one area that has to become a CxO level backed ENTERPRISE investment it is surely enterprise data management and data integration. We need it in operations, we need it in BI and we need it consistent from the ground up. So MDM for me is unstoppable but it can’t stop there. If we integrate master data and hold it centrally then why on earth would you want to leave the transaction data where it is? Surely the MDM momentum will eventually drag transaction data consolidation kicking and screaming with it for shared operational use. I wonder….it might take a while but I certainly wouldn’t bet against it.

Back