Six Sigma is a management strategy developed by Motorola in 1986. Basically, it’s a set of tools designed to analyze and improve business processes by removing defects. DMAIC is one of the project methodologies used in Six Sigma to improve existing business processes. DMAIC has 5 steps: Define, Measure, Analyze, Improve and Control. Thinking about data along these lines will greatly improve the quality of data in an organization.
A critical part of most IT projects is defining the data requirements properly. The challenge is capturing all the nuances of the business in well defined entities and attributes so a complete picture of the business transaction is captured. IT projects dealing directly with business processes define the initial data capture for the business. The data needs to be complete enough to support subsequent business activity.
Customer Relations management tools are a good example. You need to capture enough information about the customer to faciliate a good client customer relationship.
In Six Sigma, measure is the step to define what and how you’ll measure the process and what your KPIs are. This same concept can be applied to data quality. One approach is to measure Data Quality (i.e. whether the data is ‘reasonable’, and if the data is captured in a timely manner). Data quality ensures the facts are correct but not necessarily business meaning.
Business Intelligence, on the other hand, is looking at the data captured during a business transaction and reporting back what that data means in a business context. Business Intelligence is a good check that the data, as defined, represents the essence of the business transaction.
One of the key activities in Business Intelligence is analyzing all the data captured about a business and putting that data back into a business context. Often, it’s not always possible to report as complete a picture of the business because of Data Shrinkage. Simply put, there’s a data gap because not all data captured in the business processes are available for reporting.
This brings us to the Improve step in the DMAIC process. Looking across the business, you’ll find many silos of data, multiple applications collecting data, inconsistent data definitions and gaps in data. This all comes to light when the business asks a simple question like ‘What is my retention rate in the West region for medium sized customers who have been with us for at least 3 years?’ Chances are it will be difficult to find the region, size and longevity of a customer in one place.
Most likely, your Business Intelligcnce team will be asked to answer the question. The Buiness Intelligence team is highly skilled technically with a good understanding of the business. They have all the tools they need to cobble data together from multiple sources and answer the business question. Unfortunately, it usually ends there.
Control is the last step in the DMAIC process. For Six Sigma, this is the step to look for deviations in the process to prevent defects. For data, this concept can also be used to look for defects in the data when doing Business Intelligence reporting. Inability to easily answer business questions with the data available should be a red flag indicating defects in the how the data is captured and managed.
DMAIC is a great framework for looking at data holistically for an organization. It’s a way to think about data from when it is first captured to how it’s used to measure and understand the business. Business Intelligence uses data to provide insight into the customers and business trends and help to drive marketing strategies and grow the business. Too often it ends there.
Business Intelligence should also be the feedback loop to continually improve how data is captured about the business. The feedback loop completes the data life-cycle from data collection to consumption to continuous improvement of data collection. This feedback loop is a good way to continually improve data quality and drive business results. The Business Intelligence team is the catalyst that drives data quality in your organization.