There are many reasons why an organisation may recognise the requirement to address data quality such as: –
- To comply with regulations
- To gain or maintain competitive advantage
- To reduce costs and manage complexity
- To ensure survival through management of risk
- Mergers and acquisitions
- Technical / Systems upgrades
Embarking on a data quality program can be daunting and organisations will often seek to utilise or develop a framework. A framework should help to organise how people think and communicate about complex issues. Some published frameworks appear complex in themselves. It is easy for an in-house developed framework to progress to the point where it becomes more of a business model than a tool for helping to give direction.
At the heart of any data quality program is the organisations attitude towards quality management. There have been many books written on the subject of leadership and culture and I do not intend to embark on a detailed discussion of this complex issue. Suffice to say that senior business leaders must be seen to support and contribute to any data quality initiatives. Without an open involvement by senior management it becomes easy for individuals throughout the business to dismiss initiatives on the basis of being – “too busy, not my problem, my data is fine…” Similarly it is necessary for everyone throughout the business to know that appropriate action will be taken regarding non-compliance with quality initiatives. It is down to the business leaders to set the appropriate “tone” here and this should be aligned with the business aims for data quality i.e. zero tolerance or best endeavours. In establishing the desired culture, communication is key as it is in so many other aspects of business.
Governance is a big issue and can mean different things to different organisations. However there are a number of areas to focus on.
- Roles & Responsibilities – who is responsible for data quality? Ultimately the CEO needs to take responsibility as they do for establishing the organisations culture. But they will no doubt need to identify and assign responsibility to other key individuals across the business to carry out and monitor progress.
- Data Ownership – who owns data can be a particularly thorny issue. Is it the person or department who create the data? Or is it the data consumer – the person who makes use of the data? The IT department is often the “default” owner of data since it is usually their systems that store and process the data. In reality all of these have a role to play which is why organisations often assign an owner or steward who is able to focus on the data alone and identify and resolve issues wherever they originate.
- Data Profiling & Reporting – It is essential that organisation understand the “as is” position at the beginning of a data quality program to provide a benchmark against which progress can be measured. Regular monitoring is essential in order to quantify the impact of data quality activities and results must be reported to the project sponsors in order to maintain momentum.
- Decision Making – How are decisions relating to data made? One of the major contributors to data related issues across an enterprise is the lack of a central control mechanism when it comes to making decisions relating to data formats, storage, access etc. This is why organisations often end up with data stuck in isolated silos with no easy way to access or combine the data.
Physical components can be considered as the first line of defence in the protection of the organisation from poor quality data.
- Data capture screen design
- Point of entry data validation
- User Training – manual data validation
- Data Quality firewalls
- Automated testing of IT changes
It is important to spend time and understand the motivation for embarking on a data quality initiative. Such initiatives can be lengthy, disruptive, and costly and carry an element of risk. It may be tempting to just go ahead and start correcting whatever data attributes can be corrected as quickly and simply as possible – this is a natural reaction. However it is worth drawing a distinction between data quality and information quality. Data is data. Information is intelligence that is derived from the data and is used to make business decisions. So the data quality program should be directed at those data attributes that contribute to the business decisions that support an overall business development strategy. So understanding data issues across the business is key – what issues do people have, what do they need in order to improve business performance?
Whist data quality may be the right thing to do, information quality is the smart thing to do.