# Sustaining Data Quality – Creating and Sustaining Data Quality within Diverse Enterprise Resource Planning and Information Systems

Markus Helfert, Tony O'Brien

► To cite this version:

Markus Helfert, Tony O'Brien. Sustaining Data Quality – Creating and Sustaining Data Quality within Diverse Enterprise Resource Planning and Information Systems. Governance and Sustainability in Information Systems: Managing the Transfer and Diffusion of IT (Working conference), Sep 2011, Hamburg, Germany. pp.317-324, 10.1007/978-3-642-24148-2_25 . hal-01571741

HAL Id: hal-01571741

https://inria.hal.science/hal-01571741

Submitted on 3 Aug 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Sustaining Data Quality- Creating and Sustaining Data Quality within diverse Enterprise Resource Planning and Information Systems

Markus Helfert

Dublin City University
Dublin 9, Ireland
+353 1 700 8727

markus.helfert@computing.dcu.ie

Tony O'Brien

Manor Mill Lane
Leeds, West Yorkshire
+44 79 67477336

tony.obrien@hotmail.com

**Abstract.** Many studies have confirmed the challenges relating to data quality in enterprises. This practice oriented research confirms the premise that data quality is of paramount importance to the efficiency and effectiveness of all organizations and that data quality management needs to be embedded within the organizational routines, practices and processes. In this paper we present a study on how to incorporate data quality management principles into organisations. The overriding measure for 'real' success is the sustainability of quality data, thus improving the quality of data over time, to engender long term success. The proposed principles and concepts were applied within a case study. The conclusions drawn from this study contends that this research has unearthed new knowledge as to the means by which data quality improvements may be sustained within diverse enterprise planning and information systems.

**Keywords:** Data Quality, Data Governance, Enterprise Resource Planning, Key Performance Assessments.

## 1 INTRODUCTION

Over the last two decades data quality has been identified as a major concern for many enterprises Redman (1995); English (1998); Redman (1998); English (1999); Loshin (2001); Redman (2001); Eckerson (2002); Redman (2002); Redman (2004); English (2009), none more so than those operating enterprise resource planning and information systems (Deloitte 1999). For this reason this study, allied to a practical data quality improvement initiative, attempted to discover 'How can an organisation
create an environment where data quality improvements can be sustained'. Any
attempt to determine the means by which the quality of data can be improved will
produce only temporal ameliorations, unless such improvements become embedded.
Without the latter, any gains emanating from the former will be merely marginal or
short term at best.

In this paper we use a dual research approach employing both qualitative and quantitative research strategies focussed within the researcher's own organisation. Section 2 places this study within the confines of an actual ongoing data quality improvement initiative, within which the research process described within Section 3 is undertaken. In Section 4 we summarize the practical benefits of this study and present our research findings. Finally in Section 5 we conclude that the outcomes of this research have important implications for both theory and practice

2 CASE STUDY

**2.1 Case Background and Description**

The basis of the detailed research undertaken as part of this entire study was
carried out within the researcher's own organisation Remploy. Remploy is the largest
provider of employment opportunities for disabled persons in the UK, currently
employing over three thousand disabled people in over sixty individual factories and
offices across the entire country, whilst placing over 10,000 others into external open
employment each year. A Baan/Infor ERP system was implemented over ten years
ago and whilst there have been many benefits overall it was identified that there was
still scope for further improvements especially within the areas of data quality and
system complexity.

**2.2 Initial Progress**

This research study coincided with the commencement of a data quality
improvement initiative within Remploy during 2005. An initial approach was made
across a number of fronts to attempt to promote education and training;
documentation of procedures; the acceptance of responsibility, ownership and
accountability at all levels for processes and data; together with better management of
master data with the identification and implementation of 'quick wins'.

**2.3 Development of Data Accuracy Key Performance Indicators**

As part of this initiative seven key performance indicators (KPIs) were established
around the order fulfilment process, historically the sources of many of the data
quality issues. The KPIs were chosen specifically to reflect the salient elements of
these essential commercial operations relating to servicing customer needs. The KPIs
were designed to reflect the view of the world as seen through the lens of the ERP
system, compared with an *actual* view which could be obtained by direct observation
of the actual physical order process. In other words how closely the ‘system’ (*data*
within the ERP system) reflects reality (the real world) in the manner described by
Wand and Wang (1997: 94), whilst also providing a measure of the quality of the
actual *data* and the related *processes*. From the individual KPIs an aggregated *Index*
was developed weighted to take account of the aging of the various transactions and
this was then used as the definitive measurement of the ongoing quality of the data
within the KPIs.

3 THE RESEARCH PROCESS, RESULTS AND DISCUSSION

**3.1 Qualitative Study**

The qualitative element of the research took the format of a series of discussion-
type focus group meetings sharing experiences, ideas, issues, problems and successes,
around a basic flexible agenda, employing an action research approach. This approach
attempted to generate discussion and interaction to discover peoples' real feelings and
attitudes towards *their* data. In all, forty eight of the fifty four factories and seven
business operations and sales teams were covered.

The use of action research in this environment provided the study with a
considerable degree of richness in that the researcher had been a member of the
organisation for almost twenty years. During this time this ‘insider researcher’ had
worked directly with the majority of the participants and was known to virtually all.
This unique approach generated loyalty and trust amongst all parties and not only
provided rich material for this study, but also enabled the researcher’s colleagues to
gain a greater understanding of the significance of quality data and to appreciate the
importance of taking ownership of ‘their’ data. These latter consequences are seen as
key to the subsequent improvements that were achieved.

From the outset certain important notions and impressions emerged from the
discussions and the analysis and these were subsequently developed as key findings.
It was felt that these fell in three broad categories relating to: *lessons learnt* that
should be put in practice at all sites, involving basic quality management principles,
ownership, responsibility and support, together with measurement and reporting;
positive personal *motivational factors* which help to engender commitment from
individuals, relating to internal competition and targets, an acceptance of best
practices and how these relate to one’s ideas and principles; together with
organizational and cultural *environmental elements* essentially involving leadership
and management issues.

**3.2 Measurement and Reporting Process**

The importance of measurement, analysis, reporting and feedback was emphasised
continually throughout the entire research. Figure 1 below traces the progress of the
improvement programme by tracking the Index over the initial three years,
highlighting a real trend of improvement over this period, albeit with various
explainable fluctuations.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Data Accuracy KPI Index Improvement tracker</figcaption></figure>

A summary of the progress indicates:

* 29% improvement in the first six months to March 2007

* 33% improvement in the first year to September 2007

* 16% improvement in the year to March 2008

* 40% improvement in the first eighteen months to March 2008

* 27% decline in the eight months to November 2008- which coincided with the
Company's Modernisation Programme

* 37% improvement in the year to November 2009

* 52% improvement between September 2006 and November 2009

Table 1 below relates the movements in the KPI Index with progression of the site
meetings qualitative study both during and following the programme.

<table><caption>**Table 1.** Data Accuracy KPI Monthly Performance</caption><thead><tr><th>Month</th><th>11</th><th>12</th><th>01</th><th>02</th><th>03</th><th>04</th><th>05</th><th>06</th><th>07</th><th>08</th><th>09</th><th>10</th><th>11</th></tr></thead><tbody><tr><th>Year</th><td colspan="12">08</td></tr><tr><th>No of Meetings</th><td>0</td><td>13</td><td>18</td><td>3</td><td>15</td><td>8</td><td>0</td><td>0</td><td>0</td><td>0</td><td>1</td><td>0</td><td>0</td></tr><tr><th>Index Impr. % Month</th><td>0%</td><td>9%</td><td>7%</td><td>-4%</td><td>16%</td><td>1%</td><td>1%</td><td>5%</td><td>0%</td><td>2%</td><td>-3%</td><td>7%</td><td>2%</td></tr><tr><th>Index Impr. % Cum</th><td>0%</td><td>9%</td><td>15%</td><td>12%</td><td>26%</td><td>27%</td><td>27%</td><td>31%</td><td>31%</td><td>32%</td><td>30%</td><td>35%</td><td>37%</td></tr></tbody></table>
From analyzing the Table 1, it is evident that there was a significant improvement in the Index (27%), following the commencement of the factory and business meeting programme from December to April, in line with the number of meetings carried out. In addition it may be seen that this level of improvement was maintained immediately following the study and then further improved as the concept of data quality became more established within the organisation.

### 3.3 Summary of the Findings of the Study

There is considerable evidence to suggest that the progress and improvement described above have real significance leading one to believe that there is potential for real cultural change to take place if improvement initiatives are managed correctly.

This has implications for the wider context of this research as seen by the evolution of the measurement, reporting and overall improvement process, which may best be summarised below in Table 2.

<table><caption>**Table 2.** Evolution of the improvement process</caption><tr><td>“What gets measured gets done”</td><td>A good start, but by whom?</td></tr><tr><td colspan="2">moving to:</td></tr><tr><td>“What gets measured by the Exec gets done quicker”</td><td>A further improvement, but too top-down</td></tr><tr><td colspan="2">leading finally to:</td></tr><tr><td>What is measured, communicated, discussed and agreed at all levels has a very good chance of becoming embedded”</td><td>Bottom-up supported by top-down<br>A potential key to sustaining any kind of change?</td></tr></table>

This highlights the way the improvement process progressed from a traditional top-down management approach to an all encompassing company-wide programme embracing all areas of the business, in line with the development and progression of this entire study

There is considerable evidence to claim that any improvement initiative cannot be undertaken in isolation and that everyone needs to become involved. There is also however a caveat, in that at this stage it appears that the level of progress achieved appears to be commensurate with the levels of activity of the internal champions or change leaders, suggesting a climate of ‘controlled sustainability’ rather than ‘self sustainability’. This may be a reflection of the relevant infancy of the overall initiative.
## 3.4 Quantitative Survey

To build upon the progress made within the Data Quality Improvement Programme and the qualitative study, a web-based survey was carried out during the summer of 2009 amongst fellow Remploy colleagues as a means of determining their views, attitudes, thoughts, feelings and opinions with regard to data quality. The overall response to the survey was extremely constructive and encouraging and it identified a significant positive attitude towards data quality in critical areas. One significant result identifies a disparity between peoples' perception of the quality of the data they receive and their own perceived ability to influence and pass on data of sufficient quality to satisfy their own information customers' needs.

## 3.5 Summary of the Findings of the Survey

Table 3 below summarises the organisational-specific findings into a more 'general' format that will enable them to be applied within a wider environment.

<table><caption>Table 3. Summary of the survey findings</caption><thead><tr><td>Generic Data Quality</td><td>Remploy Data Quality</td></tr></thead><tbody><tr><td><ul><li>There is a high appreciation of the influence that People, Processes and Data have on the quality of data</li><li>There is a realisation of the importance of having the data right first time</li><li>The level of positive responses compares very favourably with the previous survey held amongst the data quality community</li></ul></td><td><ul><li>The overall attitude towards measurement, reporting and feedback was very positive</li><li>There was an appreciation of the importance of education and training</li><li>Almost 60% felt that 'everyone' has a responsibility to improve the quality of their own and the organisation's data</li><li>There was a huge disparity between the respondents' perception of the quality of the data they influence (82%) and that which they receive (26%)</li><li>90% of respondents identified measurement and reporting, problem resolution and process improvement as key elements for improving data quality</li></ul></td></tr></tbody></table>

## 4 ANALYSIS OF FINDINGS

The direct operational benefits to Remploy of this study as highlighted by the improved Data Accuracy Index have been referred to in depth, but there is also evidence to suggest that there have also been considerable improvements of a cultural and strategic nature. Further operational and strategic advantages have been derived
from enhanced reporting, budgeting and forecasting. The myriad of small meaningful
ameliorations, both technical and procedural, which have been applied by passionate
people during the period since the original Baan implementation, are now gaining
greater maturity alongside higher quality data to generate both operational and
informational benefits. Finally the recognition of the importance of data in relation to
overall governance, risk and compliance has provided enhanced levels of authority
and control.

The analysis of the *key findings* from this research and the subsequent detailed discussion enabled certain principle findings to emerge and these are seen as the main outcomes for knowledge and learning for both practice and theory that have emanated from this study to bring about change not only to improve the quality of data but also impart some degree of permanency.

The *role of the champion* is seen as key to promoting and embedding change and innovation. Local champions have emerged at various levels and from various functions within the organisation embracing the essential cultural and motivational philosophies to make improvements within their spheres of influence. The concept of *measurement, reporting and feedback* is a prerequisite for any successful change programme. Within this study this element is not viewed as a mere ‘central’ monitoring and control mechanism, but has been developed into a reporting process to provide sites and businesses with the information to manage their operations on a day to day basis. The necessity of *time and maturity* is seen as vitally important in embedding change. This element of ‘organisational patience’ is important to allow new ‘processes’, changes and improvements to become accepted and embedded as well as to enable ‘people’ to accept change, develop themselves, gain experience and learn new skills. In discussing the concept of *sustainability*, emphasis has been placed upon maintaining the momentum of improvement particularly within the process of measuring the quality of data. One has to be aware however that there may be occasions where the ‘costs’ of making further improvements within a particular field may outweigh the benefits that may accrue with the risk of sub-optimization. The quantitative survey and subsequent related detailed discussions have highlighted the huge disparity in peoples’ *perceptions of data quality* between the data they influence (82%) and that which they receive (26%). This raises concerns in general as to the quality of communication and the way people view their roles and their own performance and the performances of others.

**5 CONCLUSIONS AND IMPLICATIONS**

The conclusions drawn from this study contend that this research has unearthed
new knowledge as to the means by which data quality improvements may be
sustained within diverse enterprise planning and information systems. Further
evidence of some form of real sustainability becomes apparent when one examines
the overall performance of the KPI Index in subsequent years. During the year to 31st
Match 2011 a further 7% improvement was achieved making an aggregate 59%
improvement overall in the four and a half years. This substantiates the belief that the
data quality improvement processes described are becoming truly embedded in many area of the organisation.

## REFERENCES

- Deloitte Consulting LLP. 1999. ERP's Second Wave-Maximizing the Value of Enterprise Applications and Processes, Deloitte Consulting.

- Eckerson, W. 2002. Data Quality and the Bottom Line: Achieving Business Success through a Commitment to High Quality Data, The Data Warehouse Institute.

- English, L. P. 1998. "The High Cost of Low-Quality Data," Information Management Magazine, January: 1-5.

- English, L. P. 1999. Improving Data Warehouse and Business Information Quality- Methods for Reducing Costs and Increasing Profits, New York: Wiley Computer Publishing.

- English, L. P. 2009. Information Quality Applied. Indianapolis: Wiley Publications Inc.

- Loshin, D. 2001. "The Cost of Poor Data Quality," DM Review Magazine, June.

- Redman, T. C. 1995. "Improve Data Quality for Competitive Advantage," Sloan Management Review, Winter, pp. 99-107.

- Redman, T. C. 1998. "The Impact of Poor Data Quality on the Typical Enterprise," Communications of the ACM, (41:2), pp. 79-82.

- Redman, T. C. 2001. Data Quality: The Field Guide. Woburn, MA: Butterworth-Heinemann.

- Redman, T. C. 2004. "Data: An Unfolding Quality Disaster," DM Review Magazine, August, pp. 1-7.

- Wand, Y. and Wang, R. Y. 1996. "Anchoring Data Quality Dimensions in Ontological Foundations", Communications of the ACM, (39:11), pp. 86-95.

## AUTHOR BIOGRAPHIES

**Markus Helfert** is a Lecturer in Information Systems at the School of Computing, Dublin City University (Ireland) and head of the Business Informatics Group. His research centers on Information Quality Management. He teaches and supervises research projects, both at undergraduate and postgraduate level. Markus Helfert has authored academic articles and book contributions and has presented his work at international conferences. Markus Helfert holds a Doctor in Business Administration from the University of St. Gallen (Switzerland).

**Tony O'Brien** is a Finance Manager within Remploy, the largest provider of employment opportunities for disabled people within the UK and is currently leading the information and data quality initiative within the organization. He has over thirty-five years experience within the fields of Finance and IT and has been involved in the implementation and development of a number of Enterprise Resource Planning systems. Tony holds an MBA and recently completed a Doctorate of Business Administration at Nottingham Trent University, focusing upon sustaining data quality within planning and information systems.