Data Quality Assessment for the hypothetical company: Internet Training Solutions (ITS)
By: Graham Waters
Tags: Governance, Hypotheticals, Data Quality, Data Quality Assessment, Data

Graduate Professional Studies, Brandeis University

Abstract

The hypothetical company Internet Training Solutions (ITS) is considering amending or changing current data quality policies and procedures by either hiring new staff in a management capacity or modifying some of the methods currently in place. This whitepaper examines the structure of ITS for potential areas of improvement and provides an introduction to the concepts of data quality in organizations.

Keywords: Data Quality, Implementation, ITS, Data Ownership, Data Stewardship

Travis Mason (CFO),

Internet Training Solutions (ITS) has expressed interest in exploring improvements in the data quality platform within the company. This is an effort to mediate the stakeholder concerns and highlight opportunities for growth within ITS and a response to management's request for a comprehensive review of several key issues. This whitepaper is the cumulation and culmination of three months of interviews and observations of ITS staff. In addition to the results of these interviews, it will also address some of the senior management’s questions received over the past few months as well as present a comprehensive plan for a data quality program within ITS as dictated by best practices in data quality.

Data Quality Assessment at ITS

Defining Data Quality

What makes data be considered quality data? This question represents a vital mental exercise for companies in every industry. There are two categories of organizational responses to it. First, some organizations may assume they already have quality data or that their systems and processes are already performing well enough for their needs. These companies run the risk of being put out of business by a simple lack of clerical dependability. The other category of organizations contains truly visionary companies, and they see that data quality is not just a bonus or an optional perk for their business.

Figure 1. ITS Data Quality Structure

They see that without strong data quality, a company is already on the road to bankruptcy. ITS is, of course, in the latter category, and for this reason, ITS management and staff at all levels of corporate competency will agree that a robust data quality foundation is a priority. With that being said, it now becomes necessary to dive deeper into what data quality actually is and how it can be explicitly pursued at ITS. Many different sources have defined the quality of data, and one of the aptest of these refers to the quality of data as “its ability to satisfy its user requirements” (Bhansali, 2014, p.11).

Senior Management Questions

What are the main points (or parts) of data quality?

Laura Sebastian-Coleman (2013) surveys the goals of data governance in her book, Measuring Data Quality for Ongoing Improvement, and one of the primary goals that she mentions is data quality. This quality of data can be split into more granular aspects of that data, as is shown by Bhansali. “Data quality dimensions, such as accuracy, timeliness, completeness, and trustworthiness, are defined in the context of the end-use of data” (Bhansali, 2014). Data that is not accurate is very nearly useless data unless it can reveal some pattern that allows for reverse engineering of what the actual data was. If it is predictable, then it has some redeemability; however, this is very often not the case for companies where data does not fit into this type. If data is timely, it is where it is needed when it is required. Data that is complete has referential integrity and is easily found. Trustworthy data is the responsibility of the data steward, and if data is found to be untrustworthy, then a conversation should be had about that. It involves a fitness for purpose test as well as several others (Bhansali, 2014).

Recommendations for the data quality program at ITS

Why is a Data Quality Program Team Essential?

In management at ITS, there seems to be much focus on monetary concerns and budgeting. This concern is a perfect complement to why a data quality program team is a good investment. The main objection that was raised when discussing this in interviews was that it would cost too much and that the account relationship manager position was sufficient in scope to service all of the needs that would be under the purview of a theoretical DQ Program team or manager.

“In a best-case, DQ program team members are focused on implementing data quality strategy, either directly as hands-on analysts or indirectly as propagators of and subject matter experts in the data quality methodology. Not all organizations will have a data quality program, and not all teams working toward these goals will have the words data quality in their names. However, in order to make significant headway on data quality management, it is better to have individuals assigned to lead these efforts” (Sebastian-Coleman, 2013). This concept of assigned leaders will be discussed further later on. The negative impacts of low data quality are multifaceted. “poor data quality can impact an enterprise at both operational and strategic levels” (Bhansali, 2014).

Figure 2. Data quality team at ITS roles and responsibilities. Definitions supplied by (Zatyko, 2015).

What does a DQ Program Team look like at ITS?

There are many predefined roles for members of a good DQ Team. These include data owners, data stewards, data consumers, data producers, data analysts, and data custodians. To properly unpack these roles and how they come together, consider Figure 2. Given ITS commitment to high-quality data, as evidenced by the interview with CFO Travis Mason, ITS is ready to begin the process of reeducation that will propel ITS into a data quality success scenario.

ITS Cultural Reframing Opportunities

It was a privilege to perform interviews with all levels of ITS staff over the past three months, and the findings presented a lot of promising opportunities for growth at ITS in the next year. During the interview process, it became apparent that several cohorts with differing opinions are presenting potential roadblocks to increased productivity and more excellent data quality. The contentious issue surrounded a discussion of accountability. It was clear that many of the general staff at ITS are of the persuasion that management, and management alone, should be in charge of ensuring data quality. The interviews revealed an undercurrent of bitterness towards those in management and, at the same time, a lack of understanding of data ownership and stewardship. ITS is going to see extremely positive results if management prioritizes the education of the general staff in areas like data quality and data governance. It should be stressed that employees cannot become passionate about something if they lack buy-in. The pursuit of organizational data quality is a cross-functional effort that is best managed at the top, but that should remain a focal point and a common impetus at every level of the organization. Once employees believe that ITS data is their data and that low-quality data generates low-quality business, which directly impacts their wallets, there will likely be changes in sentiment in the workplace surrounding this issue. However, as noted by Zatyko (2015), data consumers should be highly involved because they are the ones that are responsible for the definition of requirements for data. If these requirements are neglected, then management is placed at a disadvantage. If ITS data consumers truly desire positive results in the companies’ efforts towards data quality, then there needs to be an emphasis placed on these responsibilities. Also, if consumers are not actively involved in reporting data issues to the appropriate members of management, then the entire process falls apart, and quality goes down.

One user responded that “at the end of the day, they (management) should be the ones who have to ensure it works properly.” From a data governance point of view, this is true. After all, “Data Governance is the exercise of decision-making and authority for data-related matters” (Thomas, 2008, p.8). While the sentiment has validity, it should be noted that without the support of the general staff, those in data consumer positions, and a symbiotic consensus between the two groups (managers and the managed), there can be no progress towards data quality at ITS. That being said, the most impactful members of a data quality team are the data stewards. They are responsible for all of the data within organizational repositories (Wassén, 2017). In Figure 2, these are represented by the plum color. This response indicates potential friction between the management cohort and the data consumers. Steps may need to be taken to remediate this animosity and reinstitute positive company culture and accountability.

Another employee noted that “The pressure and the resources, the sort of hard answers and decisions have to come from the top.” This statement is partially accurate as the difficult decisions often to get made at the top. It may be helpful to reframe the statement in a less this or that manner. If the phrase have to come from the top could be rephrased as often end up at the top, it would exchange some of the helpless phraseology there for more empowering language. Decisions are ultimately decided at the top level, but they may have their roots in the general employee level. The data consumers are in charge of defining what standards must be held to, and without those stipulations, management should not be blamed for what is, in fact, the inadequate supply of regulatory feedback and requirements.

Crafting A New Framework for a Data Quality Team at ITS

The current structure at ITS has the potential to yield a highly functional data quality team. As the company moves towards a more efficient and lean data quality control implementation, it will be beneficial to consider the following data quality directives.

Assessing Management Commitment. In interviews with management, it was ascertained that data quality is a priority for ITS. This is a foundational piece of ITS’ future as it shows that the leaders of the company are committed to moving the needle on improving data (Sebastian-Coleman, 2013).

Characterizing Organizational Data Assets. Moving forward, it is essential that ITS define what data means to the company. How does this data represent and support the mission of ITS? What damage could maintaining false data or subpar data and allowing it to mingle with high-quality data? It is vital to diagnose how data is currently being handled within ITS fully, and if those existing processes could be augmented, improved, or replaced (Sebastian-Coleman, 2013).

Focusing on Quality by Application of Resources. ITS’ data quality team will be cross-functional and skilled in multiple areas of the business. ITS mission is delivering high-quality educational data to clients with a certain degree of confidence in the longevity of its value. In this process, it will be necessary to assess the readiness for a data quality team implementation at ITS (Sebastian-Coleman, 2013).

Questions to Assess Data Quality Readiness. The following questions are suggested by Laura Sebastian-Coleman (2013) for assessing the readiness of organizations to create a data quality team.

How does the organization currently assess and communicate about data quality? As shown in Figure 2, ITS relies presently on the Account Relationship Manager to be the go-between for suppliers and data custodians. She is performing a data stewardship role, and thus, she makes sure all of the data is correctly inputted by the suppliers and clients. That role is essential and will be integral in the data quality plan.

Who is responsible for the content and consistency of assessments and communications? The content and consistency of data is the purview of IT Management. The IT Manager is in charge of maintaining the data on ITS multiple systems, and she also checks the content for consistency, which is a crucial tenant of data quality. Travis and James, CFO, and CAO, respectively, are in charge of watching for any requirement violations as defined by the data consumers. As data comes into the system, data consumers at ITS perform initial data assessments.

How successful are they? In the past six months, ITS data quality has been slowly but steadily decreasing as recent onboarding has increased the volume of new data from the new clients and suppliers.

Are any activities in place directed at sustaining data quality? Travis has been working closely with Janice in IT to craft a periodic measurement system that assesses the quality of data every week. This may be better augmented with an in-line measurement solution that supplants or supplements the current one, which has proven to be less effective over the past six months.

What activities could be integrated? In the same interval, Janice has been doing a fantastic job as an Account Relationship Manager, and her task completion numbers have been notably better. It may be beneficial for all ITS staff to glance at some of her methods to determine if any emulation should be enacted across the whole of ITS (Sebastian-Coleman, 2013).

More tactics for assessment are noted in Appendix A. These questions, and the subsequent discussions they generate will become integral to developing a strategy of data quality in these next few months. Cheryll has proven herself to be capable in her role as Account Relationship Manager, which indicates she should be considered strongly as a candidate for the title data quality manager. Expanding her responsibilities may require expanding those of several employees in each section of ITS. There need to be leaders in ITS that function as middle management for optimal data quality control. It may be prudent for CFO Travis to begin selecting and promoting potential outstanding leaders in these positions.

Most Important Points in Data Quality for ITS

Which controls are most important?

Controls that are automated by the computer systems are much more reliable than humans by virtue of their computational power, but they also lack the insight that the human mind has. Considering that most ITS data is gathered from internet sources, it stands to reason that systems controls are on the front lines. This does not discount the importance of the organizational controls, and if anything, it increases their importance by association. Organizational tools are prone to higher errors than systems controls but may be better at specific tasks like ensuring timeliness. A useful control tool that should be implemented and updated to increase the success rate of data quality efforts on the front-end is data quality thresholds. These provide a way to automatically detect low-quality data that is coming into the system. Tools like these are essential for ITS data quality implementation.

Costs and Benefits to the Firm

Hiring someone to perform the role of a data quality specialist or manager has a higher impact on companies that deal with large volumes. Still, if the perception of responsibility remains bifurcated between the two groups: management and general staff, then a data quality professional and reeducation of staff are must-haves. The benefits of having high quality are notable. The Data Quality roadmap that is outlined by Neera Bhansali (2014) enumerates the necessary steps that must be taken to achieve data quality. The first step is the design stage. A sound data quality system must be well planned out before implementing it. Second is the development, which is a crucial step as many things can go wrong and may need adjusting. Implementation is the final step in this initiation process, and yet there are many more steps that go beyond initiation. To delegate this kind of complexity to the role of one individual increases the likelihood that the efforts will be fruitless. This illustrates why stratified leadership and management are so beneficial. Whether the data stewards are being trusted with far too large a responsibility or one manager is fighting to keep the organization’s data perfect, relying on one person is a recipe for disappointment. Prioritizing data quality also will increase monetary revenues at ITS by reducing incidents involving inaccurate data that is entered into the systems, whether educational data, institutional data, or even budgeting data. Another benefit of creating a data quality team is positive optics. It shows to the community that this company is committed to data quality, which in turn will drive more prospective clients to ITS (Sebastian-Coleman, 2013, p. 179).

Conclusion and Next Steps

Implementing a data quality team at ITS is not only a fantastic way to demonstrate a commitment to excellence, but it also increases the longevity of the business. Following the plans outlined in this whitepaper and remaining flexible and ready for change will enable ITS to cross-functionally transform into a leaner and better organization primed for greater volume.

References

Bhansali, N. (Ed.). (2014). Data Governance — Creating value from informational assets. CRC Press, Taylor & Francis Group.

Sebastian-Coleman, L. (2013). Measuring data quality for ongoing improvement: A data quality assessment framework. Morgan Kaufmann.

Wassén, O. (2017). 5 essential elements of data governance. Node Graph. https://www.nodegraph.se/5-essential-elements-of-data-governance/

Zatyko, S. (2015). Building a data quality team. Experian. https://www.edq.com/blog/building-a-dq-team/

Appendix A. Further Steps and Tactics to Assess DQ Readiness (Sebastian-Coleman, 2013)

1. Reviewing documentation (such as help desk tickets or break-fix project information) about data quality issues.

2. Surveying different teams to get an understanding of what activities they engage in to assess or ensure data quality.

3. Surveying teams to understand whether they feel they have the tools to successfully respond to data issues.

4. Surveying customers to understand their perceptions of the quality of the organization’s data.