Traveling

Through

Time 

The Forum Guide to 

Longitudinal Data Systems 

Planning and Developing an LDS 

National Cooperative Education Statistics System
 

The National Center for Education Statistics established the National Cooperative Education Statistics System 

(Cooperative System) to assist in producing and maintaining comparable and uniform information and data on early 

childhood education and elementary and secondary education. These data are intended to be useful for policymaking at 

the federal, state, and local levels. 

The National Forum on Education Statistics (the Forum), among other activities, proposes principles of good practice to 

assist state and local education agencies in meeting this purpose. The Cooperative System and the Forum are supported 

in these endeavors by resources from the National Center for Education Statistics (NCES). 

Publications of the Forum do not undergo the formal review required for products of NCES. The information and 

opinions published here are those of the Forum and do not necessarily represent the policy or views of the U.S. 

Department of Education or the National Center for Education Statistics. 

December 2010 

This publication and other publications of the Forum may be found at the NCES websites. 

The NCES World Wide Web Home Page is http://nces.ed.gov 

The NCES World Wide Web Electronic Catalog is http://nces.ed.gov/pubsearch 

The Forum World Wide Web Home Page is http://nces.ed.gov/forum 

This publication was prepared in part under contract number ED-06-CO-0056 with the Council of Chief State School 

Officers. Mention of trade names, commercial products, or organizations does not imply endorsement by the U.S. 

Government. 

Suggested Citation: 

National Forum on Education Statistics. (2010). Traveling Through Time: The Forum Guide to Longitudinal Data Systems. 

Book Two of Four: Planning and Developing an LDS (NFES 2011–804). Washington, DC: National Center for Education 

Statistics, Institute of Education Sciences, U.S. Department of Education. 

For ordering information on this report, write: 

ED Pubs 

U.S. Department of Education 

P.O. Box 22207 

Alexandria, VA 22304 

Or call toll free 1–877–4ED–PUBS; or order online at http://www.edpubs.gov 

Technical Contact: 

Tate Gould 

202–219–7080 

tate.gould@ed.gov 

ii Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
The Longitudinal Data 

Systems Task Force 

A volunteer task force of the National Forum on Education Statistics (the Forum) 

produced this document. It was developed through the National Cooperative Education 

Statistics System and funded by the National Center for Education Statistics (NCES) of 

the U.S. Department of Education. 

Chair 

Bruce E. Dacey 

Delaware Department of Education 

Dover, Delaware 

Task Force Members 

Nancy Burke Nancy J. Smith 

Century Elementary School DataSmith Solutions, LLC 

Grafton Public School District Vienna, Virginia 

Grafton, North Dakota 

Steve Smith Corey Chatis Cambridge Public School District (formerly of the Tennessee Department Cambridge, Massachusetts of Education) 

Bill and Melinda Gates Foundation 

Seattle, Washington Raymond Yeagley 

Northwest Evaluation Association 

Laurie Collins Lake Oswego, Oregon
 

(formerly of the SIF Association)
 

Mizuni, Inc.
 

Addison, Texas 


Patrick Sherrill 

Office of Planning, Evaluation, 

and Policy Development 

U.S. Department of Education 

Washington, DC 

Consulting Author 

Anthony Garofano 

Quality Information Partners 

Fairfax, Virginia 

Project Officer 

Tate Gould 

National Center for Education Statistics 

Washington, DC 

Book Two of Four: Planning and Developing an LDS iii 
Acknowledgments
 

The members of the Longitudinal Data Systems Task Force of the National Forum on 

Education Statistics would like to thank everyone who reviewed drafts of this document or 

otherwise contributed to its development. This includes the Forum Steering Committee 

and Technology (TECH) Committee and the members of the National Forum on 

Education Statistics. The Task Force wishes to specifically acknowledge the contributions of 

Lee Hoffman 

formerly of the National Center for Education Statistics 

Maureen Wentworth 

Council of Chief State School Officers 

Beth Juillerat and Marsha Ward 

Ohio Department of Education 

Susan VanGorden 

Lakota Local School District, Ohio 

Mark Russell 

formerly of the Idaho State Department of Education 

Khaled Falah and Tom Cosgrove 

formerly of the Washington, DC 

Office of the State Superintendent of Education 

Robert McGlaughlin 

New Hampshire Department of Education 

Richard Hezel 

Hezel Associates 

The Task Force also thanks eScholar and several of its clients (Pennsylvania Department 

of Education, Nebraska Department of Education, and New Mexico Public Education 

Department) for their insights and suggestions to improve this product. Frances Erlebacher 

provided overall editing and the Creative Shop formatted and designed this publication. 

iv Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
 

Foreword
 

The National Forum on Education Statistics (the Forum) is pleased to present Traveling 

Through Time: The Forum Guide to Longitudinal Data Systems. This document, Book Two 

of Four: Planning and Developing an LDS, is the second installment of this Forum series 

of guides on longitudinal data systems (LDS). One goal of the Forum is to improve the 

quality of education data gathered for use by policymakers and program decisionmakers. 

An approach to furthering this goal has been to pool the collective experiences of Forum 

members to produce “best practice” guides in areas of high interest to those who collect, 

maintain, and use data about elementary and secondary education. Developing LDSs 

is one of those high-interest areas. These systems hold promise for enhancing both the 

way education agencies use data to serve students and the way they do business, from the 

policy level to the school office and into the classroom. 

LDSs are increasingly becoming the state of the art in education data. These systems 

move us from relying on blunt, aggregate, snapshot student data; to detailed and timely, 

student-level data that reflect the student’s entire academic history. An LDS makes it 

possible to not only monitor the success of individual students, but also to identify trends 

in those students’ education records. Freeing educators from guesswork and lessening the 

burden of painstaking data analysis, these systems provide powerful and timely insight 

about students and allow educators to tailor instruction to better meet individual needs. 

An LDS can reveal with great clarity what effects our policies, programs, and decisions have 

on schools. These systems allow agencies to track students across institutions to facilitate 

appropriate course placement and to determine who has transferred and who has dropped 

out. Longitudinal data systems also offer a new level of sophistication at the business 

level that can streamline operations; improve data quality; and free up valuable resources 

previously allocated to inefficient data entry, maintenance, and reporting practices. 

For these and others reasons, states should continue to introduce, develop, and 

expand their LDSs. The Traveling Through Time: The Forum Guide to Longitudinal Data 

Systems series is intended to help state and local education agencies meet the many 

challenges involved in developing robust systems, populating them with quality data, and 

using this new information to improve the education system. The series will introduce 

important topics, offer best practices, and direct the reader to additional resources related 

to the LDS development process. In sum, it is intended to help agencies establish LDSs 

that will have lasting, far-reaching impact on the education system and on students’ lives. 

For a description of the entire guide series, see appendix A. 

Book Two of Four: Planning and Developing an LDS 

This second book in the guide series delves into the planning, implementation, and 

evaluation phases of a longitudinal data system (LDS) project. It guides readers through 

the process of engaging a wide variety of stakeholders to create a vision for an LDS, build 

support for the undertaking, develop the system, and gauge how well it is meeting intended 

goals. 

and conventions used throughout the series. 

	 The introduction explains the purpose of book two, as well as the format 

Book Two of Four: Planning and Developing an LDS v 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

	 Chapter 1 introduces the information life cycle, a helpful concept when

developing any data system.

	 Chapter 2 discusses the systems development life cycle, another useful

concept for developing a data system.

	 Chapter 3 discusses the people and processes that characterize effective

project management of an LDS development effort.

	 Chapter 4 provides guidance on engaging stakeholders to define the

organization’s vision for an LDS that will meet end users’ needs.

	 Chapter 5 discusses the critical relationship between the state education

agency (SEA) and districts, and offers best practices on how to strengthen 

these ties.

	 Chapter 6 discusses the self-assessment phase of the effort, during which an

agency should identify its current system and functionalities.

	 Chapter 7 presents the concept of “enterprise architecture,” a helpful

framework for planning and evaluating the agency’s current and desired data 

systems.

	 Chapter 8 discusses the needs-assessment phase, during which an agency

should define its desired system based on stakeholder requirements.

	 Chapter 9 addresses how to identify the data elements that need to be

collected to meet the stakeholders’ information needs.

	 Chapter 10 covers the important issues of interoperability and portability.

	 Chapter 11 discusses strategies for gaining sustained support for the system.

	 Chapter 12 addresses the need for, and approaches to, promoting the LDS

and gaining grassroots support.

	 Chapter 13 discusses the choice between building an LDS in-house and

hiring a vendor, addressing some of the pros and cons of each approach. 

	 Chapter 14 provides an introduction to effective request for proposals (RFP) 

writing.

	 Chapter 15 addresses system evaluation, during which an agency assesses

the “success” of the system based on many criteria. Findings should inform 

iterative system refinement. 

The appendices include an overview of the four books in this series, references, 

additional resources, an overview of information technology audits, and other relevant 

Forum and NCES resources. 

vi Traveling Through Time: The Forum Guide to Longitudinal Data Systems
The National Forum on Education Statistics 

The work of the Forum is a key aspect of the National Cooperative Education Statistics 

System (the Cooperative System). The Cooperative System was established to produce 

and maintain, with the cooperation of the states, comparable and uniform educational 

information and data that are useful for policymaking at the federal, state, and local levels. 

To assist in meeting this goal, the National Center for Education Statistics (NCES), within 

the U.S. Department of Education, established the National Forum on Education Statistics 

(the Forum) to improve the collection, reporting, and use of elementary and secondary 

education statistics. The Forum deals with issues in education data policy, sponsors 

innovations in data collection and reporting, and provides technical assistance to improve 

state and local data systems. 

Development of Forum Products 

Members of the Forum establish task forces to develop best-practice guides in datarelated areas of interest to federal, state, and local education agencies. They are assisted 

in this work by NCES, but the content comes from the collective experience of the state 

and school district task force members who review all products iteratively throughout 

the development process. Documents prepared, reviewed, and approved by task force 

members undergo a formal public review. This public review consists of focus groups with 

representatives of the product’s intended audience, review sessions at relevant regional 

or national conferences, or technical reviews by acknowledged experts in the field. In 

addition, all draft documents are posted on the Forum website prior to publication so 

that any interested individuals or organizations can provide feedback. After the task force 

oversees the integration of public review comments and reviews the document a final time, 

publications are subject to examination by members of the Forum standing committee 

sponsoring the project. Finally, the entire Forum (approximately 120 members) reviews and 

formally votes to approve all documents prior to publication. NCES provides final review 

and approval prior to publication. 

Book Two of Four: Planning and Developing an LDS vii 

Contents
 

The Longitudinal Data Systems Task Force ............................... iii
 

Acknowledgments................................................... iv
 

Foreword........................................................... v
 

Introduction......................................................... 1
 

Part I. Planning Prerequisites:
 

What to Think About Before
 

Developing an LDS .................................... 5
 

Chapter 1. The Information Life Cycle............................ 7
 

Chapter 2. The Systems Development Life Cycle ................ 11
 

Chapter 3. Project Management for an LDS Implementation . . 15
 

Part II. Knowing What You Have,
 

Seeing What You Want ........................... 19
 

Chapter 4. Engaging Stakeholders: 


Bringing Everyone Along ......................................... 21
 

Chapter 5. Building State–District Relationships ............... 25
 

Chapter 6. Self-Assessment: You Are Here, But... 


Where Exactly is That? ............................................ 31
 

Chapter 7. Enterprise Architecture ............................... 35
 

Chapter 8. Needs Assessment: Defining “There”................ 39
 

Chapter 9. Data: Knowing What You Have, 


Identifying What You Need ....................................... 45
 

Chapter 10. Some Critical “Abilities”: 


Interoperability and Portability .................................. 51
 

Book Two of Four: Planning and Developing an LDS ix 
 

Chapter 11. Staying “There”: Ensuring System 


Sustainability ...................................................... 57
 

Chapter 12. Marketing and Communicating 


About Your LDS ................................................... 63
 

Part III. Getting from “Here” to 


“There”: Developing What You Want ... 69
 

Chapter 13. Procurement Planning: Build or Buy? ............. 71
 

Chapter 14. Writing a Strong Request for Proposals ............ 77
 

Chapter 15. Are We “There” Yet? Evaluating Your LDS ........ 83
 

Appendices ......................................................... 87
 

A. Overview of the LDS Guide Series ................................... 87
 

B. References ...................................................... 89
 

C. Additional Resources.............................................. 93
 

D. Information Technology Audits .................................... 109
 

E. Forum and Other NCES Resources .................................. 113
 

x Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
 

Introduction 

T his book, Planning and Developing an LDS, is the second in a four-part 

series about longitudinal data systems (LDS). The first book, What is an 

LDS?, focused on the fundamental questions of what an LDS is (and what it is not), what 

steps should be taken to achieve a sound system, what components make up an ideal 

system, and why such a system is of value in education. The present installment discusses 

the early stages of LDS development, and will help state and local education agencies 

through the process of determining what they want to accomplish with their LDS and 

what they will need in order to achieve these goals. The organization’s vision for an LDS 

should be heavily informed by the needs of a broad range of stakeholders. Throughout 

the systems development life cycle, policymakers and system developers need to engage in 

self-assessment, identifying the system they have before figuring out what type of system 

they want. Policymakers’ requirements should be driven by the needs of the education 

community, the costs involved given the legacy system and staff, and the institutional 

support for the project. Planners should ensure project sustainability by creating interest 

and sustained buy-in, and by securing long-term funding. Procurement planning must be 

done, that is, lining up a vendor or building the staffing capacity to construct the system. 

In addition, having the right developers may not be enough: an informed commitment to 

building, using, and maintaining the LDS must permeate the organization to ensure longterm success. And, throughout the life of the system, thorough evaluation must be done 

on a regular basis to ensure continued data quality and user satisfaction. 

Figure 1 lays out the major issues discussed in each of the four books in this series. 

For more information on the purpose, format, and intended audience groups of this series, 

see Book One of Four: What is an LDS? 

Developing 

a successful 

LDS is 80% 

planning and 

20% building. 

Book Two of Four: Planning and Developing an LDS 1 
Book III: Effectively 

Managing LDS Data 

 Defining governance structure 

 Defining roles and responsibilities 

 Collaborating to improve data quality and 

streamline operations 

 Managing changes to the system 

 Training staff to ensure data quality 

 Auditing/validating data at all levels 

 Establishing/following data standards 

 Securing data to protect privacy 

 Providing users access to key data 

Figure 1. Summary of the Forum Guide to LDS Series 

Book I:What Is an LDS? 

 Understanding what an LDS is (and is not) 

 Appreciating the organizational steps needed to 

institute and effectively use an LDS 

 Identifying the technical features and 

capabilities of an effective LDS and the 

additional features that can enhance the 

system’s utility 

 Recognizing the benefits of an LDS 

Book II: Planning and 

Developing an LDS 

 Engaging stakeholders 

 Describing the current system 

 Envisioning the desired system 

 Defining needs, including data and functionality 

 Gaining buy-in and funding 

 Building relationships 

 Writing an RFP 

 Building or buying a system or components 

 Transferring knowledge (e.g., from developers 

to staff) 

 Defining and measuring success 

 Refining the system 

Book IV:Advanced 

LDS Usage 

 Collecting, storing, and delivering key data 

 Developing useful reports to fulfill common 

data requests and needs 

 Developing user-friendly data tools to facilitate 

access and analysis 

 Training users to utilize the technology 

 Building awareness, understanding, and 

analytical capacity 

2 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
Conventions 

Throughout this series, important terms and topics will be highlighted in sidebars. Notable 

subject matter will be easily identified by the following icons: 

Definitions LDS lore Key points

(fictional tales) 

Bright ideas Tips District 

difference 

Book Two of Four: Planning and Developing an LDS 3 

 

Here

Current “as is” 

Architecture

There

Ideal “to be” 

Architecture

Seeing from here to there

Planning Prerequisites: 

What to Think About Before 

Developing an LDS 

Part I 

uilding an LDS can be a daunting task. Analysts who compare a newly 

proposed system to the current information system environment often

say, “You cannot get there from here.” Certainly the technological, organizational, and 

professional gaps between the information system that currently “is,” (the “here”) and the 

ideal system we hope will “be” in the future (the “there”) can be very discouraging. 

B 

Here 

Current “as is” 

Architecture

There 

Ideal “to be” 

Architecture

Seeing from here to there 

Gap

Book Two of Four: Planning and Developing an LDS 5 
Once the current reality is understood and the “future ideal” is clearly defined, 

an education agency can undertake the formal process commonly referred to as a “gap 

analysis,” which is often performed by third party experts. The agency can then begin to 

move from the present into the future. Later in this book, the processes of self- and needsassessment are discussed to help readers identify “here” and “there.” But first, Part I reviews 

some prerequisites to the planning process. Understanding two conceptual frameworks—the 

life cycle of information and the systems development life cycle—will help developers 

and other stakeholders throughout the creation and refinement of the system. Then, the 

practical matter of establishing a project management team, and a process for driving and 

organizing the effort, will be addressed. 

6 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
 

 

 

 

 

 

 

The Information Life Cycle 

Chapter 1 

O ne of the first concepts system designers and developers need to absorb is 

that information has a life cycle that begins with its creation and continues 

until its destruction. Understanding the life cycle of information is vital to evaluating the 

information systems policymakers have, and to designing the systems they need. Although 

the life cycle process can be described in many ways, the following verbs are used here to 

enumerate its different stages: define, create, collect, store, protect, use, share, and retire. 

(Note: The framework depicted in the graphic below applies to the information life cycle 

only. This Forum guide is not organized around this framework.) 

Figure 2. The information life cycle 

Define 

Store

Use 

Collect 

Create 

Retire 

Protect 

Share 

 Define: Identify and define the data elements to be collected. 

 Create: Create data (descriptive data exist and other data are 

created by events). 

 Collect: Collect the data in the least burdensome manner. 

 Store: Store data in accessible formats for efficient access and use. 

 Protect: Secure data from individual or technological intrusion. 

Protect the privacy of individuals. 

 Use: Use data for compliance, analyses, and education 

improvement. 

 Share: Provide the public with appropriate data. 

 Retire: Archive permanent records with historical or legal 

value. Destroy electronic records with little or no value to the 

data owner. 

The life cycle of data 

Book Two of Four: Planning and Developing an LDS 7 
Define
 

Before collecting data, organizations identify and define the data elements they need 

or want in order to comply with requirements or inform decisionmaking and business 

processes. Definition of data is not a precursor to the cycle, nor is it a one-time process. 

The Forum has more detailed information... 

... about the identification and definition of data elements: 

• NCES Handbooks Online: http://nces.ed.gov/programs/handbook/index.asp 

• National Education Data Model: http://nces.ed.gov/forum/datamodel 

This stage should occur iteratively with every cycle to refine the data and their relationships 

so they better meet evolving needs. Only data that will be used should be defined (see 

chapter 9). 

Create and collect 

While some information already exists and will remain relatively constant over time (for 

example, student’s name, birth date, race, sex), other data are the products of events or 

activities (tests, course enrollments, discipline incidents, health events, etc.). Existing data 

will need to be added or crosswalked to the LDS, and new data will need to be collected 

as students progress through the education system. Thoroughly understanding the nature 

of the various types of data to be entered into the system and the collection processes 

that will be used is essential for recognizing the quality—completeness, timeliness, and 

accuracy—of the data, and thereby determining whether or not the data will be usable. It is 

also important to consider the burden and costs of acquiring and entering the data and the 

skills required to do this critical work (see chapter 9). 

Store and protect 

Determine the storage requirements for the data and the levels of protection they will 

require. The risk of exposure will vary based on the contents of the records. Risk has 

two components: the amount of harm that will be done if the data are accessed by an 

unauthorized party and the likelihood that this might happen. If the content of the records 

is such that little or no harm will be done if the system is breached, then the risk can be 

8 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
considered low even if the likelihood of unauthorized use is high. But as the potential for 

harm increases, systems must provide higher levels of protection protocols to reduce the 

likelihood of unauthorized access to the data. 

Use 

Systems are built to facilitate the use of their data to improve the organization’s work 

and the students’ educational outcomes. The users of the data in the system should 

be the system builder’s primary “customers.” How the data are to be used, presented, 

and refreshed are just a few of the considerations that call for extensive requirements 

discussions. (Note: The purposes for which the data will be used should also be a central 

consideration in the “define” stage of the cycle.) 

Share 

How, when, under what circumstances, and with whom (individuals, organizations, other 

systems) the data will be shared is another set of questions to be deliberated. Sharing 

data often has legal and policy implications, such as freedom of information and privacy 

requirements. Clearly articulating all of these requirements, and the appropriate business 

rules to be followed, is necessary for legal and ethical compliance. 

Retire 

One of the last decisions in the life cycle of information comes when specific data cease to 

be accessed and used for the purposes for which they were originally collected and stored. 

When they are dormant yet still occupying valuable storage space, a decision must be 

made whether to archive or destroy the data. Some data by their nature are “eternal” and 

need to be properly and securely archived in case they are ever needed again (for example, 

transcript and financial data). Other records will eventually lose all of their value and 

should be destroyed in a manner consistent with their sensitivity. 

Before the design and development experts for any information system begin their 

work, system owners and planners should thoughtfully review all aspects of the life cycle 

of the information they propose to collect, store, and use. A thorough understanding 

of current information handling processes will provoke insights and suggestions for 

improvements and uncover difficulties that might otherwise not be discovered in a timely 

manner. Ultimately, the value and utility of the data will be greatly compromised if 

policymakers, system planners, and developers do not understand the whole life cycle and 

fail to ensure that the information is protected, collected, stored, and used correctly. 

Book Two of Four: Planning and Developing an LDS 9 

 

The Systems Development 

Life Cycle 

Chapter 2 

D Developing anything more than a simple data retrieval system is a multiyear 

project. The number of system users, their requirements, the network of 

relationships, the complexities of ever-changing technology, and the personal politics 

involved with any system increases the importance of sound project management and an 

in-depth understanding of the life cycle of a systems development project. While an LDS 

is not just a project—it requires ongoing maintenance, use, training, etc.—thinking of LDS 

development as a project in its early phases can be useful. 

The systems development life cycle is “a project management 

technique that divides complex projects into smaller, more easily 

managed segments or phases.” 

Source: Federal Financial Institutions Examination Council. 

Development and Acquisition. Available at 

http://www.ffiec.gov/ffiecinfobase/booklets/d_a/08.html. 

Many words can be used to name the stages and substages in an information systems 

development effort, but for the purposes of this guide, the following verbs will be used to 

describe the whole life cycle: plan, analyze, design, develop, test, deploy, maintain, and 

evaluate. (Note: The framework depicted in the graphic on the following page applies to 

the systems development life cycle only. This Forum guide is not organized around this 

framework.) 

Plan 

The importance of thorough planning cannot be overemphasized. And neither can 

the need to take as much time as necessary to implement the plan successfully. For the 

purposes of this guide, “planning” includes working with a broad range of stakeholders 

who may be affected—or believe they may be affected—by the LDS in order to articulate 

the goals of the new system (see chapter 4), as well as assigning roles and responsibilities 

for project management (see chapter 3). Developing a communication plan that includes 

Book Two of Four: Planning and Developing an LDS 11 
 

 

 

 

 

 

 

 

 

Figure 3. The Systems Development Life Cycle Design 

Deploy Evaluate 

Test 

Develop 

Analyze 

Maintain 

Plan 

 Plan: 

•	Articulate	system	goals	and	establish	a	high	level	view	of	the	project.

•		Define	responsibilities	and	roles.	

•	Define	project	deliverables,	budget,	and	schedule.

 Analyze: 

•		Refine project goals and translate them into functional activities. 

•		Analyze	system	users’	information	needs.	

•		Determine	project	risks	and	risk-mitigation	strategies.

 Design: 

•		Describe desired features including screen layouts, business rules, 

process diagrams, system documentation, and collection methods. 

 Develop: 

•		Establish coding standards and naming conventions. 

•	Write	code	to	automate	processes.	

•	Create	documentation.

 Test: 

•		Develop testing plan, testing requirements, and testing schedule. 

•		Run	user	acceptance	tests	and	obtain	feedback.	

•	Correct	defects	and	complete	documentation	manuals.

 Deploy: 

	•		Develop schedule and install applications into the production 

environment. 

•		Run	the	new	system	parallel	to	the	old	one.	

•		Verify	data	quality.

•	Establish	a	hard	cutover	date	(a	day	of	transition	from	old	to	new	system).

 Maintain: 

•		Develop an update schedule for hardware and software, as needed. 

•		Maintain	documentation.	

•	Perform	quality	assurance	and	system	security	audits.

 Evaluate: 

•		Monitor implementation and measure benefits. 

•		Assess	system’s	ability	to	meet	users’	initially	identified	and	evolving	

information needs. 

•		Identify	necessary	changes	and	return	to	planning	stage	of	life	cycle	to	

implement them. 

Developing 

a system 

all individuals and groups that may be affected by the education information system is 

essential. System development efforts might be seriously affected if a stakeholder becomes 

an active opponent of the effort. These potential surprises and other risks should be 

articulated and evaluated during the planning process. Data quality, sharing, and security 

strategies should be discussed—as well as the stakeholders and risks associated with these 

strategies. General goals should be defined by project deliverables with a set schedule and 

budget for each separate component as well as for the whole. At this point, the scope of 

the project—what is and is not to be included—should be clearly described and a method 

for managing changes established to monitor and control scope creep. In the planning 

stage, the current environment should be analyzed (see chapters 4, 6, and 7) and a clear 

picture of the future information system should emerge (see chapters 7–8). Anything 

12 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
missed in the planning stage may go unnoticed until it is too late to fix or, at best, a return 

loop to the planning stage is required. Every project returns to the planning stage in the 

iterative process of system development; while this is unavoidable, the number of times 

repeating this stage is required can be reduced through thorough planning and by using the 

systems development life cycle the first time. 

Analyze 

Analyzing the information needs of all system users is a long and tedious process. It 

requires great patience and repetition to generate the input necessary to turn vague ideas 

of what would be useful information into definable and actionable system requirements. 

This is where the project goals and deliverables become defined in functional process 

terms. This is also where the project is critically examined to articulate and define the risks 

associated with the work to be done and what resources are needed. Each of these risks 

should be classified as high, medium, or low; and assigned a strategy for mitigation in case 

it materializes. Consideration should also be given to how data quality and security will 

be ensured (see chapters 4, 8, and 9). It may be useful to consider existing working models 

(such as successful in-house-developed and “off-the-shelf” systems), while focusing on the 

most important ways the information in your desired system will be used and reported. 

Sharpening your understanding of the desired “there” system will make your analysis much 

more effective and shorten this stage of the process considerably. 

The Forum has more detailed information... 

... about these issues 

• Forum Unified Education Technology Suite (2005) 

http://nces.ed.gov/forum/pub_tech_suite.asp 

• Forum Guide to Decision Support Systems: A Resource for Educators (2006) 

http://nces.ed.gov/forum/pub_2006807.asp 

• Technology @ Your Fingertips (2001) 

http://nces.ed.gov/pubs98/tech/index.asp 

Book Two of Four: Planning and Developing an LDS 13 
Design 

In the design stage, all the mental pictures created in the earlier stages are put into clear 

and completely documented forms. Business rules are articulated and refined, screen 

layouts are developed and improved, process diagrams are drawn and redrawn, and system 

documentation is carefully and completely kept. As with the previous two stages, the 

design stage often exposes issues or problems not previously discussed. This requires a 

return to the earlier stages in order to plan and analyze the newly exposed concerns. 

Develop 

When the design documents are complete, they are given to the developers to write the 

code that will automate the processes and produce the desired result. Coding standards 

and naming conventions are but two of the many considerations that must be made by the 

system developers. 

Test 

During the development stage, the testing stage begins and testing plans, user acceptance 

tests, and a testing schedule are developed. These tests are run, shortcomings are 

discovered, and the system is modified to eliminate any defects. Any time or effort avoided 

at this stage may come back multiplied many times if a system defect survives to plague the 

system’s end users. 

Deploy 

In the deployment stage, the new system—with its hardware, software, and applications—is 

installed and often run parallel to the old system until the acceptance criteria established 

earlier (in the plan and analysis phases) are met. System security is deployed and tested in 

real time situations, and data quality is checked and verified. 

Maintain 

When the system has met all required tests, it enters the maintenance stage. The system 

operations team performs periodic quality assurance and system security audits, updates 

hardware and software as needed, and maintains documentation. The maintenance 

stage continues as long as the system is operating and serves not only to keep the system 

running, but also to make incremental improvements. 

Evaluate 

The data system should be evaluated throughout its life cycle to ensure it is being 

implemented successfully, working optimally and cost effectively, providing measurable 

benefits, and meeting user needs as initially identified in the planning stages and as they 

evolve over time. Newly identified requirements will often warrant system changes beyond 

the level of “incremental,” requiring serious monitoring and evaluation that will produce 

system change requirements. Such changes should be routed through the entire systems 

development life cycle process. When it is determined that the existing system is no longer 

as effective or efficient as it should be, the system owners will go through the systems 

development life cycle again, beginning with planning for improvements to the existing 

system or the development of a new system. 

14 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
 

Project Management for an 

LDS Implementation 

Chapter 3 

P roject management is a finite process, in which an organization launches, 

oversees, and concludes a project. As the management and enhancement 

of an established LDS is an ongoing endeavor that will support the education agency and 

external stakeholders, these ongoing stages should not be viewed as long-term projects. 

However, approaching the initial planning and development phases of the effort is 

appropriate. In these phases, the system is planned along a timeline, roles and deliverables 

are identified, resources are allotted and managed, communication is coordinated, and 

progress is overseen. Project management is especially important in LDS development 

because the effort touches numerous program areas within the organization, and 

perhaps many outside contractors as well. Therefore, it is vital to define the roles and 

responsibilities of the various stakeholders, their relationships, and timelines for each 

task. Effective project management will help the agency maintain a clear scope, manage 

expectations, and deliver components and functionalities successfully and on time. A clear, 

organized process is critical in such a complex project and will enhance the resulting LDS 

in terms of operations and ultimate utility. (Tennessee Department of Education) 

The Team 

Early on, a project management team should be established: a core group of education 

agency staff (and perhaps external participants) who will plan and manage the LDS effort. 

Though the makeup of this team will vary by agency and project, ideally its structure 

should be kept simple and include key leaders selected for their expertise and authority. 

 Executive sponsor 

A high-level authority within the organization (e.g., commissioner, assistant commissioner, 

or deputy commissioner) gives the project traction and deals with political issues that may 

impede or even doom the project. Specific responsibilities may include approving overall 

project scope and budget, securing funding for project costs, and overseeing project phases. 

Book Two of Four: Planning and Developing an LDS 15 
 

 

 

 

 

 

 Project manager 

The project manager (PM) drives the project management process. This leader should 

develop the project scope statement; define project deliverables, milestones, budget, and 

detailed work schedule; and assign project roles and responsibilities. In addition, the 

PM should establish the project management team, work with the sponsor to review the 

project plans, drive and lead team meetings, ensure that team members understand and are 

fulfilling their roles and responsibilities, and ensure that the project is ultimately completed 

in accordance with agency and external stakeholder expectations. While the agency’s 

chief information officer often fills this role, another agency leader may be selected as PM 

instead. (Tennessee Department of Education) 

 Data governance coordinator 

The leader of the data governance process works with the team to ensure the project is 

driven by education and programmatic needs rather than by IT. The coordinator’s primary 

responsibility should be to ensure the appropriate data managers are involved in the 

development and implementation phases as they are needed, for example, when the data 

they manage are rolled into the data warehouse or made available to users. 

 Program area data manager 

This standing seat on the team should be filled by different data managers as needed. 


As various data are incorporated into the system or involved in project activities, the 


appropriate manager should be engaged to ensure, for instance, that programmatic needs 


are reflected in the data model, definitions, and reports created from the LDS.
 

 Core information technology staff 

In-house IT leaders either implement the system or, perhaps, work with vendors and 

sustain the system once outside developers have finished their work. 

 Vendor project manager 

If the agency is buying an LDS (in part or whole), the vendor project manager should keep 

the project on track and the agency up to speed on progress. 

 Database administrator or key technology lead 

If the LDS will be populated from existing data sources within the agency, a lead database 

administrator (DBA) or key technology lead should participate on the project management 

team as needed. The DBA understands how the source systems work, including their 

limitations and problems, and can help ensure effective incorporation of disparate data 

into the LDS. 

The Process 

The project management team should meet frequently, perhaps on a weekly basis, for 

planning, status updates, and discussion of problems and concerns. The following outlines 

some of the team’s key activities. 

 Planning the project scope 

In this first phase of project management, education goals should drive design—define 

your enterprise’s questions and requirements before developers build solutions. Engage 

stakeholders to define your current system as well as your desired system in terms of 

architecture, content, access, and use. Consider the work that will be necessary to ensure 

the quality of the data contained in the system, as well as the steps necessary to ensure 

data security and the privacy and confidentiality of sensitive student and staff records. 

16 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
 

 

 

 

 

Devise a plan for evaluating the system’s success throughout the process, including the 

metrics that will be used to measure this success. Additionally the team should, based on 

broad stakeholder input, clarify how the system will ultimately be used, the training and 

professional development that will be needed, and the types of reports and technology 

solutions that will most help end users. 

 Identifying and prioritizing specific goals and deliverables 

Determine what needs to be done, and in what order, for each phase and portion of the 

development process. 

 Assigning work and roles 

Assign roles and responsibilities early on. This includes both in-house staff and external 

vendors, who should be assigned very clear roles and deliverables. 

 Monitoring work and deliverables 

Supervise the project work closely to ensure that activities are on track and to clear 

unexpected hurdles along the way. 

 Identifying critical issues 

Team members should communicate major problems that might affect implementation, 

and work together to find solutions as appropriate. Project management meetings should 

have a standing agenda item to identify these issues, determine a plan of action to address 

them, and update the team on progress towards the resolution of previously identified 

concerns. 

 Managing communication 

Keep stakeholders informed and involved in the LDS development and implementation 

effort in order to build and maintain interest. This will also help the team collect adequate 

input to create a system that meets user needs. Communication should not be limited to 

the project management team; rather, it should include coordinating updates about the 

project to the entire agency and all relevant stakeholders (legislators, state board, school 

districts, etc.). 

Book Two of Four: Planning and Developing an LDS 17 

 

Part II 

Knowing What You Have,

Planning What You Want 

efore an education agency begins developing a new system—writing code or 

an RFP—time should be taken for reflection or “discovery.” That is, before

beginning its LDS journey, an agency should figure out where “here” is (what it has) and 

then carefully plan where “there” will be (what it will have). This often overlooked phase 

should begin at the start of the LDS development process with the engagement of a broad 

range of stakeholders, a thorough and collective self-assessment, and the identification of 

what the current system and environment look like. The resulting information, of course, 

will vary widely among organizations in terms of technology, applications, data, politics, 

resources, and other relevant factors. Once an agency has clearly established its “here,” it 

can pinpoint where it wants to go by thoroughly assessing stakeholders’ needs, then move 

on to carefully planning their desired system. When the self- and needs-assessments are 

complete, agencies should compare their “here” and “there” analyses to figure out how the 

current data system and organizational culture will need to change in order to fulfill the 

stakeholders’ stated needs. 

The following chapters address these crucial early stages of the LDS development 

process. While the activities of engaging stakeholders, conducting self-assessments, and 

identifying needs are commonly referred to collectively as “needs assessment,” they are 

disaggregated here and discussed as distinct yet interrelated parts of the planning process. 

B 

The early stages of “discovery” should not be rushed, so 

please drive slowly across the “Here to There Bridge.” States 

that have done this agree that a year allows ample time 

for thorough self- and needs-assessment—to solicit broad 

stakeholder input and to carefully plan a new system. 

Book Two of Four: Planning and Developing an LDS 19 

 

Engaging Stakeholders: 

Bringing Everyone Along 

Chapter 4 

E arly on, an education agency should pull together a broad range of 

stakeholders in a collaborative effort to define the organization’s LDS 

vision. Because a wide array of groups have a stake in the system, this early stage is critical 

for two primary reasons: it increases the usefulness and relevance of the system to users; 

and it increases the visibility and use of, and demand for, the system across the education 

community. 

Without diverse input from a range of perspectives, the resulting system may not be 

useful to all who should benefit from it. And if the system is not relevant to certain groups, 

it will not be used. Engaging users early in the design process increases the likelihood they 

will value and utilize the resulting system and, since they were given the opportunity in 

the design stages to ask for certain information, it will hopefully be more relevant to their 

efforts to improve student outcomes. Involving stakeholders in the LDS design process also 

serves as the first step in marketing the system (see chapter 12). This process educates users 

and gets them thinking about the system’s potential while spreading excitement, increasing 

buy-in, and fostering lasting executive and grassroots support for the project. 

Mix it up 

Enlist stakeholders who vary in terms of: 

•	interest	in	the	project;

•	expertise	(e.g.,	business	and	technology	specialists);

•	responsibilities	(e.g.,	data	collectors	and	data	providers);

•	level	of	government	(e.g.,	state,	regional,	or	district);

•	geography;

•	benefits	to	be	gained;	and

•	perspective	(e.g.,	insiders	and	outsiders).

Book Two of Four: Planning and Developing an LDS 21 
See the forest and the trees 

In addition to talking about data assets and needs, leaders should carefully 

emphasize the LDS goals: first, to provide education practitioners with the 

information	they	most	need	and	want;	second,	to	inform	policy	development	

and	resource	allocation;	and,	third,	to	ease	accountability	and	reporting	

burdens. The culture of data collection for compliance must be overcome and 

stakeholders should focus on how the data can help them improve education 

from the bottom up as well as from the top down. 

A Model for Engagement 

An effective model for bringing stakeholders together involves creating several individual 

standing committees and stakeholder groups that hold periodic information-gathering 

meetings. The number of groups necessary to accommodate all interested parties, and the 

means of bringing participants together (in one central location, at regional sites, or via 

telephone or online conferences), will depend largely on the size of the organization, the 

geographic size of the state or district, and available resources. 

In these meetings, participants should identify their data concerns, define needs, 

and pinpoint aspects of the “here” system that need improvement. The groups may study 

other systems, learning what other agencies’ data systems look like and how they are being 

used. Such analysis can provide examples that can be used as models, starting points, or 

outcomes to avoid (see chapter 8). 

In addition to these meetings, consider using a variety of information-gathering 

strategies such as in-person and virtual focus groups, interviews, roundtable discussions, 

or paper or online surveys (Wilson and Nunn 2007). Also, take advantage of already 

established groups that may be able to help define LDS requirements and facilitate 

communication. If there are relevant task forces, working groups, or data-user groups 

already in existence, for example, ask them to carve out some time for LDS discussions. 

A representative from each stakeholder group should also serve on a core committee 

that should meet frequently to share findings from each group’s meetings. This core 

committee should play a continuing role in overseeing and facilitating communication 

about the LDS planning process, and in fostering a “living system” by ensuring ongoing 

feedback on how to design and improve the project so it meets stakeholder needs. 

Innovation requires everyone to step out of the box and consider how the system 

can work for them and what types of information would help them be more successful in 

their jobs. Participants should be encouraged to be bold and creative in their suggestions. 

Create a collaborative environment where all stakeholders feel comfortable contributing as 

equals. 

22 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
Who to Engage 

The stakeholders invited to contribute to the early design process should vary in terms of 

their interest in the project as well as their expertise, responsibilities, geography, and the 

ways in which the system can benefit them. Include insiders familiar with education data 

and the workings of the agency, as well as outsiders who can provide a fresh perspective. 

Invite both the tech savvy and those who know the business end of the enterprise. In 

addition, involve those who collect and provide the data as well as those who use them. 

Stakeholder engagement should also acknowledge the need for a change in “data 

culture” in the education community. While data systems have historically been built 

primarily for compliance, the recent shift of emphasis towards using data to inform 

decisionmaking, improve educational strategies, and enhance student learning requires that 

these systems take local educators’ needs into consideration. For these reasons, education 

agencies should also be sure to include, in addition to state-level personnel, ample 

representation from schools and districts in the planning process. 

Contact other agencies, organizations, or other potential stakeholders who may be 

interested in the system and invite them to participate. Include as many groups as possible 

at first, and let participants decide whether to continue attending meetings. This will give 

everyone an introduction to the project and allow anyone with a special interest in the 

endeavor to become more involved. A word of caution: while it is beneficial to include 

a broad collection of stakeholders through these planning stages, there is such a thing 

as “too much” input and such inclusiveness might risk hampering the project. Setting 

goals, ground rules, and strategies early on for handling ideas efficiently will help keep the 

process moving and on track. 

Table 1 lists the types of stakeholders that might be enlisted in the planning stages of 

the LDS project. 

Book Two of Four: Planning and Developing an LDS 23 
Table 1. Stakeholders who might be involved in LDS planning 

Internal  Elected officials  Secretaries 

stakeholder  Legislative and governor’s staff  School counselors 

groups (state,  Governing boards  Librarians/media specialists 

district, and  State education agency program coordinators  Program area experts 

school)  District superintendents and assistant •	Special	education	directors

superintendents 	•		English	language	learner	program	

 Chief information officers directors 

 Public information officers •	Title	I	coordinators

 Local accountability officers •	Title	III	coordinators

 District-level data stewards •	Gifted	and	talented	education	

 Content supervisors coordinators 

 Human resources staff  Information technology staff 

 Early learning coordinators •	Project	managers

 Guidance services directors •	Hardware	engineers

 Curriculum/instruction staff •	Software	developers

 Career technical/adult learning staff •	Network	engineers

 Teacher certification staff •	Database	designers

 School administrators (principals and  Graphic display experts 

directors) 

 Teachers 

 Registrars 

External  Advocacy groups  PTA representatives 

stakeholder  Children’s services  Parents 

groups  School board members  Media/press 

 Teacher retirement board  Researchers 

 Institutions of higher education  Community members/the “public” 

 Business and industry 	Vendors

 Support organizations  Other state agency representatives (e.g., 

 Union representatives Department of Labor) 

24 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
 

Building State-District 

Relationships 

Chapter 5 

W hy are state-district relationships so important, especially in the case of LDS 

development? Why are such relationships beneficial to both states and 

districts? How does a good relationship improve the quality and usefulness of the LDS? 

How can these relationships be forged and fostered? How can states get districts to buy 

into the process? This section offers best practices and insight from the states on these 

issues. 

Building Bridges and Gaining LEA Buy-in 

As discussed earlier, LDS development should involve all stakeholders (see chapter 4). 

Districts are an important part of this group and their engagement is vitally important 

to the creation of a successful, statewide LDS. In some situations, a culture shift may be 

needed to move to a more inclusive, collaborative environment. A strong relationship 

between a state and its districts offers many advantages in the development and use of 

LDSs, whether the systems are statewide or built locally to serve district or regional needs. 

Too often, the relationships between states and districts are very limited, and collaboration 

is an unfamiliar, even unwelcome idea. Communication barriers between these levels 

often have serious consequences, including mistrust or frustration, poor data quality, 

misunderstandings, or the establishment of unrealistic or unachievable requirements. 

The benefits of a good district–state relationship flow both ways. State agencies 

developing statewide LDSs can gain tremendous insight from local leaders, who can 

benefit from sharing their knowledge by getting a system that better meets their needs and 

makes their work more efficient and effective. Districts interested in building their own, 

local LDS may learn from the state. And the opposite may be true: local districts may have 

a more sophisticated system than the state’s and can share their experiences with their state 

counterparts. In either case, state and local agencies should work together to build a single 

system, or systems that will work together, to serve common goals, facilitate simple data 

transfer, and create only one version of the “truth.” (DQC 2006a) 

Book Two of Four: Planning and Developing an LDS 25 
Keeping districts in mind: 

Considerations for state and regional 

agencies building an LDS 

 Consider the varying needs of your districts. Be flexible enough to serve both large 

and small districts, as well as districts with varying degrees of experience with data 

systems. 

 Consider the extra burden being placed on districts to report data elements. 

 Involve districts in the entire process. Find out what local leaders need from an LDS. 

•		What	questions	need	to	be	answered	at	the	district-,	school-,	and	classroomlevels? What data are needed to answer them? 

•		What	access	is	needed	at	the	district-,	school-,	and	classroom-levels?

•		What	reports	and	tools	will	help	local	staff	do	their	jobs	better?

 Consider interoperability with existing systems in districts in terms of standards and 

common definitions. 

 Consider the possibility of providing added value back to districts as incentive to use 

the system (for example, combining the state-hosted student information system with 

the LDS). 

 How will the LDS address the need to collect and analyze local assessment data over 

time? (These assessments can be unique to schools and districts.) 

 How will districts exchange student records (for example, release to a transfer 

student’s new district)? 

With a statewide LDS project, states should think of districts as partners, rather than 

as customers. As such, the LDS should be conceptualized and developed not as something 

that districts simply need to comply with, but as a valuable tool that will benefit both the 

state and the districts. The state should think of districts not only as providers of data, 

but also as users who will benefit from data sharing and access. And conversely, districts 

should not think of the state only as a collector of data, but also as a provider of useful 

information. This sort of state–district partnership can be cultivated in several ways. 

Engage LEAs early 

Ideally, local education agencies (LEA) will buy into the LDS development process because 

they see its value, and their involvement will be voluntary. State communication with 

districts about the LDS should be framed in terms of its value to the districts, and local 

engagement should be encouraged as early in the process as possible. 

For example, district representatives should be included from the start in committees 

or working groups focused on LDS oversight and development. Small states may seek 

to include all districts in this process, while larger ones might choose a sample group of 

diverse, representative districts. In this process, states should try to gauge districts in terms 

26 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
of expertise and the time they will have to help with the project. States may also involve 

districts in the grant-writing process or the RFP committee, both to get them engaged and 

to gain input. District representatives should also be involved in the state’s data governance 

process. In terms of data collection, for example, when a state proposes the collection of a 

new data element, participating LEA representatives will be able to offer input. The state 

may also ask all of its districts for feedback. Ample time should be allowed for response, 

perhaps up to a year prior to the collection of the new element. This way, the LEAs will 

know what is coming well in advance, and their feedback can be incorporated to make data 

collection as smooth as possible. 

Moreover, ask local representatives what kinds of questions they want answered 

(see chapter 5 of Book One: What is an LDS?). Additionally, how do they want the data 

returned to them? What information will most alleviate their pain points? Along the way, 

find out what the districts think of certain aspects of the system. For instance, in the portal 

or business intelligence tool development phase of the project, what types of tools will be 

most useful? How can certain reports be made more useful or user friendly? Flashy tools 

are nice, but simple low-tech improvements that save time are often hugely appreciated 

and can strengthen support for the project. 

Respond to LEA feedback with action 

Many states cite the importance of not only soliciting and considering LEA feedback, but 

also responding with action. For instance, if districts say some areas of the data collection 

are problematic, the state might seek ways to resolve the issue by adjusting the collection, 

making it optional, or even eliminating it if necessary. Districts should not feel ignored; 

this can lead to alienation and even cynicism and mistrust. However, while state agencies 

should be responsive to districts, they should also strike a balance and make only promises 

they can keep, offering a realistic view of what can be accomplished in both the short- and 

long-terms. All parties will benefit from an open and frank relationship. 

Return the data to the district 

Districts submit large quantities of data to states. This responsibility can be burdensome 

and may seem without reward. State agencies should strive to quickly return data to the 

districts in a useful form. If the districts are able to review and use the data for analysis, or 

see them in reports that show comparisons among schools and districts within the state, 

this reinforces their importance as more than just a troublesome requirement. Showing 

the districts how the data are used, and giving them a chance to use the information for 

their own needs, is an excellent incentive for cooperation from the districts, encouraging 

them to devote more energy to submitting high quality data. States can enhance this 

incentivation by seeking to collect data that the districts find most useful. 

Maintain district engagement 

District involvement should continue throughout the development process. Some 

states have brought some or all districts into the product testing phase by conducting 

pilot studies with them. Districts get a chance to try the system and give feedback to 

the state. One large state, for instance, included 10 percent of its districts in one such 

study. Continued engagement should also be sought through regular communications, 

as well as through training and feedback sessions that cover system uses and benefits. 

Book Two of Four: Planning and Developing an LDS 27 
In such training, district staff can be shown why it is vital they provide the necessary 

data accurately and on time. Some states have also strengthened their relationships with 

districts through financial means. According to the Data Quality Campaign (DQC 2006b), 

providing funds, perhaps by earmarking a percentage of state funds for districts to support 

infrastructure development, has been a source of good will. In some cases, when buy-in is 

not forthcoming and districts are resistant, states may seek ways to incentivize data quality 

and timely submission of data. One approach that has been considered is tying funding to 

data quality and timely submission. Other states have had high-level executives send letters 

to districts stressing the data’s importance. 

Strategies to facilitate state–district communication 

LDS lore: 

Leveraging past partnerships
 

In the past, the state’s districts were 

grouped into regions, with a regional


department servicing districts in each


region. After budget cuts, many of the 


staff in regional departments were moved


to the state education agency. Since 


then, the state has taken advantage of

relationships that were fostered by the


regional service centers. Communication 

continues between the same personnel at


the state and the districts they served, and 


these relationships continue to benefit the 


system. 

28 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
Having existing relationships between the state and districts prior to the start of an LDS 

project is, of course, better than starting from scratch. Some state agencies have program 

area staff who already have close working relationships with the districts. They have 

leveraged these relationships in building the different pieces of the system in order to 

meet the needs of the program areas and the districts. These staff members now serve as 

intermediaries with the districts. 

Such history may be ideal, but many states and districts do not have these working 

ties, and any connections they do have can stand to be strengthened. States pursue 

various strategies to bridge this divide, but regular communication is fundamental. Some 

periodically send a newsletter to the districts, discussing specific data issues and reviewing 

progress and successes in the state’s LDS development. Others hold regular conference 

calls, online sessions, and face-to-face meetings with district staff. Small states should 

consider face-to-face meetings with districts, while larger ones may find it more feasible 

to work with regions rather than with many individual districts. Districts also vary in their 

capacity to participate in such efforts, both in terms of expertise and time, so states should 

try to assess their districts and be flexible. 

Communication with staff from all levels should be sought so that a consistent view 

of what is happening in the state is shared by everyone. The least informed groups can be 

the biggest challenge because they do not understand the issues involved. Thus, educating 

these groups is vital. In general, educating as many people as possible is essential because a 

single individual may not disseminate information to any of his or her colleagues. 

Districts should also be represented on committees, advisory boards, and working 

groups; this will help spread knowledge and facilitate communication about data issues, as 

well as build relationships among staff and other stakeholders. States might also facilitate 

communication by making it easier to contact staff members. One state, for instance, is 

creating a data area-specific directory of district and state staff that will make it easier for 

the different agencies to communicate with one another. The directory will also identify 

which staff are in charge of which data areas, allowing questions to be directed to the right 

people and increasing efficient communication. 

Another option is to use a third party to bring districts and the state together. One 

state used a consulting firm to help forge and strengthen its SEA/LEA relationships. 

Book Two of Four: Planning and Developing an LDS 29 

 

Self-Assessment: You Are Here, 

But...Where Exactly Is That? 

Chapter 6 

everyday reality. But to get the LDS project off to a good start, you and your stakeholders 

What is your current, or “here,” data system? The answer might seem obvious 

since, after all, this is the current operations of the organization—the 

should step back and carefully depict your system environment and capabilities. The 

results may surprise you. 

Invite representatives from a wide range of stakeholder groups (see chapter 4) to look 

at the organization’s current data system and data use practices. Though some technical 

staff should be involved, an understanding of the nuts and bolts is not required. On 

the contrary, the most important input in self-assessment will come from those who are 

involved with the day-to-day business operations and goals of the organization. 

Self-assessments can be carried out in several ways, such as through an LDS 

steering committee, advisory board, or working group; personal or group interviews with 

stakeholders; written questionnaires; and focus groups. Look at what system components 

and functionalities exist currently and what developments are under way. Table 2 offers 

questions to be answered in the self-assessment process. 

Table 2. Self-assessment sample questions 

Book Two of Four: Planning and Developing an LDS 31 

What does your 


data system look 

like?
 

 Do you have a data collection system? Is it web-based? 

 How do you collect the data (via paper, electronic transfer, etc.)? How often are they 

collected and updated? What is the path of data collection (e.g., from schools to districts 

to state)? 

 How granular are your data (individual- or aggregate-level)? 

 Do you have a unique student identifier system? Can you use it to match records across 

databases? If not, do you use Social Security numbers? In which databases are these 

identifiers used as the primary ID? 

 Do you have a unique teacher identifier system? 
32 Traveling Through Time: The Forum Guide to Longitudinal Data Systems

 Are student and teacher data linked? If so, how (individual IDs, classroom IDs, etc.)?

 Are your data linked across years?

 Are your data integrated with postsecondary, workforce, social services, or other data outside 

of K–12? If so, how often is a match-rate analysis conducted? How good is your match rate?

 Are the data linked across state borders?

 Do you use electronic transcripts to share student information?

 Are your systems interoperable?

 Do you have a central data warehouse or do programs use individual silos?

 What infrastructure and technology support the system (servers, software, etc.)?

 What parts of the system are run in-house? By a vendor?

 How are data secured?

What data do 

you collect?

 What data do you collect on students (enrollment, demographics, test scores and information 

on untested students, program participation, course completion, graduation, etc.)?

 Are these data matched for students from year to year? 

 How often are students tested in each subject and can testing data show annual growth for 

students in any subject?

 What data do you collect on teachers (certification, professional development, Highly 

Qualified, salary, etc.)?

 What other data do you collect on the education system (financial, facilities, etc.)? At what 

levels do you collect these data (e.g., school, district)?

 When are these data collected and who provides them? (Catalog all current and planned 

data collections.)

How is data  What types of quality assurance processes and audits are utilized to ensure data quality?

quality ensured?  Are business rules in place?

 Do automated data edit processes ensure compliance with the business rules?

 What governance structure ensures data quality?

 Are common course codes used? Is there a central, authoritative data dictionary?

 Does a data model depict the data environment?

 Is a staff training program in place at the local level to improve data quality?
How are your  How are data transmitted to the state or federal government? 

data used?  What reports are produced with the data? 

 How are these reports made available to users? 

 What ad hoc querying is available? 

 Is access to data role-based? Who is allowed access to what data? 

 How are the data presented to users (spreadsheets, web-based analysis tools, digital 

dashboard, etc.)? 

 What does staff do with the data (federal and state reporting, program performance 

monitoring, student tracking, data-driven decisionmaking, etc.)? 

 What types of professional development are provided to help staff access, analyze, and 

interpret the data? 

What other  Do any federal or state laws and regulations control the collection and use of individual 

factors affect student data and protect the privacy of student records? 

your data?  Do any federal or state laws prohibit or mandate linkages between P–12 data systems and 

postsecondary or other outside databases? 

 Do any federal or state laws or regulations require the data system to have certain 

components? 

 What is the culture of your organization in terms of data use, data sharing, and 

collaboration? 

The next chapter introduces enterprise architecture, which is a process used to 

systematically conduct both self- and needs-assessments, as well as to guide efficient and 

effective system development thereafter. Whether an organization follows this rigorous or 

a less formal process, findings from the self-assessment should be carefully documented 

before moving to the next stage of information gathering and planning: needs assessment. 

Book Two of Four: Planning and Developing an LDS 33 

 

Enterprise Architecture 

Chapter 7 

E nterprise architecture (EA) is a planning and analysis tool that can help 

education agencies through the self- and needs-assessment stages of the LDS project (see 

chapters 6 and 8). Various complex definitions of EA are available, but this guide follows 

the Microsoft Corporation’s description of enterprise architecture as “a conceptual tool 

that assists organizations with the understanding of their own structure and the way 

they work. It provides a map of the enterprise and is a route planner for business and 

technology change.” 

Enterprise architecture is “a conceptual tool that assists 

organizations with the understanding of their own structure and the 

way they work. It provides a map of the enterprise and is a route 

planner for business and technology change.” 

(Source: Microsoft Corporation) 

For the purposes of LDS development, the education agency and all of its parts 

make up the “enterprise.” The “architecture” is both the process of describing, and the 

description of, “the fundamental organization of a system, embodied in its components, 

their relationships to each other and the environment, and the principles governing its 

design and evolution” (IEEE 2000). Viewed as a process, EA identifies the mission and 

goals of an organization and the applications, technology, data, relationships, and other 

resources it uses to accomplish its work. As a description, EA documents the organization’s 

findings in the form of a top-level, low-detail blueprint that can be used to efficiently guide 

LDS development and maintenance. Though EA focuses largely on technology and the 

organization of data, its main function is to clarify the technological nuts and bolts so that 

technology and business needs can be better aligned. 

The EA should first describe the current system. The architecture exists whether you 

describe it or not, but depicting and documenting the “here” gives staff a better grasp of 

how the agency and its information system work. This depiction, at least at first, should 

Book Two of Four: Planning and Developing an LDS 35 
 

focus on high-level business operations rather than get mired in details. Then an EA 

of the future, ideal system can be created. With these blueprints, an agency will be in a 

better position to make decisions about how the enterprise needs to be modified to meet 

evolving goals (Aden 2008). The EA process also identifies who has authority over different 

system components, and who has been assigned responsibility for certain activities. EA will 

therefore be beneficial in both planning and governing the LDS. 

Figure 4 divides EA into five areas or perspectives, drawing focus to increasingly 

detailed aspects of the enterprise. 

Figure 4. Enterprise architecture in five parts 

Business What is my business? 

Information What information do I need? 

Applications How is that information presented? 

Data What are the data components? 

Technology What technology supports that data? 

While many details need to be determined as organizations drill down into the 

EA process, some high-level questions will help in early LDS planning and are presented 

below. A variety of stakeholders should be involved in answering these questions about 

what the system looks like and the purposes it serves. 

Business architecture 

 What is our business?
 

 Why does our business exist? What is its mission? What does it accomplish?
 

 How do we do what we do? What are our core processes? Who do we serve?
 

 How are we organized? How do people and processes interact to do what we do?
 

 What are the strengths of our enterprise? What do we do well and very well?
 

 What are our weaknesses or failures? What have we learned from these failures?
 

 How will our business change in the future? What are our growth challenges?
 

Information architecture 

 What information do we need?
 

 What decisions do we make? What information do we need to make each decision?
 

 What are the components of that information? How do we obtain each part?
 

 Where does that information originate? Who creates it? What is its quality?
 

 What information is needed to create the products our business produces?
 

 Is any of the information highly sensitive? How is that information protected?
 

 Is there information we do not have that could be valuable to our business?
 

36 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
 

 

 

 

 

Applications architecture 

 How is that information presented?
 

 What automated services support our business processes?
 

 How do our applications interact and depend on one another?
 

 How are data presented to users?
 

 How do our applications link various staff within the organization? With the outside world?
 

 How do our applications help us transform data into information?
 

 How do our applications serve different groups to achieve common business objectives?
 

 Do we have plans for developing new applications and revising old ones to meet our goals? 


(Microsoft) 

Data architecture 

 What are the data components?
 

 What are the sources of our data?
 

 How are data managed? What business rules and quality assurance procedures are in place?
 

 Do we have a data model?
 

 What metadata do we maintain? Is a system in place to manage these metadata?
 

 Do we have an authoritative, agencywide data dictionary?
 

Technology architecture 

 What technology supports our data?
 

 What technology standards and services are used to accomplish our mission? (Microsoft) 


 What technologies are used to collect and maintain our data?
 

 What technologies protect our data?
 

 What technologies provide access to our data?
 

 What technology expertise is needed to support this effort?
 

Developing an enterprise architecture is most beneficial before or during the planning 

and analysis stages of the LDS development process. But it is never too late, and great insight 

can be gained from the process at any point throughout the LDS’s life cycle. 

Book Two of Four: Planning and Developing an LDS 37 

Book Two of Four: Planning and Developing an LDS 39

Needs Assessment: 

Defining “There”

Chapter 8

any states jump head first into developing or purchasing an LDS (or 

certain components) before spending time to think about exactly why 

they are doing so, what their stakeholders need, and what it will take to get the job done. 

Experience suggests that successful LDS development is roughly 80 percent planning and 

only about 20 percent building.* Careful planning can make the difference between a 

clumsy belly flop and a graceful dive into system development.

During the needs assessment phase, policymakers should establish the business 

justification for what kind of LDS they want and why. In addition, stakeholders should 

define their requirements so that the resulting system reflects their needs. Needs assessment 

is also another step in creating early buy-in for the project, getting everyone—both internal 

and external stakeholders who use, or would like to use, education data—involved in 

creating a vision for a system that they will one day find useful.

80% Planning

20% Building

Why Do You Want an LDS?

Before the developers start defining requirements and building the system, some fundamental

questions should be answered about why the system is being built and what it will do for 

the education community. Early on, decisionmakers should ask themselves the following 

questions:

“Why do we want an LDS?”

? “What are our ultimate goals for the system?”

M

*This statement refers to initial development of the system, rather than the entire life cycle of the system, which will 

include ongoing maintenance of the system, substantial communication, change management, and end-user support.

?
Preferably, the answers will not be:
 

“To check off the 12 elements of the America COMPETES Act.”
 

“To keep up with other agencies.” 

“To improve instruction by helping teachers identify student needs, discover and 

practice the most effective teaching strategies, and tailor instruction; by helping 

administrators target teachers for professional development; and by helping 

researchers conduct more informative studies to identify effective strategies.” 

“To inform policy and resource allocation decisions at the state and local levels 

with better information, and to help state program staff target district and school 

improvement needs.” 

“To calculate academic growth; and follow students and maintain their academic 

histories as they advance to higher grade levels, transfer across districts within the 

state, drop out or transfer to private school or another state and come back into the 

system, and move into higher education.” 

“To track staff and maintain their professional histories as they enter and progress 

through teacher preparation programs, receive professional development, and 

transfer among schools.” 

Better answers are: 

✓

✓

✓

✓

✓ “To streamline operations and improve data quality by automating processes such as 

ata entry and loading, making data collection more efficient; by helping state staff 

roduce federally- and state-mandated reports; and by conforming to broadly shared 

ata standards.” 

d

p

d

What Do You Need from an LDS? 

Once decisionmakers and other stakeholders are familiar with the LDS concept (see Book 

One: What is an LDS?), the benefits it can provide (see chapter 5 of Book One) what their 

current system looks like (see chapters 6, 7, and 9), and why they need such a system, they 

can create a vision of what they want their own LDS to do and what functionalities they 

want it to have. 

Determine system requirements early so that expectations can be adjusted for 

everyone involved in the development process. Establishing goals and expectations for 

the system is also a good practice. This will help focus system design and keep IT staff, 

vendors, legislators, and others on the same page in terms of what needs to be done and 

what will actually be available in the future. 

Everyone wants to improve student achievement, but the important question is, 

“How do you want to do it?” Policymakers and other stakeholders need to determine 

40 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
Ask decisionmakers: 

“What do you want and why do you want it?” 

ducation experts: 

“What will the system do for you?” 

ystem developers: 

“What will make the system work?” 

veryone: 

“How do we know if we achieved the goal?”
 

“How do we identify success and anticipate failure?”
 

Ask e

Ask s

Ask e

? 

? 

? 

? 

Key questions for key audiences 

what functionalities they want and establish requirements for the developers. If system 

developers have clear requirements to fulfill, policymakers and other stakeholders are much 

more likely to get what they need. 

You may want a souped-up, cutting edge system that will cost a lot. Or you may just 

want a basic, no-frills economical system. Either way, you must be clear about what you 

need and what you can afford. Without careful reflection and planning, your organization 

may find itself with a poorly designed system that does not really do what it needs it to do. 

Others’ LDSs 

The National Center for Analysis of Longitudinal Data in Educational 

Research (http://www.caldercenter.org) offers links to several LDS 

datasets along with descriptions of their source systems. The State 

Data page (http://www.caldercenter.org/research/statedata.cfm) is a 

good place to research other agencies’ LDSs. 

Knowing what you do not know 

It is sometimes difficult to figure out what you need because you may not be familiar 

with what you do not have. You may not realize that certain functionalities might make 

your job easier. To get started, you might compare your current system, as depicted in 

the self-assessment, to the ideal LDS outlined in chapter 3 of Book One: What is an LDS? 

Researching other data systems or visiting an education agency with a more sophisticated 

system can also provide context and insight into available options (see NCES’s Personnel 

Exchange Network at http://ies.ed.gov/whatsnew/conferences/?id=185&cid=2). Also, review 

the software applications on the market. Consider the functionalities they offer and 

determine if they would work in your environment and meet your stakeholders’ needs. 

Book Two of Four: Planning and Developing an LDS 41 
When working with stakeholders, try to operate in concrete rather than in abstract 

terms. Instead of asking vague, open-ended questions about what they need, it may be 

helpful to start with questionnaires to gauge interest in certain LDS components and 

capabilities. For instance, an agency might survey staff and other stakeholders with a list of 

questions that the new system could be designed to answer. The staff could then rate the 

questions in terms of the value they think the answers would provide, thus allowing you 

to gauge stakeholder interest in a systematic fashion. And, since some stakeholders will 

inevitably be more outspoken in stating their needs in meetings and other venues, such 

surveys are a good way of leveling the playing field so prioritization of needs can be done 

fairly. 

Specific questions included in such a survey might be drawn from the stakeholders 

themselves, and the rating process could provide a basis for prioritizing stakeholder needs. 

Surveys like this might be written for different groups. For instance, teachers might be 

asked to evaluate questions about curricula or student achievement, while other staff 

might be asked to assess operational functionalities such as data entry automation and data 

sharing between databases. Based on the answers provided and the resources available, 

a state might decide which data to collect, which tools and reports to offer, and which 

capabilities to prioritize. 

Assessing what you need 

Third-party needs assessment 

You may conduct your needs assessment on your own or work with a vendor to define 

your requirements. Both approaches are common. Some agencies hire a vendor to do the 

requirements gathering, allowing an expert to figure out what the education agency and 

its stakeholders need. If your agency is allowed to enter into personal service contracts, 

you may be able to employ a vendor under a time and materials contract to do the 

needs assessment for, or with, you before the RFP is written (see chapter 14) and the 

development project begins. While some agencies find this approach very helpful, others 

prefer to do the needs assessment, or at least a large share of this phase, on their own 

before the vendor is brought on board. In other words, they feel the agency should know 

what it wants, rather than let a vendor determine what it wants. However, if you elect to 

have a vendor do requirements gathering for, or with, you, make sure the contractor is 

knowledgeable about your agency and about education data. And make sure stakeholders 

are being asked the right questions to elicit helpful responses. Stakeholders usually know 

what they want, but they often do not know how to articulate their needs unless asked the 

right questions. 

42 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
To help guide discussions, develop some questions that should be asked of the various 

audiences. Some examples are listed in table 3. 

Table 3. Sample needs-assessment questions for stakeholders 

 What information would help you improve instruction and programs, and better understand and meet students’ needs?
 

 What common data requests do you receive that cannot be answered with the currently available data?
 

 What additional data would you need in order to answer your questions? How can you get those data?
 

 What database linkages would be necessary to answer your questions (both within and outside the institution)?
 

 What access to data should be provided to various types of users to facilitate easier, more effective use of the data?
 

 What tools would facilitate access to, and analysis of, the data?
 

 What kinds of additional reports would be useful to staff and outside researchers?
 

 What types of professional development would be necessary?
 

 What user support would be helpful?
 

 What business operations do you want the LDS to improve?
 

 How could data collection be improved (e.g., move to web-based system)?
 

 How could the new system help you better comply with federal and state collections (e.g., collect data elements 


required by federal or state reports that are currently not collected or able to be submitted on time)? 

 What technology would you need to make the system work? What new technical capabilities would be necessary? 

 What additional security measures would need to be implemented to protect the new data? 

 What capabilities do other systems have that would potentially help your agency better achieve its goals? 

Documenting requirements in some form, such as in a needs statement, is very 

important. This is “a description of the functional needs, technical requirements, and 

security and ethical standards that need to be met by a technology solution” (National 

Forum on Education Statistics 1998). An agency should take its findings, categorize them, 

and try to boil them down to discrete needs. The resulting set of requirements can help 

guide the work of in-house system developers or help an agency find a product on the 

market that will meet its needs. If you are planning to hire a vendor (or several vendors), 

your findings will help in the creation of an RFP (see chapters 13–14). Either way, be sure 

your identified needs can be measured by tangible criteria, so that success in attaining the 

organization’s goals can be assessed during evaluation of the system later on (see chapter 15). 

After figuring out what you have and what you want, a “gap” analysis should be 

done to identify the discrepancies between “here” and “there,” and determine what work 

will be necessary to achieve the desired system. For instance, if the agency does not 

currently have a student identifier system, but the needs assessment calls for one, a “gap” 

has been found that needs to be addressed. 

Building an LDS with stakeholders’ input is an iterative process. Give it time and 

be persistent. Expect needs assessment to be ongoing, rather than completed once at the 

beginning of the project. As parts of the system are developed and go live, continue to get 

feedback to find out what stakeholders think and what new needs they might have. New 

ideas often arise from new developments. 

Book Two of Four: Planning and Developing an LDS 43 
District difference: District-level LDS considerations 

Most of the discussions about LDSs focus on state-level systems. But state agencies are not the only 

ones building these systems—many school districts and regional agencies are as well. While fewer 

people may work on a local-level LDS, the same need exists and the same mistakes are often made. 

Below are questions and considerations for school-district staff to ponder before building their LDS. 

(Many of these are also applicable to a state-level effort.) 

When assessing the need for a district-level LDS, ask 

 What are the existing options? 

• Can the state offer LDS services? 

• Can the region offer LDS services? 

• Is it possible to form a partnership or cooperative with other school districts in order to share 

the LDS effort? 

 Does your district have the resources to implement and maintain an LDS? 

• Can you cover the initial costs? 

• Will you need additional staff to maintain and manage data loads and reporting? 

• Can you afford the additional costs of training staff to use the system? 

• Can you afford to train staff to make informed decisions at the classroom-, school-, and districtlevel based upon the new LDS data? 

If the decision has been made that a local LDS is necessary, then 

 Investigate what other districts in your state are doing, as well as the state agency. 

• Ensure that whatever you do fits into the big picture and your investment will not be lost 

because the state agency requires use, or interoperability with, another unique system. 

 Coordinate with the state agency to maximize alignment with state data standards. Consider using 

data standards such as SIF specifications to connect systems. 

 Address quality issues at the source(s) of the data before they are integrated into the LDS. 

 Ensure that the LDS is able to handle your district’s unique local assessment data needs. 

Realistic LDS expectations 

When negotiating with a vendor or working with in-house staff to design your LDS, 

consider asking for these important LDS features: 

 Unique identifier system that allows individual student achievement to be tracked yet 

protects	student	privacy;

 Interoperability	with	other	K–12	systems;

 Interoperability	with	PK	and	postsecondary	systems;

 Teacher	value-added	data;

 Data	warehouse	(or	comparable	alternative);	and

 Reporting and analysis tools with easy-to-use interface. 

44 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
 

Data: Knowing What You Have, 

Identifying What You Need 

Chapter 9 

O nce the decisionmakers and stakeholders have considered the types of 

questions an LDS can answer, the organization should assess what data it 

has and what it will need to answer the questions they deem most useful. Most agencies 

already have a large amount of data, especially if they are collecting and maintaining data 

on individual students. However, while more is not necessarily better, the data currently 

maintained may not be sufficient to answer questions stakeholders have determined are 

important. This section offers some data an education agency may need to achieve its 

goals. 

Your data may be stored in a central data warehouse, or in many separate data stores. 

Likely source systems include the 

• student information system;

• curriculum management system;

• program systems (e.g., Title I);

• student transportation system;

• food services system;

• assessment/accountability system;

• human resources and teacher certification system;

• financial system;

• instructional management system;

• student health system; and

• library/media system.

After taking an inventory of what data it has, cataloguing all current and planned 

data collections, and identifying where data items are housed (and which system is the 

authoritative source for each data element), an agency should determine if it should collect 

any additional data. 

While the education community often focuses on student traits and learning 

outcomes, truly informative education research also requires context—the students’ learning 

opportunities and learning climate. In addition to outcomes, therefore, data users should 

look at information on the inputs and processes that contribute directly and indirectly to 

student learning. For instance, in which programs does the student participate? Who are 

Book Two of Four: Planning and Developing an LDS 45 
the student’s teachers? What classroom strategies are used? Are there differences in student 

learning opportunities by race, sex, and/or socioeconomic status (e.g., representation in 

special education and non-college-preparatory tracks, teacher experience levels, resources, 

expectations)? What are the local financial and hiring practices? 

Keep in mind that while individual data items about students and staff are extremely 

valuable in efforts to monitor and understand student experiences, deeper analysis will 

view certain data elements in concert (as “derived” data elements, indexes, or indicators) 

and track them longitudinally. Doing so allows data users to examine the relationships 

between various aspects of reality and illuminates the trends that occur over time, showing 

what educational inputs contribute to what kinds of results for which students. For 

instance, when evaluating the success of a particular program, researchers may look at 

more than the participating student performance on assessments. The relative effectiveness 

of a particular instructional strategy, or strategies, may also depend on context and input 

variables such as the background and preparation of the teachers implementing the 

program, how closely the program is implemented, and the characteristics of the students 

receiving the instruction. And a fair evaluation of the program will look at a host of 

discrete measures of success in addition to year-end summative test scores. Such holistic 

methodologies that combine a range of relevant data can generate powerful guidance and 

help educators more effectively meet individual student needs. 

Using the right data architect for your LDS 

The usefulness of your LDS will be greatly affected by its data architecture. A good data 

architect can create a flexible data model from the outset that will help avoid extensive 

(and expensive) changes later on. In addition to helping the agency identify the right data 

elements, the data architect can help create a fully integrated system by defining the 

relationships among those elements at different levels: conceptual (relationships among 

major concepts), logical (in terms of a data manipulation technology such as a relational 

database or XML), and physical (in terms of a particular product and means of storage 

such as a server or disk drives). (ANSI 1975) 

Of course, an agency may opt for an off-the-shelf, one-size-fits-all data model rather than 

building a custom solution with an in-house data architect. This approach may potentially 

reduce risk and time to implementation, but is likely to work better for districts than for 

states because of the wider variety of district-level products and the fact that districts’ 

research needs are usually less extensive. While a data architect can be invaluable in 

designing a data model specific to the needs of a state, existing models can be useful if 

two conditions exist: 

 Someone on staff has sufficient expertise in data architecture and data modeling 

to make a valid judgment that the commercial product meets the needs of the 

organization. 

 Someone on staff or on the LDS design team is capable of making the modifications that 

will almost inevitably be needed to meet the state’s specific needs. 

46 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
Education indicators 

Many resources are available to help determine what data are needed to answer 

education questions. For more information on developing and using education indicators 

to measure status and outcomes, see 

 Forum Guide to Education Indicators (Forum 2005) 

http://www.nces.ed.gov/forum/pub_2005802.asp 

 This document is designed to help readers properly create, use, and interpret education 

indicators. It also identifies standard definitions and calculations, and warns of common 

misuses of education indicators. 

 From Information to Insight: The Point of Indicators (ESP Solutions 2007) 

http://www.espsolutionsgroup.com/espweb/assets/files/ESP_Point_of_Indicators_ORG.pdf 

 This document discusses various types of education indicators as well as education 

“indexes,” which are combinations of related indicators that offer more thorough views 

of educational values and trends than single indicators can provide. It also discusses 

the selection of data elements required, and the establishment of thresholds, to indicate 

the need for action. 

 Comparative Indicators of Education in the United States and Other G–8 Countries: 

2006 (NCES 2007) 

http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2007006 

 This report presents 20 indicators used to compare the education system of the United 

States to those of other G–8 countries. Indicators focus on population and school 

enrollment, academic performance, context for learning, expenditure for education, and 

education return, educational attainment, and income. 

 Buried Treasure: Developing a Management Guide From Mountains of School Data 

(Wallace Foundation, 2005) 

http://www.wallacefoundation.org/sitecollectiondocuments/wf/knowledge%20center/ 

attachments/pdf/buried_treasure.pdf 

 Geared towards district-level management, this report presents 7 key types of schoollevel education indicators. The authors suggest that less may be more: rather than 

developing an indicator for every need, they encourage parsimony. 

 Study of Leading Indicators of Educational Improvement (Annenberg Institute for School 

Reform, 2008) 

http://www.annenberginstitute.org/pdf/leadingindicators.pdf 

 This study looks at leading indicators used to identify early signs of academic progress 

before test scores come in. These indicators may help agencies think about what 

questions they want to explore and the data they will need to answer them. 

Book Two of Four: Planning and Developing an LDS 47 
When identifying new data for collection, overarching goals should be used as a 

framework for selecting new elements and education agencies should collect and store 

only those data that will benefit the enterprise. Elements that stakeholders think would 

be nice to have, but which do not lend themselves to achieving stated goals, should be 

avoided. In addition, the data collected should capture the appropriate level of detail. For 

instance, when collecting data on attendance, should you collect by day, by period, or by 

some other unit of time? If attendance by day is sufficient, an agency may not want to 

burden staff further (National Forum on Education Statistics 2009). Also, widely accepted 

standards and definitions should be followed so that the records are consistent and 

comparable to other agencies’ data. 

Table 4 presents many of the key types of data that may be contained in a P–12 LDS 

and used for longitudinal analyses, but it is not exhaustive. Appendix C offers sources 

of more detailed and exhaustive data. Ultimately, agencies should collect all other data 

required for state and federal reports, as well as other key data necessary to answer its 

stakeholders’ questions. 

Table 4. Key data to collect for a P–12 LDS 

Student Personal and demographic information Enrollment information 

data •	Unique	student	identifier

•	Sex

•	Campus	of	enrollment

•	Grade	level

•	Date	of	birth •	Attendance/truancy	data

•	Race

•	Ethnicity Attainment information 

•	Language	 •	High	school	graduate

•	Economically	disadvantaged	status •	Diploma/credential	type

•	Limited	English	proficient	(LEP)	status •	School	dropout

•	Title	I	status	(or	schoolwide	status) •	Dropout	follow-up

•	Migrant	status •	Grade	progression	and	retention

•	Homeless	status

•	Disability	status Transcript/curriculum information 

•	Parent	education	level •	Course	codes	and	descriptions

•	Truant	status •	Completion	grades

•	Dual	enrollment	courses

Program participation information 

•	Bilingual/ESL	program Other information domains 

•	Gifted	and	talented	program •	Student	health	and	nutrition

•	Early	childhood	learning	program •	Safety	and	discipline

•	Individualized	Education	Program	(IEP) •	Transportation	data	(e.g.,	length	of	bus	ride)

•	Special	assistance	program •	Family	history

•	Library	records	(e.g.,	books	checked	out)

Performance information •	Meal	data

•	Assessments	(summative,	formative,	and	interim) •	Perceptions	data	(e.g.,	student–teacher		

•	Untested	student	records relationships, school climate) 

•	College	readiness	data	(AP,	SAT,	and	ACT	scores) •	Fidelity	of	implementation	(of	

•	Grades programs and strategies) 

•	Credits	earned

•	Awards	(e.g.,	diplomas)

•	Displaced	status

48 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
Teacher and Personal and demographic information Professional development information 

staff data •	Unique	teacher	identifier

•	Date	of	birth

•		Professional	development	training	(e.g.,	record	

of in-service credits, type of training) 

•	Sex •	Hours	of	professional	development

•	Ethnicity

•	Race Personnel information 

•	School	identifier

Qualifications information •	Job/subject	assignment(s)	(e.g.,	teacher,	

•	Years	of	experience	(by	location) librarian) 

•	College	attended/certifying	institution •	Program	assignment	(e.g.,	Special	Education)

•	Highest	degree	earned •	Position	title

•	Academic	major	and	minor •	Position	codes

•	Highly	qualified	status •	Schedules:	grade/course/period	taught

•	Graduation	(with	dates) •	Compensation	(e.g.,	salary,	benefits,	

•	Certificates	(with	dates) supplemental contracts) 

•	Licenses •	Employment	status	(e.g.,	full-time	equivalency;	

•	Endorsements start/retirement/leave dates) 

•		Staff	assessment	results	(e.g.,	subject	 •	Time	spent	on	administrative	duties

knowledge test scores) •	Tenure

•	Mobility	and	attrition	data

School Finance information District demographic information 

system data •	Revenues and expenditures 

•	Salaries and benefits 

•	School size 

•	Class size 

Facilities and technology information 

•	School safety 

•	AYP 

•	Building identifiers 

•	Building area and space utilization Community demographic information 

•	Building condition •	Locale 

•		Classroom type (e.g., conventional, distance •	Adult education levels 

learning) •	Income single parent households 

Organizational information 

•	Property values 

•	Labor force data 

•	School 

•	Accreditation 

•	Relationship between schools 

•	District- and school-level directory data 

Sources: EIMAC 2007a and 2008, DQC 2009, NCEE 2007, Davis 2008, Center for Strengthening the Teaching 


Profession 1994, Berry et al. 2007, and Nunn et al. 2006. Refer to appendix A for a complete list of references.
 

Book Two of Four: Planning and Developing an LDS 49 

Some Critical “Abilities”: 

Interoperability and Portability 

Chapter 10 

J ust as monetary standards simplify marketplace transactions, exchanging 

education data between systems and applications is difficult without data 

standards. Without interoperability—the quick and easy transfer of data between systems 

via a common set of data definitions, codes, and technical software standards—resource 

exchange is laborious and taxing. In other words, whether resources are being exchanged in 

the marketplace, or data are being transferred among data systems, shared standards allow 

easy and reliable transactions. 

Interoperability is the quick and easy transfer of data between 

systems via a common set of data standards (definitions, codes, 

and technical specifications). 

An interoperable system is “an environment in which diverse data systems 

seamlessly exchange information with little or no additional effort” (Collins and Fruth 

2007). Use of widely accepted technical specifications based on common definitions and 

codes facilitates this kind of environment and allows information to be easily and safely 

shared among numerous systems and applications regardless of the platform or vendor. 

The standards offered by the Schools Interoperability Framework (SIF) Association are 

perhaps the most commonly used, though some states and districts have achieved or are 

exploring interoperability through other means. While many states have implemented 

or are pursuing interoperability, and the federal government strongly encourages the 

establishment of integrated and interoperable data systems, the majority of education 

systems are still working with numerous isolated applications.* This reality costs countless 

staff hours and resources, and limits education staff's ability to effectively use the data 

collected. 

Interoperability, by 

allowing us to link 

and easily access 

previously isolated 

data sets, lets us 

easily utilize a fuller 

range of data about 

student and staff 

experiences. 

*The seventh action step of the National Education Technology Plan (USED 2004) states that “integrated, interoperable 

data systems are the key to better allocation of resources, greater management efficiency, and online and technologybased assessments of student performance that empower educators to transform teaching and personalize instruction.” 

Book Two of Four: Planning and Developing an LDS 51 
 

 

 

An LDS should allow for the timely and simple exchange of data between 

applications in and among schools, districts, states, and other education institutions; as 

well as agencies and organizations outside the education system. By ensuring compatibility, 

interoperability opens the door to vast quantities of longitudinal data that may otherwise 

be too laborious to acquire. These diverse data allow us to explore questions previously 

difficult to answer due to an inability to link data from various sources—sources that 

illuminate many dimensions of students’ lives. In this sense, interoperability allows us to 

easily view a more complete and accurate picture than is possible using only fragmented 

sets of data. 

Effective use of data requires information from various sources and applications 

to work together to enable easy analysis and reporting. Data should enter the system at 

a single point; each data element should have a single, authoritative “source of truth”; 

the various applications should share data; and the data should be usable many times 

and for many purposes. This will free staff from entering information into each of the 

systems (enrollment, library, school lunch, etc.). By eliminating the need for redundant 

effort, interoperability also improves data quality by decreasing the risk for data entry 

errors, and saves time for activities other than tedious data entry and management. Staff 

will, therefore, have more time to offer better services to students, focus on teaching, and 

improve student achievement through more effective and timely analysis and data-driven 

decisionmaking. (Schools Interoperability Framework Association 2006) 

Achieving Interoperability 

How does a state or local education agency achieve interoperability using, and based 

upon, education technology standards? The shift to interoperability requires the support 

of many agency staff, including decisionmakers, data system managers, technical staff, 

program areas, and the end users of the system. This is an overall culture change in the 

organization in terms of the way data are collected, viewed, used, and shared. By bringing 

together the different players in the agency to discuss and answer key, guiding questions, 

you can begin to form a team and garner the support needed to make the project a success. 

Without support across the education agency, the chances of succeeding will diminish 

because, while interoperability is a great enabler, it is also viewed with skepticism because it 

is change that is being enabled. 

When deciding to implement an interoperable solution, an organization will need to 

address several key questions, such as:

	 What is the ultimate goal of the interoperability effort? 

	 Will this project connect systems from multiple local levels to gain a 

comprehensive view?

	 What are the data you are trying to share and why? 

	 Are the data compatible? 

	 What standards protocols will be used? 

	 Will the project be developed and implemented in-house or will a vendor direct 

the process? 

	 Will the existing software applications and data structures be repurposed or will 

new ones be purchased or developed? 

52 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
The answers to these questions will determine the scope and breadth of the project. 

As we drill into these key guiding questions, it becomes apparent why discussing and 

addressing them with the team forms the foundation for interoperability. 

“What is the ultimate goal of the interoperability effort?” 

When considering an interoperable solution, you should be able to clearly state, in a 

one- to two-sentence goal statement, what the project is about. If you go beyond a simple, 

realistic goal, you run the risk of creating a project with a very broad scope that can be 

cumbersome to implement. 

For instance, a goal might be: 

“Our goal is to have one point of data entry to improve data quality, reduce data 

latency, and advance data entry efficiency, thus improving our ability to service our 

stakeholders and effectively make sound educational, data-driven decisions based on 

the most accurate information available.” 

“Will this project connect systems from multiple local levels to 

gain a comprehensive view?” 

This is the second guiding question for the group to consider, and it is perhaps the most 

important as its answer can greatly expand the reach and timeline of the project. If this 

project is for one district or state, that is one reflection. But if it is to span entities, many 

additional considerations come into play, such as which systems need to be interoperable 

and which contain the data needed, and are there multiple data collections involving 

multiple program areas? 

“What are the data we are trying to share and why?” 

These two questions of “what” and “why” go hand in hand. They will take time to analyze 

and discuss as a group in order to determine what is, or is not, comprehensive enough to 

be included in a meaningful interoperable system. 

“Will the project be developed and implemented in-house or will 

a vendor direct the process?” 

This question is significant to the project for several reasons. If you have the technical staff 

and capacity to take on a project of this scope in house, the cost may be greatly reduced 

but it may also increase implementation time. By the same token, working with a vendor 

whose core competence is implementing solutions may greatly reduce the implementation 

time, but it can increase the cost and the final system may also be based on a proprietary 

solution rather than a true, publicly available and widely used interoperable solution. This, 

to some degree, can be solved if you tell the vendor your project must be built on specific, 

commonly used education technology standards such as those developed by the SIF 

Association (http://sifinfo.org) or PESC (http://www.pesc.org). 

“Will the existing software applications and data structures be 

repurposed or will new ones be purchased or developed?” 

When repurposing existing software applications and data structures, you will either 

use existing software that has already used, or could use, widely accepted standards. For 

example, many existing applications can be repurposed and adapted to incorporate (or 

accept) SIF standards. Alternatively, you can buy or develop new applications based on 

standards. An example would be one of the many software applications that have been 

SIF-certified. 

Book Two of Four: Planning and Developing an LDS 53 
 

 

 

 

Interoperability Timeline 

Different states and districts will pursue different timelines for an interoperability project. 

With the help of a vendor specializing in educational data systems, one state is working 

on a 12-month rollout timeline that includes processes such as hardware and infrastructure 

development, training, and knowledge transfer. While this may be considered an aggressive 

timeline to many, it may be appropriate for others. Generally, districts and states plan 

a phased implementation over a period of 2 to 5 years. This may seem a long time to 

implement an interoperable system, but experience shows that this is the reality. The time 

involved should be considered especially when developing a project that incorporates 

multiple entities. 

Phases of the project based on the systems development life cycle include, but are 

not limited to, the following:

	 Planning: Data-needs discovery and analysis.

	 Design: Writing and release of requests for information, proposals, and/or

quotation (see chapter 14). 

	 Development: 

• Implementing a skeleton of connectivity including hardware.

• Structuring and building, or buying, an LDS.

• Develop validations and custom reporting tools.

	 Testing: Evaluating and/or piloting the system.

	 Deployment: Project roll-out, training, and knowledge transfer.

	 Maintenance: Ongoing system maintenance.

Experiences on the Ground 

Some states have experienced pushback from their districts because of the extra workload 

and significant costs that may be associated with implementing an interoperable solution; 

similarly, some districts have experienced resistance from their schools. This resistance may 

be more likely in states where districts are especially autonomous or resistant to change. To 

overcome these obstacles, some states try to include at least several of the largest districts in 

the interoperability framework project with the hopes that this success will interest smaller 

districts that may have the most to gain from such data sharing. Other states have decided 

to temporarily leave some districts out of the interoperability framework, either because 

they are too small or their current software applications are not ready. In these cases, the 

state may allow the districts to submit and receive data as they have in the past, sometimes 

even on paper, as they wait to move to a fully interoperable system. The ultimate, realworld benefits to all involved should be stressed to gain buy-in for the transition to 

interoperability. 

54 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
Portability 


Portability is the ability to exchange student record and transcript 


information electronically from system to system, across districts, 


and between P–12 and postsecondary institutes—within a state and 


across states. 


(Collins and Fruth 2007) 


Hurricane Katrina and the resulting displacement of students 

highlighted the need to be able to transfer student information, not 

just within a state but between systems around the country. Data 

portability is “the ability to exchange student record and transcript 

information electronically from system to system, across districts, 

and between P–12 and postsecondary institutes—within a state and 

across states.” This option of maintaining, moving, and sharing a set 

of personal student information allows districts and states to quickly, 

easily, securely, and quite cheaply attain information on students 

who transfer in and out of their school system, and helps them to 

distinguish transfers from dropouts. 

(Collins and Fruth 2007) 

When an interoperable solution is implemented, portability and interoperability can and 

should work hand in hand. With the thought of allowing information to flow seamlessly 

or be “ported” between systems, you are enabling data portability at the level of the 

project’s scope—locally or between entities. Many refer to this in terms of the portability 

of “content” rather than of data but, viewed holistically, portability can refer to any 

large amount of data or content that needs to be shared. Interoperability also relates to 

the meaning of the data once they have been ported between systems. Some states are 

working on content delivery systems that are both interoperable and portable. Others are 

seeking the ability to move a student’s e-transcript, whole student record, or portfolio of 

work between trusted entities, using portability as well as interoperability. Portability can 

also imply the ability to exchange large amounts of data, such as a state report, packaged 

in a way that allows for easier movement within an interoperable framework. Many 

experts believe that future innovations will allow for greater levels of data portability 

and interoperability across sites. Others are proponents of storing all student transcript 

information in a central location and allowing secure, on-demand access by authorized 

parties rather than relying on other means of moving the data between education agencies 

and across operating platforms. 

Book Two of Four: Planning and Developing an LDS 55 

Ensuring System Sustainability: 

Staying “There” 

Chapter 11 

B y definition, longitudinal data take years to amass. This means LDSs need 

to be around for a long time if they are going to fulfill their intended 

purposes and potential. The question then arises, what does it take to sustain these systems 

over the long haul? This chapter offers some basic strategies for the long-term, while the 

sections that follow focus on some specific solutions vital to winning broad and lasting 

commitment for your system, including marketing and relationship-building among the 

state and local education agencies. 

Winning Early and Lasting Support 

Generating the charter: 

Action steps 

1. Get executives to sign off on the LDS. 

2. Establish goal(s)—articulate the business case for building the system. 

3. Identify the seed team. 

4. Develop a high-level timeline. 

5. Develop a high-level budget. 

6.		Do	the	self-assessment;	revisit	charter;	and	re-examine	the	project’s	scope,	

budget, and timeline. 

A sustainable system has broad and deep stakeholder support, as well as long-term 

commitments of funds and staff. If the system is to be a success, strong interest in the 

project is vital and should be built by using effective marketing and outreach to gain 

early support throughout the education community (see chapter 12). This is vital because 

Book Two of Four: Planning and Developing an LDS 57 
 

 

legislators can provide needed resources and support. Agency executives will authorize 

system development and implementation. Staff will share responsibility for the system and 

have a stake in its success. And local administrators and teachers will use the system and 

come to rely on it for valuable information. This wide-ranging support will provide the 

foundation of a sustainable system. 

Legislative support 

Early on, the project should be promoted to legislators to explain the value of the planned 

system, establish expectations, and garner both political and financial support. Lawmakers 

can provide funding for the system; they can also pass legislation that supports the system 

by, for instance, writing its major tenets into law or mandating system compliance. Project 

leaders should determine what benefits should be emphasized to legislators and how to 

present them effectively. Whoever makes the proposal must have a broad understanding 

of the LDS and be ready to impart the right information. That is, they must understand 

what the legislators need to know—the purpose, potential benefits, and estimated costs of 

the system. In addition, they must understand what education specialists need, as well as 

basic technical details. They should make a compelling yet realistic proposal, taking care 

not to make promises that cannot be kept or set goals that cannot be reached. They should 

convey the big picture in plain language and avoid excessive detail, keeping in mind that 

their audience may not have expertise in education, let alone education data. 

Additional sources of funds 

In addition to relying on legislators for resource commitments, consider other sources of 

funding. As detailed by the Education Information Management Advisory Consortium 

(EIMAC 2007), potential sources of financial support for building and maintaining an LDS 

include 

	 federal grants (e.g., Institute of Education Sciences Statewide Longitudinal Data 

Systems grants);

	 foundational support (e.g., the Bill and Melinda Gates Foundation); 

	 separate program areas (assessment, Individuals with Disabilities Education Act, 

Limited English Proficient, Title I, etc.);

	 state-level technology bonds; 

	 universities and higher education research institutions; and 

	 business/private sources. 

Estimating costs 

According to some estimates, the development of a state-level LDS will cost around $1 

million per year for three to five years. However, there are many variables that affect how 

much an LDS will cost an education agency. Research how much other agencies have 

spent on their systems or various components, keeping in mind any differences between 

your agencies, environment, and system requirements (see chapter 14). Some important 

factors that may affect system costs are: 

58 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

	 Starting point 

What does your education agency already have? Can parts of the existing system 

be modified or will you need to start from scratch? What infrastructure will you 

continue to use with the new system? 

	 Size 

How populated is your state or district? How many students and staff will the 

system follow? 

	 State laws 

Do mandates require the system have certain characteristics, such as 

implementation of an interoperability solution? 

District difference 

Estimating the cost of implementing an LDS may be more difficult for 

a district than a state agency because of several factors. For instance, 

districts have a relatively limited ability to learn from each other’s 

experiences due to the lack of networks and means of collaboration 

that exist among states. Also, the number of districts developing an LDS 

within a state may be limited, yet the experiences of distant districts 

may not translate well across state lines. Further complicating the 

task is the wide variation that exists among districts in terms of size, 

circumstance, and needs. However, in most cases, the cost of a district 

LDS should be substantially less than a state’s. 

	 Environment 

What is the existing level of communication and collaboration among districts, 

and between districts and the state? Establishing new lines of communication 

will take time and money. How uniform is your current system across districts 

and what standardization efforts will be required? 

	 Data demands 

How many users will need access to the data and what security demands will this 

access entail? 

	 Local costs 

If you are building a statewide system, what participation costs will be shouldered 

by the districts rather than the state? 

	 Scope 

Do you want a top-of-the line system or a basic one? How, and to whom, will 

the data be made available (agency staff, teachers, students, parents, researchers, 

etc.)? What data linkages will be created (to postsecondary institutions, workforce 

organizations, other agencies within the state, etc.)? Beware of costly scope creep. 

	 Procurement 

If you are building the system, what staff will work on the project and what 


additional staff or contractors will be required? If you buy, consider both vendor 


and in-house staff costs.
 

Book Two of Four: Planning and Developing an LDS 59 
 

 

 

 

 

 

 

 

 

 

	 Overlapping efforts 

In building infrastructure or developing system components, savings may result 

by working towards multiple goals in a single effort. 

	 Change 

Program changes, new definitions and updated standards, and the addition of 

new data elements may require programming changes and involve significant 

additional costs. 

	 Training and user support 

How many staff members will need to be trained to use the system and new 

tools, do data analysis, interpret results and reports, etc.? What user support will 

be necessary? 

Success 

 Define it. 

 Measure it. 

 Record it. 

 Publicize it. 

	 Savings 

How much will the new system save? How will new efficiencies save staff time? 

What collections can be eliminated or streamlined by the new system? Weigh 

the benefits against the costs: over the long run, the benefits should amount to 

big savings. 

	 Maintenance 

The LDS will have not only startup costs, but ongoing maintenance costs as 

well. Who will maintain the system? A vendor or in-house staff? How often will 

hardware and software be updated? Who will update documentation? 

(EIMAC 2007a and EIMAC 2007b) 

Maintaining Long-Term Support 

Though LDS development is often referred to as a “project” (both in the real world 

and in this guide), it is much more than that. Whereas a project has an end, an LDS is 

ongoing and requires constant maintenance and enhancement. Implementation is just 

the beginning. Everyone involved should understand that achieving the system’s benefits 

will take hard work and several years. Maintain excitement along the way by structuring 

the project around incremental stages with frequent milestones. Each intermediary 

achievement should be announced to the education community so that the results of 

their efforts and resources are apparent (see chapter 12): patience and determination are 

important, but everyone needs to see results once in a while or faith in, and commitment 

to, the system may wane. 

A successful LDS may outlast political leaders, so bridging administrations is 

also important. Different people value education data differently. Therefore, a new 

commissioner or a new governor can change the support for the project. Be flexible 

when new leadership with new agendas comes along (DQC 2006). At the same time, a 

deep commitment to the system across the education agency and the broader education 

community, as well as being able to show some actual benefits, will help sustain the LDS 

through changes at the helm. 

60 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
Establish early on how success will be measured. For example, a system may be 

deemed successful if it delivers high quality results within budget constraints and on time 

(see chapter 15). Document system successes such as new efficiencies and improvements in 

educational outcomes or processes, and use them in future requests for resources. Building 

a system with measurable deliverables and outcomes will make this possible. Education 

agency staff should also make an effort to win over new leaders to the importance and 

value of the LDS data. Brief them on system capabilities and equip them with data they 

need to prepare for meetings and make good decisions. One state’s new commissioner, 

for example, wanted to meet with district representatives. Agency staff provided the 

commissioner with detailed data on each district he was to visit. This had two very 

important, far-reaching effects. It informed the commissioner and proved the usefulness of 

the data; and it demonstrated to districts that the data they report is actually used, giving 

them greater incentive to submit high quality data. 

Book Two of Four: Planning and Developing an LDS 61 

Marketing and 

Communicating Your LDS 

Chapter 12 

G aining stakeholder buy-in for an LDS is critical for the long-term 

sustainability of the project, as well as for the effective design and 

utilization of the system. Unfortunately, developing a system, even one that will greatly 

benefit the entire organization and its stakeholders, may not on its own be enough to get 

everyone on board. Agencies often need to also “sell” the idea to gain support for the LDS, 

and many have pursued different tactics to do so. This section offers best practices to help 

your education agency face the marketing challenge. 

Who Is the Audience? 

History helps 

Take advantage of relationships that exist throughout 

the education community, especially between state, regional, and local 

education agency staff. Get the word out and keep these lines of 

communication open. 

Cast a wide net to promote the system, reaching out to stakeholders throughout the 

education community and focusing on all levels. Present the project to education agency 

executives and decisionmakers at the top, focusing on the policymaking advantages 

of the LDS. Brief them regularly on the progress made, as well as how the system can 

help achieve the organization’s goals. Buy-in at the top may also increase when the 

LDS team moves forward with the project—contacting districts or schools for data, for 

instance. In addition, take a grassroots approach to publicizing the project. Make sure that 

information and support travel up, as well as down, the organizational ladder and ensure 

that all stakeholders realize that the LDS will be critical to the organization’s day-to-day 

operations. 

Put special emphasis on the business side of the organization. While the project will 

need support from your organization’s technology team, buy-in from nontechnical staff is 

critical. Involving a wide variety of stakeholders in the design process will help everyone 

Book Two of Four: Planning and Developing an LDS 63 
 

 

 

 

 

 

feel connected to the project and gain early buy-in (see chapter 4). Get input from program 

area staff, district representatives (for a state LDS), school administrators (particularly 

important for a district LDS), teachers, and other interested parties before designing your 

system; and let everyone know you are incorporating their feedback. What data do they 

want? What information will help them save time and be more successful? What tools will 

best suit their needs? How will privacy be ensured? (See chapter 4.) 

To hush or to holler... 

Should you keep a low profile or grab a megaphone and yell about your LDS 

from the rooftops? Education agencies have taken both paths: some have 

worked under the radar, while others have aggressively advertised their project. 

Both approaches have their advantages. While the latter course may slow 

down the development process by inviting lots of interest and involvement, the 

limited outreach approach may lead to serious problems later on when the 

agency tries to build support for the project. 

What Is the Message? 

Communicating about an LDS development project can be a major challenge. However, 

getting everyone to understand what the new system will be, and what benefits it will offer, 

is important if the project is to succeed. 

Start with a clear vision for the project. Communicate it consistently, and be 

prepared to repeat your message multiple times. Frame the LDS around organizational 

goals and how the new system will better equip everyone to achieve them (ESP Solutions 

2007). Get the word out: 

 What will the new system be? How will it be different from the current system? 

 Why is the system being developed? How will it benefit the education 

community? Stress streamlined daily operations, time and money savings through 

increased efficiency, quicker access to data and better informed decisionmaking, 

and improved services. Make sure the LDS is not perceived as a burden. (See 

chapter 5 of Book One: What is an LDS?) 

 How will the system better equip the organization to meet its goals? 

 How will the data be protected? 

 How will the culture of the educational community need to change in terms of 

the way people think about and use data? 

Also provide frequent updates: 

 Who is backing the project? What support and resources have been won (governor 

buy-in, grants awarded, etc.)? 

64 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
 

 

 What progress has been made toward system goals? Which benefits and 

functionalities have already been achieved, and which will materialize in the future? 

 What is the timeline for the system? Be realistic in your estimates so that 

expectations are appropriate. Highlight planned milestones and gain support for 

reaching them. 

Converting the LDS nonbelievers 

While many will get on board early without much convincing, there will probably be 

individuals who are not so willing to embrace the project. For instance, many schools and 

districts do not see data systems as critical to their day-to-day operations. Show them the 

value of the system. Win these people over by showing them that the LDS will improve 

their days and make their jobs easier. For example, what information will they be able to 

get through the portal? What reports will be made available to them? What data entry 

will become automated? How will greater efficiency improve the data’s timeliness and 

usefulness? How will better information equip them to make better decisions and spend 

their time and resources more effectively? Note that improvements do not always need to 

be high tech. Often, the changes that please local staff the most are simple ones that save 

them time every day. 

Explain why the system they are using and the data it contains are inadequate. For 

example, the current system may collect and store redundant and/or conflicting data in 

various silos, rather than collecting data once and storing them in a centralized, integrated, 

and authoritative data storage facility. Data entry may be laborious, or getting information 

back in a timely manner may be difficult. Then show how the new system will address 

these issues and improve their data. For example, if your development project involves the 

construction of a data warehouse, identify all of the silos the LDS will replace. While the 

silos may have served the state well in the past, explain that the new data warehouse will be 

better for reasons x, y, and z. For example, how will the new system be different from—and 

better than—existing transactional or other types of historical data systems in use? What 

insights can new data linkages provide that could not be derived from a system of disparate 

silos? Ease anxieties by stressing that the system is being implemented to help staff, not to 

replace them. 

When all else fails, more coercive measures may become necessary. For instance, 

an executive such as a superintendent might send out letters or emails to noncompliant 

districts requiring them to get on board. Some states make compliance with the LDS 

mandatory. If that is the case in your state, communications can be more forceful than 

merely offering encouragement. 

This following plan has been adapted from “Strategic Communications Planning,” 

available at http://davefleet.com/2008/08/strategic-communications-planning-a-free-ebook. See 

this resource for more guidance on developing a communication plan. 

Book Two of Four: Planning and Developing an LDS 65 
Strategic communication plan 

Develop a communication plan early on to identify your outreach needs and carry 

out an effective public relations campaign. A strategic communication plan may 

consist of the following sections: 

Analysis 

 Context: What has happened before? What is the relevant history? 

 Environmental scan: What are the key factors that will affect the project’s 

success? What are other jurisdictions doing? What is the legislative context? 

 Stakeholder analysis: Who are the LDS stakeholders and what are their 

characteristics? What are their expected reactions? How can the support of 

those who react positively help mitigate the concerns of individuals who may 

resist the project? 

Planning 

 Objectives: What clear and measurable goals do you want to achieve in your 

communications effort? To educate your stakeholders? To build support or 

create demand for the LDS project? 

 Strategy: From a top-level perspective, how will you achieve your 

communications objectives? Should your approach be high- or low-profile? 

 Audiences: What specific key audiences will you try to reach? What are 

their needs and interests? How will you tailor your communications to these 

various audiences and maximize the impact of your message? 

 Announcement: Given the strategy, should you make an announcement? If 

so, how will you summarize the project in your announcement? 

 Message: What are you trying to tell stakeholders? What is the project and 

why are you doing it? What will change as a result of the project and the new 

system? 

 Tactics: How will you implement your strategy before, during, and after your 

project announcement (if you make one)? What modes of communication 

will you use (emails, direct mailings, speeches, meetings, training sessions, 

web conferences, presentations, websites, press releases, etc.)? Who will be 

responsible for different communication activities, and when will each take 

place? 

 Issues: What problems might you have to overcome? Can you anticipate any 

potential confusion or pushback? How will you deal with these issues if they 

arise? 

 Budget: What will your communication plan cost? Where will funding come 

from? Do not ignore seemingly small details. 

 Evaluation: How will you know how well you have achieved your goals? How 

will you measure your marketing success? Will you be able to see a beforeand-after effect of your efforts? For instance, did the project receive media 

coverage? How did stakeholder perceptions change? 

66 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
Who Should Do the Outreach? 

When trying to get the word out, having the right people promoting the project is 

important. Identify these passionate communicators among your stakeholder groups and 

deploy them to talk with their peers. Energized district leaders, for instance, can talk or give 

presentations to other district leaders, sharing with them the benefits of the LDS and the 

status of the implementation process. They can also solicit valuable feedback. Enthusiastic 

legislators, governors, state superintendents, and other high-level leaders can also make 

a strong impression and help overcome resistance and remove political barriers. These 

high profile advocates, or “champions,” can speak at meetings or communicate through 

mailings to raise awareness and get the system on people’s radars. Alternatively, these roles 

might be filled by outside consultants who can focus exclusively on the marketing effort. 

Either way, these champions should be knowledgeable about the system and the progress 

towards its implementation. 

What Modes of Communication Should Be Used? 

While agencies have used a vast arsenal of communication techniques, a large share of the 

information about the project should be shared during internal and external stakeholder 

meetings. Presentations in these meetings or at conferences (both in-person and web-based) 

should include progress updates. Training sessions and other development activities are 

also great venues for marketing the system. Other means of communication include email 

campaigns; paper mailings such as letters, newsletters, or brochures; and press releases to 

announce the project and milestone achievements. If the media are contacted, staff must 

be ready to effectively and consistently respond to their questions and concerns if support 

is to be gained from the general public. Assign specific personnel as the go-to people 

for specific types of information on the system, and direct calls and questions to them. 

Agencies might also consider creating a web page or site dedicated to the project, to which 

stakeholders can be directed for up-to-date information. 

Agencies must be able to disseminate information quickly in a format that is 

understandable to nonspecialists. Again, have a clear message and make sure that your 

audiences understand what you are saying. Avoid miscommunication, for example, by 

limiting the use of jargon in general presentations and communications. Agencies might 

also consider creating a common glossary of terms for stakeholders to reference. 

(DQC 2006b) 

Maintaining Support 

Communications should be used to keep everyone excited throughout the project. An 

LDS development project plan that includes many short-term deliverables will lend itself to 

a successful marketing effort. Show progress, however small, by announcing achievements 

and delivering results along the way. With the completion of each deliverable, a separate 

communication can be disseminated to celebrate each little “win.” These victories can be 

advertised via the media and throughout the organization. But be careful not to promise 

anything you cannot deliver quickly. If results are not forthcoming when promised, or if 

your first success is long to come and subsequent ones infrequent, faith in the project and 

Book Two of Four: Planning and Developing an LDS 67 
 

LDS lore: Know your audience

and communicate appropriately 

An agency staffer met with a group of superintendents after having sent a halfdozen communications about the system.

After sitting down, she quickly realized that 

none of them had a clue about the system,

but she couldn’t blame them. They were

probably jaded because numerous similar

projects had been attempted in the past and

failed. After the meeting, instead of writing

summaries of the project, the staffer started

to send out brief communication such as, 

“The system redesign is really happening

this time. Here’s a website that details the 

progress.” This saved her time and let the

reader learn about the new system on their

own time. 

your ability to create and maintain support may be diminished (DQC 2006b). And, in 

addition to talking to staff about the project, project managers should also acknowledge 

a job well done. A little recognition or token of appreciation for those working on the 

project will go a long way in keeping everyone motivated and invested in the project. 

Finally, the system and the data themselves are another important marketing vehicle. 

Along the way, get the data out and show stakeholders its usefulness. Further down the 

line, when parts of the system are ready for testing, pilots are also an effective means of 

winning over key stakeholders. They get districts and schools interested in the system, let 

them try it out, and give them the chance to help improve it and resolving any kinks (ESP 

Solutions 2007). If a new data mart is created, for instance, let the appropriate stakeholders 

explore it and see how it will benefit them. If an analysis tool is created for teachers or 

administrators, let them see its value and chime in on ways to make it more useful. 

68 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
Part III 

Getting from “Here” to “There”: 

Developing What You Want 

hen an education agency has firmly established both a vision and support 

for the LDS, focus should turn to implementation. Now that they know

where “there” is, agencies can depart from “here” and begin work towards the desired 

destination. LDS projects may either be planned in phases or confronted all at once, 

and states have taken both routes. Which is most appropriate for you depends on many 

factors; perhaps the most important are the agency’s available resources and the scope of 

the project. 

Part III discusses many of the decisions an education agency will have to make and 

the challenges it may face while building or purchasing a system. It also offers tips to help 

make the ride to “there” go more smoothly. 

W 

Book Two of Four: Planning and Developing an LDS 69 

Procurement Planning: 

Build or Buy 

Chapter 13 

W hether an agency is building a data warehouse or adding reporting and 

analysis tools to an existing depository, someone has to develop and 

implement these products. Whether to take on these tasks within the agency or hire a 

vendor to tackle them for, or with, you is an early and crucial decision. This chapter’s goal 

is to help you become an informed builder or consumer before you begin construction 

or hire vendors, in order to ensure you end up with a system that suits your needs and is 

worth the money. A collection of lessons learned and good practices, this section will help 

you choose, and steer you in a good direction once you have decided to either build or 

buy a solution, or perhaps pursue a combination of the two. 

Deciding Which Path to Take 

Take baby steps and celebrate each one 

Even with ample resources and support, the general consensus among experts is 

that breaking the project into manageable, small, short-term stages is the best way 

to go—one baby step at a time. 

Not only is attempting an entire LDS project at once a daunting challenge, it may 

also strain an organization if staff need to devote time to extra tasks, hurt morale 

if successes and benchmark achievements are infrequent, and ultimately lead 

to serious compromise in terms of quality. Establishing a clear, long-term plan 

and breaking the project into phases offers many advantages. Working on one 

or	two	goals	at	a	time	is	manageable	for	staff;	and	small,	short-term	successes	

keep everyone excited and focused. Be sure to highlight and celebrate each “win” 

along the way. On the down side, the slower pace could make some stakeholders 

impatient and draw out the timeline for the entire project. But in the end, a better 

system will result. 

Book Two of Four: Planning and Developing an LDS 71 
 

 

 

 

 

 

 

 

 

 

For some states, the choice between building or buying is obvious from the outset. 

For others, much deliberation is necessary. The decision depends on many factors. For 

instance, organizational characteristics such as technical capabilities, workload capacity, 

and culture are all important considerations. Does the organization have, and will it 

continue to have, the staff and expertise needed to carry out the project and maintain 

it throughout its life cycle; or is additional staff or a vendor needed? Beyond inhouse characteristics, organizational preferences play a role, deciding, for example, the 

fundamental question of where responsibility will reside—in-house or with a vendor? Your 

organization should also consider other questions: 

 What can we build in-house that we cannot get, or do not want to get, from a 

vendor? 

 Can we get what we want from a vendor? 

 Will we need to settle for a system that does not completely meet our needs? Or, 

will we need to spend resources to modify the product to meet our needs? 

 Do we have the technical capability to build the system on our own and keep up 

with evolving data standards (e.g., federal reporting requirements)? If so, can we 

do a better job than a vendor? 

 Can we complete the task more quickly than a vendor? Or, can we meet deadlines 

without external assistance? 

 What if we lose key staff during the project? 

 Can we reduce project risk with a vendor? 

 Can we save money by building on our own or will it be cheaper to buy? 

(Consider ongoing maintenance costs for a vendor-built system and the savings 

you might have if knowledge resides in-house.) 

 What will be the ultimate trade-off between relying on a vendor and developing 

the expertise in-house? 

 What procurement approaches have other states taken? 

Why build? 

Some states prefer to keep the job in-house. They may have great staff with the necessary 

expertise and extra resources available to direct those staff members to the new project, 

or the ability to hire additional staff to do the work. Some states believe that, if you can 

afford to do it, build the LDS in-house. This, they say, is ultimately more cost effective 

because add-ons to vendor-built systems can cost a lot of money. If you have the 

capability, keeping the knowledge and ownership “in the family” will free you from the 

potentially costly prospect of having to rely on a vendor for ongoing services. Some states 

also feel that, in general, it takes just as much effort to work with vendors to make sure 

specifications are met as it does to build applications in-house. And working with staff may 

help ensure long-term sustainability because important knowledge about the data, source 

systems, reporting needs, making modifications, and other key issues are already in-house. 

And in fact, some research findings suggest that if an agency’s capabilities are strong, or 

even just moderate, building is the better option: the organization should use its resources 

to tackle the project on its own and to further develop its staff expertise rather than bring 

in outside help. The staff’s greater understanding of the organization’s culture and goals 

may, in the end, compensate for a less technical skill set (Nevo et al. 2007). 

72 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
Why buy? 

Other states see hiring a vendor as the best way to go. This may be because they like a 

product on the market and see no need to “reinvent the wheel” when it has been used 

elsewhere with satisfactory results. States often purchase such off-the-shelf solutions and 

“brand” them—that is, putting their education agency’s logo on a store-bought solution. By 

using a vendor, the state is freed from the details and can focus on the big picture and on 

selling the system to stakeholders. 

Another reason to use a vendor is that an agency with limited resources and 

capabilities may find doing the work in-house to be unrealistic, even if it wants to. If the 

state does not have a staff member with the same level of knowledge as a vendor’s project 

manager, or the capacity to build its desired product on its own, it may need outside 

assistance. In fact, research suggests that when internal IT capabilities are weak, the support 

of a knowledgeable, external consultant can boost productivity and reduce overall costs 

(Nevo et al. 2007). In addition, state hiring regulations that limit the size of SEA staff or 

impose other constraints may also make buying a more attractive idea. These rules can 

make it difficult to hire programmers. In this case, or if the organization is having trouble 

finding or keeping the necessary in-house staff, hiring vendors may be the simpler option. 

A short timeline might also necessitate the use of vendors as the number of 

programmers, architects, business analysts, project managers, senior developers, and other 

staff they have available may allow the project to move much more quickly than it could 

if shouldered by in-house personnel. Provided your organization can find a vendor(s) and 

product(s) that will meet your stakeholders’ needs, a commercial, off-the-shelf solution 

may also offer economies of scale in terms of leveraging design, including scope of data 

elements and requirements, warehouse structure, data model, data dictionary, submission 

procedures, etc.; as well as with development, testing, enhancement, support, optimization, 

documentation, and methodology across many customers. Whatever the reason, done 

correctly, choosing to bring in a vendor can be a very rewarding and successful course of 

action. Of course, every state’s experiences will be different, and careful self- and needs 

assessments are vital whether you build or buy. 

On Building 

If your organization has decided it can handle the project without the assistance of a 

vendor, either the necessary staff are on board or new staff members will be hired. A 

central challenge your organization may face is finding and holding on to skilled technical 

staff. These experts are often drawn away from education institutions to the private sector 

for reasons such as higher pay. Many states have lost key team members in this way and 

such losses caused major setbacks to their project. Therefore, before you decide to build 

a solution, ask your organization how it will safeguard against this loss of staff and the 

knowledge they hold. If staffing is an issue, going with a vendor may be a more secure 

option. 

Book Two of Four: Planning and Developing an LDS 73 
On Buying 

If you have decided not to go it alone, your organization has chosen to buy a product 

and/or services from a vendor. You must now figure out what you need, what is available, 

what kind of relationship you want with your vendor, and how much you are willing to 

spend. Then you will need to write a good request for proposals (see chapter 14), interview 

some bidders, and select a vendor (or consortium of vendors). Finally, you must forge an 

effective relationship with the chosen vendor to get the product you require. 

Many agencies use outside contractors and consultants to build their LDS or various 

components of the systems. These agencies may have one in-house employee who is the 

project manager, while consultants serve as programmers and system builders. Other states 

hire project managers as part of the vendor team. State agencies without the necessary 

expertise rely more heavily on their vendors and often seek a consulting firm they will not 

need to micromanage. However, states should be somewhat wary of depending too much 

on vendors, since they could unexpectedly close their doors or scale back their services. 

States with considerable in-house knowledge may seek a relationship that gives them 

significant control over the vendor’s work. These states act as CEOs overseeing the work 

of others. In this case, vendors may be hired to do the technical and programmatic work, 

and may even participate in committees to bridge the divide between their staff and the 

in-house executive levels. 

Choosing a vendor 

States and vendors: 

Who’s using who for what? 

The Data Quality Campaign (DQC) offers a table of vendors listing the number of 

states they work for and on which of the DQC’s Ten Essential Elements they were 

hired to work. It can be accessed at http://www.dataqualitycampaign.org/survey/ 

vendors, and includes a line labeled “In-House” that shows the number of states 

building each of the components on their own. (Note: The table is based on 2008 

survey results.) 

Once you have written and sent out the request for proposals (see chapter 14), bids and 

proposals will come in from competing vendors. Now, of course, you have to decide 

which vendor to hire. You should use the proposals to narrow the field to a small set of 

top contenders. Keep in mind that vendors vary in many ways, including in their bidding 

practices. 

In making your decision, look beyond the vendors’ marketing to the details of what 

they can actually provide. Two core principles to follow are: know the product, and know 

the team. Rather than taking the lowest bidder, be sure the vendor can actually fulfill 

the contract, at the agreed-upon price. Look at the vendors’ references, prior work, and 

available staff to try to evaluate them. Consider their experience working with states similar 

74 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
to yours in terms of size and student population. Will their solution accommodate your 

specific needs? Do they have enough staff to deal with possible turnover? Do they have 

the staff to adequately service all of their contracts, oversee the work, and complete quality 

deliverables on time? Or are they stretched too thin? In the end, careful selection may save 

you the costs of hiring a second vendor later on to complete the work or fix issues left by 

your original consultant. Also, as you will be spending a lot of time with this team, social 

considerations are also important. Will you like working with the project manager and 

staff? 

Many agencies hold face-to-face interviews with a set of prospective vendors. These 

meetings offer the advantage of allowing you to meet the team of people with whom you 

will actually be working. Consulting firms should give presentations, demonstrate their 

products, field questions, and be allowed a chance to ask some questions of their own. A 

review board might be created by the education agency to evaluate the various vendors 

and should include agency as well as local leaders and prospective users of the system to 

get a variety of perspectives and insight. Other agencies do not hold face-to-face vendor 

interviews. They make do with reference checks, answers to standardized questionnaires, 

and phone and online meetings. At the very least, they say you can learn a good deal about 

a vendor by the types of questions they ask. What is their level of technological knowledge 

and capability? How familiar are they with education data and their nuances? For instance, 

a vendor might have an extensive background in finance data and might give you a 

product that would work well for a financial institution, but not so well for an education 

agency. Make sure the vendor is ready for the task at hand. 

Working with a vendor 

Create a Wiki 

One state is in the planning phase of developing a wiki, a “piece of server 

software that allows users to freely create and edit web page content using 

any web browser.”* This resource will be used to create a more collaborative 

environment in which the state and its vendors can work. The wiki will allow the 

state to post documents and receive feedback from vendors. After a substantial 

amount of information is accumulated, the wiki may eventually evolve into a 

user guide on the LDS. 

*Source: http://wiki.org 

Do not expect a vendor to hold your hand and guide you through the project design 

process. While many vendors have extensive experience supporting education agencies, 

others are not experts in education data or needs. In this case, many decisions will be 

left up to the client organization—you. A state or district must therefore have a complete 

understanding of what is planned in order to ensure their needs are met. Detailed planning 

must be done upfront and cannot be avoided. At the same time, the vendor team should 

have some knowledge of the education agency’s data, or be willing to learn about the 

nuances of education data (for example, the importance of striking the right balance 

between confidentiality and not eliminating too many dropouts from the dataset). If 

necessary, ensure you have the time commitment from the in-house project manager, 

Book Two of Four: Planning and Developing an LDS 75 
as well as the subject area experts and IT staff, to educate the vendor on source systems, 

reporting needs, validation of work, and other important issues. 

Make sure the vendor staff working on your project are up to par. Some vendors will 

give you a top-of-the-line project manager, for instance, while the technical and subject 

area staff doing most of the work may not be sufficiently qualified. Having an in-house 

staff member who is as qualified as the vendor’s project manager or at least able to evaluate 

the work being done will be particularly important in such a case. Without knowledgeable 

in-house staff who can monitor the vendor’s progress, you will be to some extent at the 

vendor’s mercy and may end up with a deficient system. You need an evaluator who 

understands the technology and can keep the project on track. The in-house project 

manager should hold frequent meetings with the vendor to obtain updates, assess progress, 

and address problems as they arise. Keeping minutes at these meetings is also good 

practice, as this helps keep everyone on track and on the same page. 

Everyone wants to get along, and many states have excellent relationships with 

their vendors. But if a project seems to be getting off track or falling behind, be firm 

and hold your vendor accountable for failing to deliver. Trust and deference should be 

balanced against vendor management and the need to get things done on time and to your 

specifications. Some vendors tend to give clients a bare bones version of their specified 

product—meeting the requirements of your RFP but not going the extra mile to create the 

ideal result. Even if the vendor is going to give you a no-frills product, you still want a 

quality version of what you asked for and were promised. If necessary, do not be afraid to 

fire your vendor and bring someone else in if the project is not succeeding and the working 

relationship cannot be salvaged. 

Plan for knowledge transfer 

Knowledge transfer is the passage of information or expertise from one organization to 

another, or from one part of an organization to another. From the early stages of your 

vendor search, your agency should strongly consider knowledge transfer and training. 

Indeed, a knowledge transfer plan is vital and should be drawn out in advance, perhaps 

even specified in your RFP, and implemented along the way or once the project is nearing 

completion. The first step is, make sure you have enough staff in place to maintain what 

is built. Diligent documentation of everything done by the vendor, as well as by in-house 

staff, for the entire project is also crucial to efficient knowledge transfer. Additionally, 

your staff should work alongside the vendor for some time, or receive sufficient training 

from the developer to understand how the system works. Before the vendor hands off 

the project, in-house staff should be involved at least once in various processes to learn 

their structures and contents so that they have a grasp of what the LDS contains, how it is 

structured and formatted, how to load data, how to make modifications to reflect changing 

requirements, and other key issues. Additionally, the in-house responsibility must be 

clearly defined so the transition from vendor to purchaser runs smoothly. 

Some states indicate that, once the vendor’s main tasks are completed, long-term 

contracts are not always cost effective, but this depends on your situation. On the one 

hand, developing in-house expertise may be advantageous so you are not dependent on 

outside support. On the other, developing the expertise required to fully support a system 

may be cost prohibitive for your agency, and relying on a vendor may be cheaper in the 

long run. 

76 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
 

 

 

 

 

 

Writing a Strong Request 

for Proposals 

Chapter 14 

I f

i

 your organization has chosen to buy an LDS, or at least some portion of 

t, you will need to write a request for proposals (RFP) to solicit bids from 

potential vendors. This section will provide a brief introduction to effective RFP writing. 

The information in this section draws heavily on Nancy Smith’s report, Lessons Learned: 

Writing Requests for Proposals for Statewide Student Data Systems in Education. Additional 

resources in appendix C include an extensive collection of sample RFPs from state 

education agencies (SEA). 

Basic checklist of RFP development 

 Know your needs before you write an RFP or begin development work.
 

 Follow your state’s procurement procedures.
 

 Define clear, specific, and measurable requirements.
 

 Delineate critical requirements from optional enhancements.
 

 Focus on both process and outcomes.
 

 Build in safeguards to protect your agency.
 

Define Your Needs First 

Before writing the RFP, you should have already completed thorough self- and needs 

assessments (if not, see chapters 6 and 8) and be ready to translate your requirements 

into an RFP that tells vendors what you want to purchase. If you are not sure of your 

requirements when you write the RFP, the bidding and development processes may 

become frustrating ordeals for everyone involved, and the completed system might not 

serve you well. If you did not separate needs assessment into a discrete activity completed 

prior to the development stage, analysts will have to define needs as they go along and 

solutions-building may be slowed down. 

Book Two of Four: Planning and Developing an LDS 77 
Start with an “RFI” 

Before you write an RFP, consider issuing a request for information (RFI). In 

this document, an agency presents the problems it wants to solve and asks 

vendors to propose solutions, pose questions, and make suggestions. The 

advantages of issuing an RFI are that agencies receive early feedback from 

experts on the feasibility and cogency of their planned system, as well as 

learn about the range of possible solutions available to them before they write 

their RFP. 

Take the time to discover your program area needs. How will you need to restructure 

the data for the LDS? What data will the users need? How will you need to deliver your 

data to users? How will you need to secure your data to protect student privacy? Try 

to foresee as many needs as possible to limit the number of requirements added during 

development. RFPs issued by other education agencies for similar purposes can provide 

helpful ideas (see appendix C for additional resources, including some RFP examples). 

A quality RFP contains a nearly exhaustive collection of clear and specific requirements. 

Encourage vendors to be creative, but do not leave room for misinterpretation. For 

instance, do not write “etc.” in your RFP—say what “etc.” represents. Do not ask for 

“quality”—explain how you will measure “quality.” 

Writing the RFP 

Once you have defined your needs and those of your stakeholders, the next step is to write 

a strong RFP. This will help you find the right vendor, with the right product and services 

for the right price. Some states have a standardized method of writing RFPs, others are 

less formal and prescriptive, while still others do not write RFPs at all but may still need to 

document their requirements in some form. 

The team assigned to write the RFP should include staff from various education 

agency departments. Ideally, the group should also comprise subject area experts who 

understand the content and business operations, policies and regulations, and the potential 

budget; as well as technology staff who understand the codes and bytes. District and/or 

school representatives should also be involved in defining requirements. 

There are many approaches to writing RFPs. For instance, you may write one RFP 

for the entire system, compose one RFP with multiple components and let vendors choose 

which parts they will bid on, or create a separate RFP for each system component. If you 

take the latter approach, however, do not break the project into too many parts and focus 

too closely on specific pieces of LDS functionality. If the goal is a comprehensive LDS, 

writing RFPs for many small pieces may attract specialized vendors outside of education 

while dissuading those with more comprehensive, LDS-specific products and services. 

Additionally, the RFP may be based on process or on solutions (deliverables or 

outcomes). In a process-based RFP, an education agency defines the “process” the vendor 

must follow. For instance, you might require a standardized development process based 

on the systems development life cycle, though this approach might discourage vendors 

from responding if their products have already been through a development process. 

A solutions-based RFP describes the desired results, but leaves the vendor to determine 

78 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
the process through which they are completed. A combination of these approaches may 

be best as each may lead to potential problems. Focusing solely on process may affect 

deadlines or result in final products that do not meet agency needs. On the other hand, 

concentrating on outcomes alone may ignore important details, such as the frequency of 

collections and integration with other applications. 

Another recommended approach is to find a system that you like at another 

education agency, get a copy of that agency’s RFP, and make necessary modifications based 

on your specific needs or your agency’s RFP-writing guidelines. This will save you the 

trouble of starting the RFP from scratch. 

Anatomy of an RFP 

An RFP may be composed of the following sections: 

	 Project overview and administrative section 

 Consists of an overview or summary of the problem, along with administrative 

information about the expected management of the RFP. 

	 Technical requirements 

 Provides technical requirements and enough information to help the vendor 

understand the request and write a firm proposal. 

	 Management requirements 

 Specifies the expectations for managing and implementing the project. 

 Supplier/vendor qualifications and references 

 Asks for vendor qualifications and references; ideally specifies the format in 

which these should be supplied. 

 Supplier’s section 

 Allows vendors to include information about themselves or about the project 

that is not specifically requested. 

 Pricing section 

 Specifies how the pricing information should be provided. 

The Forum has more detailed information... 

... about RFP writing: 

 Forum Unified Education Technology Suite (2005) 

http://nces.ed.gov/forum/pub_tech_suite.asp 

Book Two of Four: Planning and Developing an LDS 79 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 Contract and license agreement 

Contains the purchase contract, nondisclosure agreements, and other legal 

documents. 

 Appendices 

Includes relevant information that is too bulky to include in the RFP such as 

network diagrams, outlines and requirement studies. 

(Reprinted by permission from Smith [2004], Lessons Learned: Writing Requests for Proposals 

for Statewide Student Data Systems in Education.) 

Tips for RFP writing 

Here are some pointers to keep in mind during the RFP writing process: 

 Provide background 

Tell the vendor about the state’s data environment, database development 

standards, business requirements, and other relevant information about how the 

agency does business. Without this information, vendors may make erroneous 

assumptions in their proposal and through the development process. 

 Do not specify the solution 

Focus on the problems you want to solve and the existing environment in which 

the solution must work, not on a particular, perceived solution. Give vendors 

room to suggest their own solutions to those problems. 

 Define clear, concise, and measurable requirements 

Distinguish critical requirements from nice-to-have add-ons. The vendor needs 

to know which requirements to include in the bid and which are optional 

enhancements that can be priced separately. This will make it easier for vendors 

to bid, and easier for you to interpret the proposals. 

 Specify costs, or do not 

You may elect to specify the amount you will pay for a solution and have 

vendors say, “Yes, we can deliver for that price.” Or you may withhold cost 

estimates and ask vendors to make offers. One state uses the latter approach, 

but asks vendors to submit their costs in sealed envelopes. The agency staff then 

assess each proposal before opening the envelopes, judging each bid on its merits 

rather than by its price tag. Once the judging period is over, if favorite vendors’ 

prices turn out to be too high, the agency tries to negotiate a more acceptable fee. 

 See how much others paid 

When trying to assess how much a solution should cost, ask colleagues in other 

agencies for information on their costs. Consider differences between your state 

or district and theirs (student populations, varying requirements, etc.) and make 

necessary adjustments. 

 Inquire about management practices 

Ask how the vendor manages projects in terms of communication, risk 

management, methodology, quality reviews, etc. 

80 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
 

 

 

 

 

 

 

 

 

 

 Do not be too aggressive in your timeline 

Be realistic and try to strike a balance between quality and speed. In addition to 

deadlines for the vendor, consider your agency’s end of the bargain. What will 

you need to do or provide, and how much time will you need? In the end, your 

own education agency may cause the project to fall behind schedule. One state 

representative suggested that instead of emphasizing deadlines, agencies should 

put emphasis on quality results. 

 Ask for references from the vendor, perhaps even for the 

staff with whom you will be working 

Take the time to check these references and see what others thought of the 

services they received. Are the staff technically capable? Does the vendor have 

enough staff to support the endeavor? 

 Ask vendors to propose additional features that could 

improve the system 

Though you may not choose to accept their suggestions, they may have good 

ideas that could improve the functionality and utility of the system. 

 Get third-party opinions 

Have a variety of people beyond the writing team review the RFP to make sure it 

conveys your intent and is free of ambiguities. 

 Realize that your rfp may not be perfect 

The requirements, as you have written them, may be flawed and could cause 

problems during implementation. Allow vendors to offer corrections or 

alternatives. 

On the safe side 

What might you include to protect your organization? In your RFP, consider including 

some safeguards and stipulations to ensure you get what you need from your vendor. Here 

are a few examples: 

Staffing 

 Maintain the right to change the vendor’s project manager if you are unhappy 

with his or her service. 

 Specify a limit to vendor staff turnover, perhaps to 25 percent or less. However, 

even if you say in the RFP that staff can only change by so much or in certain 

ways, know that turnover happens and you cannot always avoid it. If change must 

occur, ensure that the required knowledge transfer will be paid for by the vendor, 

not the taxpayers. 

 Specify that the project manager and staff you interview will actually work with 

you throughout the project. One education agency staffer suggested that vendors 

sometimes send their best team to the interview and then assign staff of lesser 

skill to the project once the deal is sealed. 

 Require that at least some vendor staff work onsite rather than remotely, if 

possible. Scheduling online or phone meetings can be a hassle and may impede 

regular and effective communication. If the vendor must work offsite, specify a 

number of onsite meetings and require that they cover travel expenses. Onsite 

Book Two of Four: Planning and Developing an LDS 81 
 

 

 

 

 

 

 

 

 

 

 

 

vendor staffing offers many advantages. For instance, both in-house staff and 

consultants are more apt to ask questions if the vendor is onsite. In addition, 

more meetings can be held and they can be more easily convened with specific 

individuals or groups of people. If vendor staff will be allowed to work remotely 

(an option that may be more cost-effective and provide more flexibility), devise 

an agreement to specify requirements such as the number of site meetings 

needed (and who will pay the expenses); communication planning such as virtual 

meetings, emailing, and instant messaging; and service-level agreements for 

off-site staff. 

Deliverables 

 Define deliverables in clear and unambiguous terms. Make sure these 

requirements have measurable results that hold the vendor accountable. 

Alternatively, ask the vendor to propose measures that can be used to assess the 

deliverables. 

 Require that deliverables meet certain quality criteria before they can be delivered 

(grammatically correct, spell-checked, formatted according to standards, etc.). 

Avoid wasting time correcting simple vendor mistakes. 

 Distinguish critical requirements from optional enhancements. In addition to 

the benefits discussed above, this can protect you by preventing the vendor 

from claiming that a requirement was an add-on that should cost extra. Clear 

specifications up front save time, frustration, and money later. 

Sustainability 

	 	Ask vendors to include ongoing maintenance and support in their proposals. 

Agencies report that many vendors just want to build the system, get paid, and 

then leave the in-house staff on their own. 

	 	Request or define a plan for knowledge transfer so that your staff is able to 

manage the system when the vendor leaves. 

Communication 

	 	If you intend to rely on your vendor to communicate with the education 

community, make sure they have an organized approach to communication 

or, alternatively, include in the proposal a communication plan that they must 

follow. 

• Determine who is going to write the communications: the vendor may know 

the product best and should have a communications staff—avoid having the 

project manager or a technology staffer write communications unless a piece 

is technical in nature. 

• Define who the audience(s) will be.

• Define a communications schedule, or at what milestones or intervals 

communications will be sent out. 

• Define the ways communications will be made. 

• Figure out, in general, what the communications will say (see chapter 12).

	 	Ask the vendor to provide sample communications. 

Take care when writing an RFP. Getting it right will save time and resources; and will 

help you get the system your stakeholders need. 

82 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
Are We “There” Yet? 

Evaluating Your LDS 

Chapter 15 

T he best time to think about LDS evaluation is during the planning phase. A 

key question to consider early on, and throughout the development process, 

is “How will we know if we’ve been successful?” Planning ahead of time for the evaluation 

ensures you will know how to measure progress right from the beginning of development 

and implementation. 

During the planning phase, an agency should establish project goals and objectives 

that will become the very basis for the evaluation; as well as metrics on which the process 

will rely. In turn, your evaluator will be able to convert these objectives into evaluation 

designs and measurement protocols. Then, right from the start of the implementation, 

you (or the evaluator) can collect baseline data against which later data measures will be 

compared. 

How well does your LDS deliver in terms of your stated criteria for success? How 

does it enhance operations, improve data quality, and facilitate better data-driven 

decisionmaking? Your ability to evaluate the system depends largely on how you planned 

the project from the very beginning. How clear was your vision of the desired system? Did 

you identify unambiguous and measurable goals? Clarity at the beginning will help you 

and your stakeholders evaluate if your efforts are paying off, and help you make 

adjustments to refine the system over time (see chapters 8 and 14). 

The Subjectivity of “Success” 

Without clearly stated goals and criteria for measuring how close you come to reaching 

them, success may be depend on the user's vantage point. For instance, technologically 

savvy users may expect cutting edge applications, while average users may be more 

interested in usability and user-friendly interfaces (Miller 2000). The expectations set at 

the beginning of the project may also affect perceptions. Leaders of the marketing and 

outreach effort, for instance, will affect how others view the system. Care should therefore 

be taken not to create unrealistically high expectations. If you paint too ambitious a 

picture when trying to win support, stakeholders may end up dissatisfied with the results 

(Staples 2002). 

Book Two of Four: Planning and Developing an LDS 83 
 

 

 

 

 

 

 

 

 

Measuring LDS Success 

Before assessment begins, and ideally even before the project begins, agency leaders or 

stakeholder groups serving on an evaluation committee should reflect on some basic 

questions: 

 Who will evaluate the system? (Agencies commonly bring in an outside 

consultant.) 

 From whose perspective will the system’s success be judged (agency leaders, IT 

staff, teachers, etc.)? 

 What criteria will be used to judge success? (Identify well-defined assessment 

criteria aligned with the established LDS goals.) 

 Will greater emphasis be put on timeliness or on progress? 

 What types of information should be used to gauge success (for instance, 

qualitative or quantitative)? 

 What methods will be used to gather and analyze the information? 

And later, when interpreting evaluation results, consider changes that may have 

affected the project and the ability to achieve the original goals. Did political leadership 

change? Was there significant staff turnover in-house or in vendor staff? Were new laws 

passed? Did technology changes have any impact? Did new resources become available, or 

did a downturn in the economy affect the budget? 

Evaluation methods 

Evaluators may use various tools to gauge system success. Commonly used methods include 

 surveys (online and paper); 

 case studies; and 

 direct observation. 

Evaluation criteria and questions 

There are many possible questions you can use to evaluate your system's success. Table 5 

presents a collection of some evaluation criteria and questions. These should be answered 

using one or more of the above methodologies. While these questions are qualitative in 

nature, you should include solid, measurable criteria in the evaluation whenever possible. 

84 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
Table 5. Sample evaluation questions 

Cost and cost 

effectiveness 

Measurable 

benefits/impact 

Development and 

implementation 

Awareness and 

engagement 

Use 

Perceptions 

Lessons learned 

 How long did the project take? 

 To what extent does the LDS match the original vision and design? 

 To what extent has the implementation team included broad input into the system design? 

 To what extent has the system been made available to a wide variety of stakeholders? 

 To what extent has the system been “marketed” to those stakeholder groups? 

 What resources were used? 

 What savings in time and money were realized? 

 How have operations been made more efficient? 

 To what extent has the system helped improve academic performance? 

 To what extent has the system helped close achievement gaps? 

 How has the system been used to target and enhance professional development? 

 How has the system enhanced administrator decisionmaking? 

 In what ways and to what extent has the system saved the enterprise money? 

 How aware are stakeholders of the system? 

 How well do stakeholders understand the system? 

 What level of stakeholder engagement has been achieved? 

 How widespread is the use of the system? 

 Which system components are used and how are they applied? 

 How widespread was training participation? 

 How well has the system met reporting and decisionmaking needs? 

 How helpful are the reporting and analysis tools? 

 Has the system made data adequately available and accessible? 

 Are users getting the data they need? 

 How effective has professional development been? 

 What professional development initiatives have been created to improve LDS use? 

 What are the barriers to effective use of the system? 

 What improvements can be made to the system? 

 How can professional development efforts be improved? 

Sources:	IES	2007,	Juillerat	and	James	2006,	and	Simon	2006.

Book Two of Four: Planning and Developing an LDS 85 

Appendix A 

Overview of the LDS 

Guide Series 

The Traveling Through Time: The Forum Guide to Longitudinal Data Systems series consists of 

four guides intended to help state and local education agencies meet the many challenges 

involved in developing a robust longitudinal data system (LDS), populating them with 

quality data, and using this new information to improve the education system. 

Book One of Four: What is an LDS? focuses on the fundamental questions of what an 

LDS is (and what it is not), what steps should be taken to achieve a sound and successful 

system, what components make up an ideal system, and why such a system is of value 

to education. Book Two of Four: Planning and Developing an LDS, discusses the critical 

planning and development phases of an LDS project, from stakeholder engagement and 

needs assessment all the way through to system evaluation. Book Three of Four: Effectively 

Managing LDS Data will explore several fundamental challenges of data management, 

focusing on data governance, data quality, privacy, and security. Finally, Book Four of Four: 

Advanced LDS Usage, will address the effective utilization of LDS data, discussing the 

users and uses of the data; and emphasize the need for effective training and professional 

development. The figure on the next page lays out the major issues covered in each of the 

four books in this Forum guide series. 

Book Two of Four: Planning and Developing an LDS 87 
88 Traveling Through Time: The Forum Guide to Longitudinal Data Systems

Book I: What Is an LDS?

 Understanding what an LDS is (and is not)

 Appreciating the organizational steps needed to 

institute and effectively use an LDS

 Identifying the technical features and 

capabilities of an effective LDS and the 

additional features that can enhance the 

system’s utility

 Recognizing the benefits of an LDS

Book II: Planning and 

Developing an LDS

 Engaging stakeholders

 Describing the current system 

 Envisioning the desired system

 Defining needs, including data and functionality

 Gaining buy-in and funding

 Building relationships

 Writing an RFP 

 Building or buying a system or components

 Transferring knowledge (e.g., from developers 

to staff)

 Defining and measuring success

 Refining the system

Book III: Effectively 

Managing LDS Data

 Defining governance structure

 Defining roles and responsibilities

 Collaborating to improve data quality and 

streamline operations

 Managing changes to the system

 Training staff to ensure data quality

 Auditing/validating data at all levels 

 Establishing/following data standards

 Securing data to protect privacy 

 Providing users access to key data

Book IV: Advanced 

LDS Usage

 Collecting, storing, and delivering key data 

 Developing useful reports to fulfill common 

data requests and needs 

 Developing user-friendly data tools to facilitate 

access and analysis

 Training users to utilize the technology

 Building awareness, understanding, and 

analytical capacity

Book One: What is an LDS? is available online at: http://nces.ed.gov/forum/pub_ 2010805.asp

To order a print copy, please contact: 

ED Pubs

U.S. Department of Education

P.O. Box 22207

Alexandria, VA 22304

Or call toll free 1–877–4ED–PUBS or order online at http://www.edpubs.gov.
Appendix B 

References 

Aden, D. (2008). Enterprise Architecture Demystified. Government Technology. Retrieved 

from http://www.govtech.com/gt/articles/418008 on October 16, 2008. 

American National Standards Institute (ANSI 1975). Interim Report: ANSI/X3/SPARC 

Study Group on Data Base Management Systems. FDT (Bulletin of ACM SIGMOD), 7(2): 

1–140. 

Bellino, C. and Hunt, S. (2007). Auditing Application Controls. Altamonte Springs, Florida: 

The Institute of Internal Auditors. 

Berry, B.; Fuller, E.; and Reeves, C. (2007). Linking Teacher and Student Data to Improve 

Teacher and Teaching Quality. Washington, DC: Data Quality Campaign. 

Center for Strengthening the Teaching Profession (2008). Creating a Comprehensive Teacher 

Data System. Retrieved from http://dataqualitycampaign.org/files/publications-creating_ 

comprehensive_teacher_data_ system_report.pdf on December 4, 2009. 

Collins, L. and Fruth, L. (2007). The Right Data to the Right People at the Right Time: How 

Interoperable Data Help America's Students Succeed, June 2007. Washington, DC: Data 

Quality Campaign (DQC). 

Data Quality Campaign (DQC 2006a). Creating a Longitudinal Data System: Using Data to 

Improve Student Achievement. Washington, DC: Data Quality Campaign (DQC). 

Data Quality Campaign (DQC 2006b). Creating Longitudinal Data Systems: Lessons Learned 

from Leading States. Washington, DC: Data Quality Campaign (DQC). 

Data Quality Campaign (2009). 2009 Survey of State P–20 Data Collection Issues Related 

to Longitudinal Analsyis. Data Quality Campaign (DQC). Retrieved from http://www. 

dataqualitycampaign.org/files/2009_ DQC_ Annual_ Survey_-FINAL.pdf on January 8, 2010. 

Davis, M. (2008). Finding Your Way in a Data-Driven World. Education Week, Jan. 23, 2008. 

Education Information Management Advisory Consortium (EIMAC 2007a). Longitudinal 

Data Systems Task Force, March 2007 Meeting, Phoenix Arizona. Meeting Summary. 

Retrieved from http://www.ccsso.org/content/pdfs/LSDS0307%20Summary.doc on 

December 4, 2008. 

Education Information Management Advisory Consortium (EIMAC 2007b) LDS 

Subcommittee, Meeting Summary, October 2007. Retrieved from http://www.ccsso.org/ 

content/pdfs/101507%20LSDS%20Taskforce%20SUMMARY.doc on December 4, 2008. 

Book Two of Four: Planning and Developing an LDS 89 
Education Information Management Advisory Consortium (EIMAC 2008). Longitudinal 

Data System Roadmap. Student Longitudinal Data Systems Task Force. Council of Chief 

States School Officers. (Unpublished spreadsheet.) 

ESP Solutions (2007). Marketing Your Field of Dreams: The Process of Obtaining and Sustaining 

Buy-in. Austin, Texas: ESP Solutions Group. 

ESP Solutions (2008). D3M Framework for Building a Longitudinal Data System. Austin, 

Texas: ESP Solutions Group. 

Hirsch, J. (2008). Performance Management—Data Informed Decisions: Having Information 

Provides New Understandings and Insight. Consortium for School Networking Initiative. 

Retrieved from http://www.3d2know.org/presentations/Performance_ Management.pdf on 

November 4, 2009. 

Institute of Electrical and Electronics Engineers (IEEE 2000). IEEE Recommended Practice for 

Architectural Description of Software-Intensive Systems (IEEE Std 1471–2000). New York: IEEE 

Computer Society, Institute of Electrical and Electronics Engineers. 

Institute of Educational Sciences (IES 2007). Statewide Longitudinal Data Systems Grant 

Program: Summary of RFA Requirements FY 2007. Retrieved from http://nces.ed.gov/ 

programs/slds/pdf/FY07_SLDS_ Requirements.pdf on December 4, 2008. 

Juergens, M. (2006). Global Technology Audit Guide Four: Management of IT Auditing. 

Altamonte Springs, Florida: The Institute of Internal Auditors. 

Juillerat, B. and James, E. (2006). Evaluation of the Effectiveness of Ohio’s D3A2 Initiative. 

Ohio Department of Education. IES SLDS Grantee Meeting Presentation. 

Microsoft Corporation (2002). Microsoft Architecture Overview. Retrieved from http://msdn. 

microsoft.com/en-us/architecture/ms978007.aspx on October 4, 2009. 

Microsoft Corporation (2006). Regulatory Compliance Planning Guide. Microsoft. 

Retrieved from http://technet.microsoft.com/en-us/library/cc163116.aspx on February 2, 2009. 

Miller, H. (2000). Managing Customer Expectations. Information Systems Management 17. 

National Center for Education Evaluation and Regional Assistance (NCEE 2007). Getting 

the Evidence for Evidence-Based Initiatives: How the Midwest States Use Data Systems to 

Improve Education Processes and Outcomes (REL 2007–016). U.S. Department of Education. 

Washington, DC: Institute of Educational Sciences. 

National Forum on Education Statistics (2009). Every School Day Counts: The Forum Guide 

to Collecting and Using Attendance Data (NFES 2009–804). U.S. Department of Education. 

Washington, DC: National Center for Education Statistics. 

National Forum on Education Statistics (1998), Technology @ Your Fingertips. U.S. 

Department of Education. Washington, DC: National Center for Education Statistics. 

Nevo, S.; Wade, M.; and Cook., W. (2007) An Examination of the Trade-off Between 

Internal and External IT Capabilities. The Journal of Strategic Information Systems, 16(1); 5–23. 

90 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
Nunn, J. and Harper Mainzer, K. (2006). Longitudinal Data Systems: Summary of Current 

Issues and Potential Uses. Retrieved from http://nces.ed.gov/programs/slds/ldsshare/slds.aspx on 

December 4, 2008. 

Schools Interoperability Framework Association (2006). Analysis of Costs and Benefits 

Associated with Implementing SIF. Retrieved from http://www.sifinfo.org/upload/Docs/SIF_ 

Cost_ Benefit_ Analysis_ Summary_ June20_ Final.pdf on December 4, 2008. 

Seddon, P.; Staples, S.; and Patnayakuni, R. (1999). Dimensions of Information Systems 

Success. Communications of the Association for Information Systems. 2: Article 20. 

Simon, A. (2006). ADE LDS Evaluation. IES SLDS Grantee Meeting Presentation. 

Retrieved from http://nces.ed.gov/programs/slds/zip/granteemeeting06_4b.zip on November 

11, 2009. 

Smith, N. (2004). Lessons Learned: Writing Requests for Proposals for Statewide Student Data 

Systems in Education. Denver, Colorado: Education Commission of the States. 

Staples, D.; Wong, I.; and Seddon, P. (2002). Having Expectations of Information Systems 

Benefits That Match Received Benefits: Does It Really Matter? Information & Management, 

40 (2) 115–131. 

Tennessee Department of Education. Tennessee Department of Education Project Management 

Oversight Committee and Project Management Process. (Unpublished internal document.) 

U.S. Department of Education (2004). National Education Technology Plan. Retrieved from 

http://www.ed.gov/about/offices/list/os/technology/plan/2004/plan_ pg14.html#steps. 

Wilson, L. and Nunn, J. (2007). Stakeholder Involvement in Maryland. MIS Conference 

Presentation. 

Book Two of Four: Planning and Developing an LDS 91 

Appendix C 

Additional Resources 

Chapter 3. Project Management for an LDS Implementation 

Technology @ Your Fingertips 

National Forum on Education Statistics (1998). 

http://nces.ed.gov/pubs98/98293rev.pdf 

See sections 5.2 and 5.3 for some general lessons on project management of a technology 

solution. 

Governance Structure, LDS Project Team Organization, and 

Sustainability 

IES SLDS Grantee Meeting: Session V, SLDS (2006). 

http://nces.ed.gov/programs/slds/gms_presentations.asp#2 

Representatives from several states discussed how their project teams were organized within 

their respective state education associations (SEA), and how decisions were made. 

South Carolina’s Governance Structure (Diagram) 

South Carolina Department of Education (2007).
 

Available via LDS Share at http://nces.ed.gov/programs/slds/ldsshare/slds.aspx
 

This diagram illustrates South Carolina’s LDS project governance structure.
 

Chapter 4. Engaging Stakeholders: Bringing Everyone Along 

Alaska’s Initial Solicitation Letter Commissioner Signature 

(Invitation to Participate in Stakeholder Groups) 

Alaska Department of Education (2007).
 

Available via LDS Share at http://nces.ed.gov/programs/slds/ldsshare/slds.aspx
 

This item is a draft letter inviting stakeholder participation to Alaska’s Unity Project.
 

Don’t Get Lost in Translation: The LDS and the Data Divas, 

Data Geeks, and Data Duffers 

South Carolina Department of Education (2007). 

http://nces.ed.gov/whatsnew/conferences/statsdc/2007/ppt/II_C.zip 

This presentation from NCES’s 2007 Summer Data Conference reviews South Carolina’s 

approach to engaging stakeholders, getting the right people involved in the LDS project, 

and facilitating effective communication between these key players from various 

backgrounds to optimize results. 

Book Two of Four: Planning and Developing an LDS 93 
 

 

 

 

Stakeholder Involvement in Maryland 

Maryland Department of Education (2006). 

http://nces.ed.gov/programs/slds/zip/granteemeeting06_1b.zip 

This presentation given at the 2006 IES SLDS Grantee Meeting provides an overview of 

Maryland’s efforts to engage a wide variety of stakeholders. It includes the various types of 

stakeholders, their roles, and how needs assessment is conducted. Lessons learned and best 

practices are offered. Slide notes are also included. 

LDS Share 

The following documents are available at http://nces.ed.gov/programs/slds/ldsshare/slds.aspx: 

Alaska Unity Project: Functional Stakeholder 


Organization Chart 


Alaska Department of Education (2007).
 

This item illustrates the Unity Project’s Functional Stakeholder Organization.
 

External Communications Plan Example 

Michigan Department of Education (2007).
 

This document represents an example and/or template external communications 


plan developed by the state of Michigan.
 

Stakeholder Involvement in Designing and Developing 

Statewide Longitudinal Data Systems 

Wisconsin Department of Public Instruction (2006). 

http://nces.ed.gov/programs/slds/zip/granteemeeting06_1a.zip 

This short presentation given at 2006 IES SLDS Grantee Meeting lists the challenges 

of engaging stakeholders. 

Chapter 5. Building State–District Relationships 

What Is the Role of the SEA in Providing LEAs With Access to 

Key Data for Instruction? 

Council of Chief State School Officers. Education Information Management Advisory 


Consortium LDS Taskforce (2007).
 

What can the SEA do to help local agencies in the LDS effort? This brief covers the 


establishment of an identifier system, professional development, benchmarking, data 


management and business intelligence, and legislative support.
 

Creating Longitudinal Data Systems: Lessons Learned by 

Leading States 

Data Quality Campaign (2006). 

http://www.dataqualitycampaign.org/resources/115 

This document summarizes findings and offers lessons learned from case studies of four 

leading states that vary in terms of LDS progress, public and political support for LDS, and 

the focus of their LDS efforts. 

94 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
Chapter 6. Self-Assessment: You are Here, But... Where 

Exactly is That? 

Core Elements Completion for Establishing a Statewide 

Longitudinal Data System 

Institute of Education Sciences (2009). 

http://nces.ed.gov/programs/slds/pdf/matrixofcoreelements.pdf 

This checklist can help your organization gauge where it is and where it needs to focus 

efforts in establishing an LDS. Agencies can reflect on which components they have, 

which they want or do not care to have, and the status of support and funding for those 

components. 

Forum Unified Education Technology Suite 

National Forum on Education Statistics (2005). 

http://nces.ed.gov/forum/pub_tech_suite.asp 

See chapter 2 for a discussion of needs-assessment and, to a lesser extent, self-assessment. 

2009–10 Survey Results 

Data Quality Campaign (DQC 2009). 

http://www.dataqualitycampaign.org/survey 

This is a sample survey of state education agencies, organized around the DQC’s 10 

essential elements and 7 state actions. It asks key questions about agency progress towards, 

and activities related to, these elements and actions. 

Chapter 7. Enterprise Architecture 

Enterprise Architecture 

Microsoft Developer Network. 

http://msdn.microsoft.com/en-us/architecture/bb469938.aspx 

This site includes links to a host of articles on EA including an introduction to Microsoft’s 

approach to EA and a discussion of the most popular EA frameworks. 

Enterprise Architecture as Strategy 

Ross, J.; Weill, P.; and Robertson, D. (2006). 

Boston, Massachusetts: Harvard Business School Press 

This book structures and explains the enterprise architecture (EA) approaches that topperforming private sector organizations have taken. 

Enterprise Architecture Frameworks 

Below is a list of some of the most popular EA frameworks. Microsoft notes that none of 

these EA approaches is sufficient for all situations and recommends using the most useful 

portions of each one to meet your organization’s needs. 

Book Two of Four: Planning and Developing an LDS 95 
 

 

 

 

 

Federal Enterprise Architecture (FEA) 

http://www.whitehouse.gov/omb/e-gov/fea and http://www.whitehouse.gov/sites/default/ 

files/omb/assets/fea_ docs/FEA_ Practice_Guidance_ Nov_ 2007.pdf 

The federal government has been a leader in developing and using EA and the 

private sector is following suit. 

The Open Group Architecture Framework and Architecture 

Development Method 

http://www.opengroup.org/architecture/togaf8-doc/arch/toc.html 

The Open Group Architecture Framework (TOGAF) is a detailed method and 

supporting tools for developing an EA. It may be used freely by any organization for 

its own EA development. Architecture Development Method (ADM) is a “reliable, 

practical method... for defining business needs and developing an architecture that 

meets those needs, utilizing the elements of TOGAF and other architectural assets 

available to the organization.” In other words, ADM is the process for creating 

TOGAF. 

Zachman Framework 

Zachman International. 

http://zachmaninternational.com/index.php/the-zachman-framework 

One of the earliest EA frameworks, this is a logical structure for organizing the 

descriptive representations important to the management of an enterprise and the 

development of its systems. 

For more in-depth discussion of these and other frameworks, see Microsoft 

Corporation’s A Comparison of the Top Four Enterprise-Architecture Methodologies at 

http://msdn.microsoft.com/en-us/architecture/bb466232.aspx. 

Chapter 8. Needs-Assessment: Defining “There” 

Technology @ Your Fingertips 

National Forum on Education Statistics (1998). 

http://nces.ed.gov/pubs98/98293rev.pdf 

Chapter 2 contains a helpful discussion of needs assessment. While not specific to an LDS 

project, it offers some relevant guidelines for finding out what functional and technological 

needs your stakeholders have. It also offers tips on creating a statement of needs. 

Creating a Statewide Educational Data System for 

Accountability and Improvement: A Comprehensive Information 

and Assessment System for Making Evidence-Based Change at 

School-, District-, and Policy-Levels. 

Felner, R.; Bolton, N.; Seitsinger, A.; Brand, S.; and Burns, A. (2008). Psychology in the 

Schools 45(3), 235–256. 

This article provides a good example of how designers can think big from the outset, 

defining their needs and developing a system guided by more ambitious goals than 

96 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
 

 

 

 

 

 

usual. It covers many areas of the education system that may be measured, including 

developmental, educational, fiscal, and policy conditions, as well as students' academic 

performance and developmental and educational needs. 

Effective Use of Electronic Data Systems: A Readiness Guide for 

School and District Leaders 

Learning Point Associates and the Educational Service Agency (ESA) Alliance of the 

Midwest (2006). 

http://www.learningpt.org/pdfs/datause/DataReadinessTool.pdf 

This guide is intended to help districts figure out what they want from a data system, 

identify appropriate questions to ask about a system, and assess the district’s readiness and 

capacity to use one effectively. 

Beyond Test Scores: Leading Indicators for Education 

Annenberg Institute for School Reform (2008). 

http://www.annenberginstitute.org/pdf/leadingindicators.pdf 

This study looks at leading indicators used to identify signs of academic progress before 

test scores come in. These indicators may be useful in helping agencies think about the 

questions they want to explore and the data needed to answer them. 

LDS Share 

The following documents are available at http://nces.ed.gov/programs/slds/ldsshare/slds.aspx: 

Longitudinal Data Systems: Summary of Current and 

Potential Issues 

Maryland Department of Education (2006). 

This document discusses current issues and potential uses of LDSs. It reviews the 

basic requirements of an LDS as well as the options that can be built into the 

system. It highlights themes and poses questions that may be used to inform the 

design of a needs-assessment process. 

Maryland Longitudinal Data System Needs Assessment 

Guidelines for Internal Stakeholders 

Maryland Department of Education (2006). 

This document represents materials that Maryland’s project staff use to conduct a 

needs assessment for internal stakeholders. The document also contains proposed 

topics for needs assessment for external stakeholders. 

Questionnaire for Teacher Specialists 

South Carolina Department of Education (2007). 

This questionnaire lists a host of questions the state’s LDS can answer. It was given 

to teacher specialists who were asked to rate the questions in terms of the value their 

answers would offer. 

SLED Focus Group Requirements Traceability Matrix 

District of Columbia (2007). 

This file summarizes the findings of numerous focus groups the District of 

Columbia held with a variety of stakeholders: the mayor’s office, community/ 

principal groups, and functional groups (e.g., special education, charter schools, 

Book Two of Four: Planning and Developing an LDS 97 
funders, researchers). Potential data requirements for a state longitudinal data 

warehouse (SLED) were identified and rated on a scale of one to three by all 

stakeholder groups. The file also includes the focus group schedule. 

Chapter 9. Data: Knowing What You Have, Identifying What 

You Need 

NCES Handbooks Online 

National Center for Education Statistics. 

http://nces.ed.gov/programs/handbook 

These handbooks provide guidance on consistency in data definitions and maintenance 

for education data, so that such data can be accurately aggregated and analyzed. Use 

this searchable web tool to find standard data elements for students, staff, and education 

institutions; along with standard definitions and recommended values or responses for 

each element. 

National Education Data Model 

U.S. Department of Education. 

http://nces.ed.gov/forum/datamodel 

The National Education Data Model catalogs the data used in P–12 education and 

describes the relationships among those data. It is designed to be used as a reference tool 

to facilitate the identification, merging, and matching of data across different systems; to 

provide similar descriptions across local education agency (LEA) systems, across LEAs, and 

from LEAs to the state and federal government; and to specify the content and structure of 

logical and physical data models. 

Common Data Elements for Education Technology Assessment 

State Educational Technology Directors Association. 

http://www.setda.org/toolkit/nlitoolkit/cde/cde06.htm 

This toolkit presents the common data elements SETDA has identified for tracking state 

progress towards the goals of the technology section of the No Child Left Behind Act of 

2001 (NCLB). 

Every School Day Counts: The Forum Guide to Collecting and 

Using Attendance Data 

National Forum on Education Statistics (2009). 

http://nces.ed.gov/forum/pub_ 2009804.asp 

This guide defines “attending/present” and “not attending/absent,” categorizes attendance 

codes in an exhaustive and mutually exclusive way, and supports improved attendance data 

quality and comparability between states and districts. 

Accounting for Every Student: A Taxonomy for Standard 

Student Exit Codes 

National Forum on Education Statistics (2006). 

http://nces.ed.gov/forum/pub_ 2006804.asp 

This guidebook presents “best practice” advice for tracking and maintaining information 

98 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
on enrollment status. It presents an exhaustive and mutually exclusive taxonomy of exit 

codes. 

Longitudinal Data Systems: Summary of Current and 

Potential Issues 

Maryland Department of Education (2006).
 

Available via LDS Share at http://nces.ed.gov/programs/slds/ldsshare/slds.aspx
 

This document summarizes information related to longitudinal data systems in education, 


exploring current issues and potential uses. It will guide the external stakeholder needs 


assessment process, highlighting themes and posing questions to be addressed in 


interviews, surveys, and focus groups.
 

Getting the Evidence for Evidence-Based Initiatives: How the 

Midwest States Use Data Systems to Improve Education Processes 

and Outcomes 

National Center for Education Evaluation and Regional Assistance (2007). Issues and 

Answers Report (REL 2007–016). U.S. Department of Education, Institute of Education 

Sciences. 

http://ies.ed.gov/ncee/edlabs/regions/midwest/pdf/rel_ 2007016_sum.pdf 

This report reviews the progress of several midwestern states in developing LDSs and the 

use of data systems in general. Based on interviews with SEA officials and federal agency 

staff, the authors review the work that was done, the challenges that were faced, and the 

current requirements being pursued by the states. 

Illinois State Board of Education SIS Data Elements, Approved 

Codes and Indicators 

Illinois State Board of Education. 

http://www.isbe.net/sis/html/data_elements.htm 

This page contains documentation for all data elements collected through the Illinois State 

Board of Education’s (ISBE) student information system (SIS), including demographics, 

enrollment, assessment, English language learners, early childhood education, discipline, 

student identifiers, and more. 

Creating a Comprehensive Teacher Data System 

Center for Strengthening the Teaching Profession. 

http://www.dataqualitycampaign.org/resources/94 

Page 6 of this report includes a list of data elements suggested for a comprehensive staff 

data system. 

Forum Guide to Core Finance Data Elements 

National Forum on Education Statistics (2007). 

http://nces.ed.gov/forum/pub_ 2007801.asp 

This document provides an overview of key finance data terms. 

Linking Spending and Student Achievement: Managing Inputs, 

Processes, and Outcomes 

Data Quality Campaign (DQC 2008). 

http://dataqualitycampaign.org/files/meetings-dqc_quarterly_issue_brief-042408.pdf 

Book Two of Four: Planning and Developing an LDS 99 
This issue brief explores the benefits of linking financial and student achievement data. It 

discusses the challenges of improving current financial systems, and includes two case studies. 

Facilities Information Management: A Guide for State and Local 

Education Agencies 

National Forum on Education Statistics (2003). 

http://nces.ed.gov/forum/pub_ 2003400.asp 

This guide provides a framework for identifying a basic set of school facilities data 

elements and definitions that will meet the information needs of school and community 

decisionmakers, school facility managers, and the general public. 

Linking Teacher and Student Data to Improve Teacher and 

Teaching Quality 

Data Quality Campaign (DQC 2007). 

http://dataqualitycampaign.org/files/Meetings-DQC_ Quarterly_ Issue_ Brief_031207.pdf 

This issue brief discusses information about teachers that may be tracked in an LDS, and 

the benefits those data can offer when linked to student data. 

Victoria Bernhardt Web Page 

http://www.eyeoneducation.com/authors/bernhardt_v.asp 

This author has written a number of publications discussing the types of data that may be 

useful to educators at the local level. This site links to many of these resources. 

Ruth Johnson Web Page 

http://www.sagepub.com/authorDetails.nav?contribId=520833 

This author has written a number of publications discussing the types of data that may be 

useful to educators at the local level. This site links to a many of these resources. 

Chapter 10. Some Critical “Abilities”: Interoperability and 

Portability 

The Right Data to the Right People at the Right Time: How 

Interoperable Data Help America’s Students Succeed 

Data Quality Campaign (DQC 2007). 

http://dataqualitycampaign.org/files/Meetings-DQC_ Quarterly_ Issue_ Brief_061307.pdf 

This paper reviews the needs for, benefits of, and concurrent efforts to establish 

interoperable education systems. It offers several key definitions, a case study section, and 

a list of interoperability examples from other industries. 

SIF Association Implementation Toolkit 

SIF Association (2009). 

http://www.sifinfo.org/us/tool_ kit.asp 

This collection of documents are intended to help education institutions with the SIF 

implementation process. It includes planning questions (scope, desired automation, data 

needs, expected changes), RFP language, an implementation planning toolkit, and SIF 

support resources. 

100 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
 

Analysis of Costs and Benefits Associated with Implementing SIF 

SIF Association (2006). 

http://www.sifinfo.org/upload/Docs/SIF_Cost_ Benefit_ Analysis_ Summary_ June20_ Final.pdf 

This third-party study looks at three school districts’ experiences with SIF implementation 

and concludes that SIF standards helped to dramatically improve data interoperability, 

student achievement, funding increases, and student services. This study may be used to 

bolster the case for SIF implementation. 

Data Interoperability in PK–12 School Applications 

SIF Association (2006). 

http://www.sifinfo.org/upload/press/532964_ Interoperability%20and%20pk12.pdf 

This brief discusses interoperability and SIF, its benefits to P–12 education, current 

progress, and suggestions for moving forward. 

SIFA University 

SIF Association (2006). 

http://www.sifinfo.org/us/sifau.asp 

This site offers online overview modules and information on training courses offered by 

the SIF Association. 

SCORM and SIF—Leveraging Work for Successful Solutions 

SIF Association (2006). 

http://www.sifinfo.org/upload/press/289bza_scorm and sif_ final.pdf 

This brief discusses two organizations’ standards for interoperability: SIFA’s standards 

(SIF) and those of Advanced Distributed Learning (ADL), which are called SCORM. 

This piece introduces each and suggests that organizations consider both when pursuing 

interoperability. 

SIF Specifications 

SIF Association. 

http://www.sifinfo.org/us/sif-specification.asp 

This page includes the latest SIF implementation specifications. Access is free. 

Data Portability Project 

http://www.dataportability.org 

The Data Portability Project promotes the idea that individuals have control over their data 

(regardless of the entity that holds the data) and can determine how the information can be 

use and by whom. 

In the news... 

The SIFication of America 

Schater, R. (2009). District Administration, March 2009. 

http://www.districtadministration.com/viewarticle.aspx?articleid=1970 

This article reports on districts' growing effort to streamline data exchange and 

management through the adoption of SIF. 

Book Two of Four: Planning and Developing an LDS 101 
Chapter 11. Staying “There”: Ensuring System Sustainability 

Wisconsin Case Study: Building a Student-Level Longitudinal 

Data System 

Data Quality Campaign (DQC 2006). 

http://www.dataqualitycampaign.org/files/State_Specific-Wisconsin_ 2006_Site_Visit.pdf 

Pages 4–5 of this case study include a list of estimated costs to the state to implement a 

vendor-developed student ID system and student-level enrollment data collection. Costs 

are broken up by year and source. A discussion of costs and extra burdens imposed on 

districts is also provided. 

South Carolina Case Study: Building a Student-Level 

Longitudinal Data System 

Data Quality Campaign (DQC 2007). 

http://www.dataqualitycampaign.org/files/Publications-South_Carolina_Case_ Study_ 

Building_a_Student-level_ Longitudinal_ Data_ System-110207.pdf 

Pages 6–7 of this case study include a list of estimated costs to the state to implement SIF 

standards, bring in extra staff, upgrade infrastructure, and implement a state data manager 

and a new data warehouse. A discussion of costs and extra burdens imposed on districts is 

also provided. 

State Data to Improve Achievement 

Washington Office of the Superintendent of Public Instruction (2006). 

http://www.dataqualitycampaign.org/resources/317 

This document is a request to the Washington Legislature to fully fund a two-year, $2.9 

million statewide longitudinal student database project. The document sets the stage with 

relevant history; and discusses the need for the proposed system, as well as the associated 

costs. The system is meant to facilitate daily data extractions from districts, replace existing 

single-year achievement databases with longitudinal achievement and demographic ones, 

and build and implement value-add tools. 

Chapter 12. Marketing and Communicating About Your LDS 

Creating Longitudinal Data Systems: Lessons Learned by 

Leading States 

Data Quality Campaign (DQC 2006). 

http://www.dataqualitycampaign.org/resources/115 

This document summarizes findings and offers lessons learned from case studies of four 

leading states that vary in terms of LDS progress, public and political support for LDS, and 

the focus of their LDS efforts. 

Strategic Communications Planning 

Fleet, D. (2008). DaveFleet.com. 

http://davefleet.com/2008/08/strategic-communications-planning-a-free-ebook 

Written from a business perspective but intended for a broader audience including the 

public sector, this free, electronic book guides the reader through the development of 

102 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
 

a communications plan. Covered marketing goals relevant to an LDS project include 

educating stakeholders, building support, and creating demand. This resource may be 

useful from the early stages of stakeholder analysis through crafting and spreading your 

message, estimating costs, and evaluating the results of your efforts. 

Marketing Your Field of Dreams: The Process of Obtaining and 

Sustaining Buy-in 

ESP Solutions (2007). 

http://www.espsolutionsgroup.com/espweb/assets/files/ESP_ Marketing_ Field_ Dreams_ORG.pdf 

Written from the state’s perspective, this document offers strategies on how to use various 

marketing techniques to gain buy-in for a technology project. A marketing plan is outlined 

to help your organization gain stakeholder support. 

Education Leadership Toolkit: The Communication Plan 

National School Boards Foundation. 

http://www.nsba.org/sbot/toolkit/tcp.html 

This free, online resource addresses issues around technology and education and offers 

many tips, articles, case studies and other resources for education leaders. Though it is 

designed for a district-level audience, much of the advice may be used more broadly. 

This section focuses on developing a successful communication plan. See the community 

involvement section as well: http://www.nsba.org/sbot/toolkit/ComInv.html. 

DATA/KIDS III Projects: Integrated Communications Plan 

Oregon Department of Education (2009). 

http://www.oregondataproject.org/content/communications-plan 

This document details the state’s communication plan and governance structure for its 

professional development program and data system. 

Developing Political Support and the Will to Build and Use 

Longitudinal Education Data Systems 

Pennsylvania Department of Education and Florida Department of Education (2006) 

http://nces.ed.gov/whatsnew/conferences/statsdc2006/conference/presents/iv_ d.zip 

This presentation reviews lessons learned by Pennsylvania and Florida when marketing 

their states’ LDSs. Emphasis is on having and consistently communicating a clear vision, 

and on being persistent to maintain support over the long term. 

External Communications Plan Example 

Michigan Department of Education (2007). 

Available via LDS Share at http://nces.ed.gov/programs/slds/ldsshare/slds.aspx 

This document presents a template for organizing committees and facilitating ongoing 

communications about an LDS project. It outlines the various roles and activities of these 

groups and their members, as well as suggesting ground rules for meetings. 

Marketing and Communicating the LDS Project—Florida 

Florida Department of Education. 

http://nces.ed.gov/programs/slds/zip/granteemeeting06_10a.zip 

This short presentation offers some tips on marketing an LDS. 

Book Two of Four: Planning and Developing an LDS 103 
Chapter 13. Procurement Planning: Build or Buy? 

An Examination of the Trade-off Between Internal and External 

IT Capabilities 

Nevo, S.; Wade, M.; and Cook., W. (2007). Journal of Strategic Information Systems, 16(1); 5–23. 

This study explores the tension between internal and external IT capabilities on the 

realization of enhanced IT productivity. It focuses on short-term, small scale consulting 

rather than the large scale, multiyear outsourcing that often occurs in LDS development. In 

that respect, its applicability is limited here. However, it offers insights into the relationship 

between in-house staff and vendors; and may help enlighten your decision on whether to 

buy or build, and how to proceed down the path you choose. 

Data Quality Campaign Vendors Page 

http://www.dataqualitycampaign.org/survey/vendors 

This page lists which states are using vendors to help them develop LDS components. 

These results are from the 2008 DQC Survey. 

Technology @ Your Fingertips 

National Forum on Education Statistics (1998). 

http://nces.ed.gov/forum/pub_98293.asp 

See pages 50–57 for general discussions on conducting a build vs. buy analysis, finding a 

product to fit your needs, and other related issues. While not specific to an LDS project, 

this resource offers some relevant tips. Page 57 includes a list of sample questions that 

might be used to interview vendor references. 

Data Tools for School Improvement 

Bernhardt, V. (2005). Educational Leadership 62(5), 66-69. 

http://eff.csuchico.edu/downloads/datatools.pdf 

This article discusses the three types of data tools available—data warehouses, student 

information systems, and instructional management systems—and their uses; as well as how 

to make good purchasing decisions, get what your organization wants and needs, how to 

deal with vendors, etc. 

Chapter 14. Writing a Strong Request for Proposals 

Lessons Learned: Writing RFPs for State Data Systems 

Smith, N. (2004). Education Commission of the States/InfoSynthesis. 

http://www.dataqualitycampaign.org/resources/112 

This report summarizes findings from a survey of three state education agencies and offers 

a host of RFP writing principles and tips. Two RFP examples are provided at the end of the 

document. 

104 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
Writing RFPs for State Data Systems: Lessons Learned 

Washington Office of Superintendent of Public Instruction, Michigan Center for 

Educational Performance and Information, and Ohio Department of Education (2007). 

http://nces.ed.gov/whatsnew/conferences/mis/2007/presentations/vi_c.zip 

This file includes presentations from three states on lessons learned in RFP writing. 

LDS Share 

Institute of Education Sciences (IES), U.S. Department of Education. 

http://nces.ed.gov/programs/slds/ldsshare/slds.aspx 

Coordinated by the IES Statewide Longitudinal Data Systems Grant Program, this web 

resource contains a collection of state-level RFPs submitted by education agencies. 

SIF Association Recommended RFP Language 

SIF Association (2009). 

http://www.sifinfo.org/us/upload/implement/BF3E8E_SIF%20RFP%20Language.pdf 

This document offers language to include in your RFP if your agency wants to purchase 

an application based on SIF standards. It requests documentation of SIF certification, SIF 

implementation experience, agent costs, zone integration server(s) offered, SIF Association 

participation, and SIF support offered to clients. 

Virginia Case Study: Building a Student-Level Longitudinal Data 

System 

Data Quality Campaign (DQC 2006). 

http://www.dataqualitycampaign.org/files/State_Specific-Virginia_ 2006_Site_Visit.pdf 

See pages 6–7 for helpful tips on RFP writing and evaluating bids. 

Forum Guide to Decision Support Systems: A Resource for 

Educators 

National Forum on Education Statistics (2006). 

http://nces.ed.gov/forum/pub_ 2006807.asp 

See this guide’s appendix for an overview of issues that may be included in a decision 

support system RFP. There is considerable overlap between these requirements and those 

that might be included in an LDS. 

Technology @ Your Fingertips 

National Forum on Education Statistics (1998). 

http://nces.ed.gov/pubs98/98293rev.pdf 

Pages 57–59 of this resource offer a brief introduction of RFPs and a sample RFP table of 

contents to help guide the writing process. It also discusses requests for information (RFI) 

and interviewing references, including questions to ask interviewees. 

Book Two of Four: Planning and Developing an LDS 105 
 

 

Chapter 15. Are We “There” Yet? Evaluating Your LDS 

Evaluation of the Effectiveness of Ohio’s D3A2 Initiative 

Juillerat, B. and James, E. (2006). Ohio Department of Education. 

http://nces.ed.gov/programs/slds/zip/granteemeeting06_4a.zip 

This presentation introduces Ohio’s LDS evaluation plans and summarizes the state’s 

system as well as the criteria for, and methods used in, a planned system evaluation. 

Arkansas Department of Education LDS Evaluation 

Metis Associates. IES SLDS Grantee Meeting Presentation (2006). 

http://nces.ed.gov/programs/slds/zip/granteemeeting06_4b.zip 

This presentation by Arkansas’ evaluation vendor gives an overview of the state’s LDS and 

discusses its approach to evaluating that system. A basic overview of evaluation criteria, 

methods, and timeline are provided. 

Everybody Loves Evaluation 

Indiana Department of Education (2008). 

http://nces.ed.gov/programs/slds/zip/08_ F_ 08a.zip 

This presentation details the state’s evaluation process. It outlines the purposes of the LDS 

evaluation, procedures used, activities, and sample findings. 

You Say You Want Evaluation 

Arkansas Department of Education (2008). 

http://nces.ed.gov/programs/slds/zip/08_ F_ 08b.zip 

This presentation reviews the findings of a survey of districts intended to evaluate 

perceptions of the state’s LDS. Questions seek to ascertain users’ perceptions about the 

system’s most useful functions, ease and frequency of use, success in improving data 

quality, and effects on student performance. 

LDS Share 

The following evaluation-related documents are available at 

http://nces.ed.gov/programs/slds/ldsshare/slds.aspx: 

Longitudinal Data System Implementation and Impact 

Evaluation RFP 

Ohio Department of Education (2006). 

Ohio solicited competitive proposals for its LDS Implementation and Impact 

Evaluation of 2006–2009, and this RFP is the result of that request. The purpose 

of the evaluation is to measure the implementation and impact of the LDS, 

including related professional development, quality, and effectiveness in meeting the 

reporting and decision support needs of all its key stakeholders; and, ultimately, its 

effectiveness in closing achievement gaps and in generating academic improvement 

in all students. 

106 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
 

 

 

 

 

Supporting Ohio’s Longitudinal Data System through 

Evaluation: A Proposal to the Ohio Department of Education 

Ohio Department of Education (2006). 

This document is the winning proposal, submitted by Hezel Associates LLC, to 

Ohio’s LDS Implementation and Impact Evaluation RFP. 

Supporting D3A2 Professional Development through 

Evaluation: A Report to the Ohio Department of Education 

Ohio Department of Education (2007). 

This report offers key findings from an evaluation of Ohio’s LDS, focusing on 

data usage practices and professional development efforts. Key findings and 

recommendations are followed by discussions of methodology and more detailed 

discussions of evaluation findings. 

Supporting D3A2 Professional Development through 

Evaluation: A Preliminary Annual Report to the Ohio 

Department of Education 

Ohio Department of Education (2007). 

This LDS evaluation report follows up on the previous one. It offers key 

recommendations for future LDS efforts, focusing on implementation and 

expanding on earlier recommendations to improve professional development efforts. 

Key findings and recommendations are followed by discussions of methodology and 

more detailed discussions of evaluation findings. 

Book Two of Four: Planning and Developing an LDS 107 

Appendix D 

Information Technology 

Audits 

Considering the large outlay of resources involved in LDS development, many states will 

want to review how their tax dollars are being spent. As a result, your agency may become 

involved in federal or state auditing processes and procedures. Agencies developing LDSs 

may therefore be interested in the available types of information technology audits and 

what they encompass. 

According to Wikipedia, an information technology audit, or information 

systems audit, is “an examination of the controls within an information technology (IT) 

infrastructure. An IT audit is the process of collecting and evaluating evidence of an 

organization’s information systems, practices, and operations. The evaluation of obtained 

evidence determines if the information systems are safeguarding assets, maintaining data 

integrity, and operating effectively and efficiently to achieve the organization’s goals or 

objectives. These reviews may be performed in conjunction with a financial statement 

audit, internal audit, or other form of attestation engagement.” (http://en.wikipedia.org) 

If your agency can afford it, the creation of an internal audit team can be very 

beneficial, potentially saving your organization time and money. The first step in 

developing an audit team is to define the information technology audit functions. A 

useful approach is to define your IT function into layers: IT management, technical 

infrastructure, applications, and external connections (Juergens 2006). 

As Juergens states, IT management “comprises the set of people, policies, 

procedures, and processes that manage the IT environment.” This layer also includes 

system monitoring, programming, planning, management of outsourced vendors, and 

IT governance. Technical infrastructure usually refers to systems that underlie, support, 

and enable the primary business applications such as operating systems, databases, and 

networks. Applications may be classified into two categories: transactional and support. 

Transactional applications consist primarily of software that processes and records business 

transactions. Support applications are specialized software programs that facilitate business 

activities but generally do not process transactions such as email programs, fax software, 

and design software. External connections are those that your network connects to, such as 

the Internet or other business networks. 

Organizations developing LDSs may be interested in two types of controls to be 

audited: application controls and information technology general controls. 

Application controls are specific to each application and relate to the transactions and 

data pertaining to each computer-based application system. The objectives of application 

controls are to ensure the completeness and accuracy of records and the validity of the 

entries made resulting from programmed processing activities. Examples of application 

controls include: 

Book Two of Four: Planning and Developing an LDS 109 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 Input controls: These controls are used mainly to check the integrity of data 

entered into a business application, whether the data is entered directly by staff, 

remotely by a business partner, or through a web-enabled application or interface. 

Data input is checked to ensure it remains within specified parameters. 

 Processing controls: These controls provide an automated means to ensure 

processing is complete, accurate, and authorized. 

 Output controls: These controls address what is done with the data, and should 

compare actual results with the intended result by checking the output against 

the input. 

 Integrity controls: These controls monitor data being processed and in storage to 

ensure they remain consistent and correct. 

 Management trail: Processing history controls, often referred to as an audit 

trail, enables management to identify the transactions and events they record by 

tracking transactions from their source to their output, and by tracing backward. 

These controls also monitor the effectiveness of other controls, and identify errors 

as close to their sources as possible. (Bellino and Hunt 2007) 

General controls apply to all system components, processes, and data present in an 

organization or systems environment. The objectives of these controls are to ensure the 

appropriate development and implementation of applications; as well as the integrity of 

program and data files, and of computer operations. The most common general controls are 

 logical access controls over infrastructure, applications, and data;
 

 system development life-cycle controls;
 

 program change management controls;
 

 physical security controls over the data center;
 

 system and data backup and recovery controls; and
 

 computer operation controls. 


(Bellino and Hunt 2007)
 

Risk Assessment 

One of the many benefits of an information audits is to identify risks to your organization. 

Juergens (2006) has identified the following as possible risks: 

 availability, when the system is unavailable for use; 

 security, when unauthorized access to systems occurs; 

 integrity, when the data is incomplete or inaccurate; 

 confidentiality, when information is not kept secret; 

 effectiveness, when the system does not deliver an intended or expected function; 

and 

 efficiency, when the system causes a sub-optimal use of resources. 

110 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
The Audit Process 

The following is an example provided by Microsoft Corporation’s Regulatory Compliance 

Planning Guide (Microsoft 2006) on what to expect during an IT audit. 

1. Plan the audit (auditor). 

2. Hold audit kickoff meeting (auditor/organization). 

3. Gather data and test IT controls (auditor/organization). 

4. Remediate identified deficiencies (organization). 

5. Test improved controls (auditor/organization). 

6. Analyze and report findings (auditor). 

7. Respond to findings (organization). 

8. Issue final report (auditor). 

IT audits create challenges for organizations by forcing them to come into, and 

maintain, compliance. But audits also offer benefits, increasing your organization’s process 

improvement and business advantage. 

Book Two of Four: Planning and Developing an LDS 111 

Appendix E 

Forum and Other NCES 

Resources 

The Forum Guide to Data Ethics (NFES 2010–801) 

http://nces.ed.gov/forum/pub_2010801.asp 

Every day, educators collect and use data about students, staff, and 

schools. Some of these data originate in individual student and staff 

records that are confidential or otherwise sensitive. And even those 

data that are a matter of public record, such as aggregate school 

enrollment, need to be accessed, presented, and used in an ethically 

responsible manner. While laws set the legal parameters that govern 

data use, ethics establish fundamental principles of “right and 

wrong” that are critical to the appropriate management and use of education data in the 

technology age. This guide reflects the experience and judgment of experienced data 

managers; while there is no mandate to follow these principles, the authors hope that the 

contents will prove a useful reference to others in their work. 

Crisis Data Management: A Forum Guide to Collecting and 

Managing Data about Displaced Students (NFES 2010–804) 

http://nces.ed.gov/forum/pub_ 2010804.asp 

This publication provides guidelines that can be used by elementary 

and secondary education agencies to establish policies and procedures 

for collecting and managing education data before, during, and after a 

crisis. 

The Forum Guide to Metadata: The Meaning Behind Education 

Data (NFES 2009–805) 

http://nces.ed.gov/forum/pub_ 2009805.asp 

This guide offers best practice concepts, definitions, 

implementation strategies, and templates/tools for an audience of 

data, technology, and program staff in state and local education 

agencies. This resource was developed to improve these audiences’ 

awareness and understanding of metadata and, subsequently, the 

quality of the data in the systems they maintain. 

Book Two of Four: Planning and Developing an LDS 113 
Every School Day Counts: Forum Guide to Collecting and 

Using Attendance Data (NFES 2009–804) 

http://nces.ed.gov/forum/pub_ 2009804.asp 

This guide offers best practice suggestions on collecting and using 

student attendance data to improve performance. It includes a 

standard set of codes to make attendance data comparable across 

districts and states. The publication includes real-life examples of how 

attendance information has been used by school districts. 

NCES Handbooks and NCES Handbooks Online 

http://www.nces.ed.gov/programs/handbook 

The NCES Handbooks are a valuable source of metadata 

for organizations and individuals interested in education data. These print and online 

resources define standard education terms for students, staff, schools, local education 

agencies (LEA), intermediate education agencies, and state education agencies (SEA). The 

Handbooks are intended as reference documents for public and private organizations, 

including education institutions and early childhood centers; as well as education 

researchers and other users of education data. In order to improve access to this valuable 

resource, NCES has also developed the NCES Handbooks Online, a web-based tool that 

allows users to view and download information via an electronic table of contents, a drilldown finder, element name and first letter searches, and advanced query options. 

National Education Data Model 

http://nces.ed.gov/forum/datamodel/index.aspx 

The National Education Data Model (NEDM) is the first 

non-proprietary, national education data model developed to help schools, LEAs, and 

states design or guide the selection of systems for instructional delivery, data-driven 

decisionmaking, data collection, operations, and reporting. The model provides a national 

blueprint to help schools evaluate and improve instructional tools; communicate needs to 

their umbrella agency or to vendors; enhance the movement of student information from 

one LEA to another; and, in the end, have better tools to inform instruction. NEDM can 

be used by educators, vendors, and researchers to understand the information required for 

teaching, learning, and administrative systems. 

Managing an Identity Crisis: Forum Guide to Implementing New 

Federal Race and Ethnicity Categories (NFES 2008–802) 

http://nces.ed.gov/forum/pub_ 2008802.asp 

This best-practice guide was developed to help state and local 

education agencies implement the new federal race and ethnicity 

categories, thereby reducing redundant efforts within and across states, 

improving data comparability, and minimizing reporting burden. 

Users may select and adopt strategies that will help them quickly begin 

the process of implementation in their agencies. 

114 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
Forum Guide to Core Finance Data Elements (NFES 2007–801)
 

http://nces.ed.gov/forum/pub_ 2007801.asp 

This publication establishes current and consistent terms and 

definitions for maintaining, collecting, reporting, and exchanging 

comparable information related to education finances. It is designed 

to accompany Financial Accounting for Local and State School 

Systems: 2003 Edition by identifying common reporting requirements 

and defining frequently used indicators and calculations that use data 

elements from accounting and other data systems. 

Forum Curriculum for Improving Education Data: A Resource for 

Local Education Agencies (NFES 2007–808) 

http://nces.ed.gov/forum/pub_ 2007808.asp 

This resource supports efforts to improve the quality of education 

data by serving as training materials for K–12 school and district staff. 

It provides lesson plans, instructional handouts, and other resources; 

and presents concepts necessary to help schools develop a culture for 

improving data quality. 

Forum Guide to Decision Support Systems: A Resource for 

Educators (NFES 2006–807) 

http://nces.ed.gov/forum/pub_ 2006807.asp 

This Forum guide was developed to help the education community 

better understand what decision support systems are, how they are 

configured, how they operate, and how they might be developed and 

implemented in an education setting. 

Forum Guide to Virtual Education (NFES 2006-803)
 

http://nces.ed.gov/forum/pub_ 2006803.asp 

This publication offers recommendations for collecting accurate, 

comparable, and useful data about virtual education in elementary and 

secondary education settings. It highlights policy questions and data 

elements critical to meeting the information needs of policymakers, 

administrators, instructors, and parents of students involved in virtual 

education. 

Book Two of Four: Planning and Developing an LDS 115 
Forum Guide to the Privacy of Student Information: 

A Resource for Schools (NFES 2006–805) 

http://nces.ed.gov/forum/pub_ 2006805.asp 

This publication was written to help school and local education agency 

staff better understand and apply the Family Educational Rights and 

Privacy Act (FERPA), a federal law that protects the privacy interests 

of parents and students with respect to information maintained in 

student education records. It defines terms such as “education records” 

and “directory information,” and offers guidance for developing 

appropriate privacy policies and information disclosure procedures 

related to military recruiting, parental rights and annual notification, videotaping, online 

information, media releases, surveillance cameras, and health-related information. 

Accounting for Every Student: A Taxonomy for Standard Student 

Exit Codes (NFES 2006–804) 

http://nces.ed.gov/forum/pub_ 2006804.asp 

This publication was developed to help education agencies develop 

effective information systems for tracking students’ enrollment status. 

It presents a student-level exit code taxonomy for states and districts 

that accounts for 100 percent (not 90 or 110 percent) of all students. It 

also offers “best practice” advice regarding tracking students, collecting 

exit codes data, and distinguishing among high school completion 

credentials. 

Forum Guide to Education Indicators (NFES 2005–802) 

http://nces.ed.gov/forum/pub_ 2005802.asp 

This publication provides encyclopedia-type entries for 44 commonly 

used education indicators. Each indicator entry includes a definition, 

recommended uses, caveats and cautions, related policy questions, data 

element components, a formula, commonly reported subgroups, and 

display suggestions. The document will help readers better understand 

how to appropriately develop, apply, and interpret commonly used 

education indicators. 

Forum Guide to Building a Culture of Quality Data 

(NFES 2005–801) 

http://nces.ed.gov/forum/pub_ 2005801.asp 

This publication focuses on data entry: getting things done right at the 

source. It recommends a practical process for developing a “Culture 

of Quality Data” based around individual tip sheets for individuals 

involved in providing data, including principals, teachers, office staff, 

school board members, superintendents, data stewards, and technology 

staff. 

116 Traveling Through Time: The Forum Guide to Longitudinal Data Systems 
Forum Unified Education Technology Suite (2005)
 

http://nces.ed.gov/forum/pub_tech_ suite.asp 

This publication presents a practical, comprehensive, and tested 

approach to assessing, acquiring, instituting, managing, securing, 

and using technology in education settings. It is written for 

individuals without extensive experience with technology who have been tasked with 

leading technology initiatives in a school or district setting,. 

Forum Guide to Protecting the Privacy of Student Information: 

State and Local Education Agencies (NCES 2004–330) 

http://nces.ed.gov/forum/pub_ 2004330.asp 

This publication presents a general overview of privacy laws and 

professional practices that apply to information collected for, and 

maintained in, student records. The guide provides an overview of key 

principles and concepts governing student privacy; summarizes federal 

privacy laws; identifies issues concerning the release of information 

to both parents and external organizations; and suggests good data 

management practices for schools, districts, and state education 

agencies. 

Facilities Information Management: A Guide for State and Local 

Education Agencies (NCES 2003–400) 

http://nces.ed.gov/forum/pub_ 2003400.asp 

This publication provides a framework for identifying a basic set 

of school facilities data elements and definitions that will meet the 

information needs of school and community decisionmakers, school 

facility managers, and the general public. It presents recommendations 

for designing and maintaining an information system that addresses the 

condition, design, use, management, and financing of elementary and 

secondary education facilities. Commonly used measures, data elements, 

and additional resources for the practitioner are also included. 

Planning Guide for Maintaining School Facilities 

(NCES 2003–347) 

http://nces.ed.gov/forum/pub_ 2003347.asp 

This publication is intended to help school facilities managers plan 

for efficient and effective operations. It provides practical advice on 

a range of topics, including how to conduct a facilities audit, plan for 

maintenance to ensure smooth operations and avoid costly surprises, 

manage staff and contractors, and evaluate maintenance efforts. 

Book Two of Four: Planning and Developing an LDS 117