# Presentation: 272501

## 

December 13, 2007
Session: “Performance Measurements for Transportation Libraries”
Hank Zaletel, MTKN/Iowa DOT Library/CTRE
Maggie Sacco, Transportation Library Connectivity Pooled Fund Study

www.mtkn.org     www.libraryconnectivity.org 

- December 13, 2007
- Session: “Performance Measurements for Transportation Libraries”
- Hank Zaletel, MTKN/Iowa DOT Library/CTRE
- Maggie Sacco, Transportation Library Connectivity Pooled Fund Study
- _[www.mtkn.org](http://www.mtkn.org/)__    _ _[www.libraryconnectivity.org](http://www.libraryconnectivity.org/)__ _

## Performance Measurements for Transportation Libraries

- Hank Zaletel
- MTKN/IOWA DOT/CTRI
- Maggie Sacco
- Transportation Library Connectivity Pooled Fund Study

**Performance Measurements**** ****for Transportation Libraries**

## 
Definitions of Performance Measures

Ways to objectively measure the degree of success a program has had in achieving its stated objectives, goals, and planned program activities. ...www.ojp.usdoj.gov/BJA/evaluation/glossary/glossary_p.htm

Indicators of transportation system outcomes with regard to such things as average speed, reliability of travel, and accident rates.plan4operations.dot.gov/Glossary.asp

The actions that can be objectively observed and measured to determine if a task performer has performed the task to the prescribed standard.www.neiu.edu/~dbehrlic/hrd408/glossary.htm

Assessment methods based on student activities or products, as opposed to tests or surveys, to evaluate students' knowledge, skill and development.www.unk.edu/academicaffairs/assessment/Resources/index.php

Qualitative measures or indicators of progress toward specified outcomes or benchmarks.https://competitivesourcing.navy.mil/reference_documents/defs.cfm

_**Definitions of Performance Measures**_
- Ways to objectively measure the degree of success a program has had in achieving its stated objectives, goals, and planned program activities. ..._[www.ojp.usdoj.gov/BJA/evaluation/glossary/glossary_p.htm](http://www.ojp.usdoj.gov/BJA/evaluation/glossary/glossary_p.htm)_
- Indicators of transportation system outcomes with regard to such things as average speed, reliability of travel, and accident rates._[plan4operations.dot.gov/Glossary.asp](http://www.google.com/url?sa=X&start=1&oi=define&q=http://plan4operations.dot.gov/Glossary.asp&usg=AFQjCNEtijkLCHKrEBVNtOJ7DMhi7yqBfw)_
- The actions that can be objectively observed and measured to determine if a task performer has performed the task to the prescribed standard.www.neiu.edu/~dbehrlic/hrd408/glossary.htm
- Assessment methods based on student activities or products, as opposed to tests or surveys, to evaluate students' knowledge, skill and development.www.unk.edu/academicaffairs/assessment/Resources/index.php
- Qualitative measures or indicators of progress toward specified outcomes or benchmarks._[https://competitivesourcing.navy.mil/reference_documents/defs.cfm](https://competitivesourcing.navy.mil/reference_documents/defs.cfm)_

**Notes:**
A Google search produced the following 10 definitions of Performance Measures. They are all pretty similar, but when you’re approaching this as a group you need to make sure everyone agrees with the stated objectives.

## 
Definitions of Performance Measures, cont'd.

The indicators used to measure the performance of policies, programs and processes.www.fiu.edu/~pie/sec8appglossary.htm

Performance measures can be either outcome or output measures. Program performance should be monitored and assessed.ocsevalinitiative.com/resources/glossary.asp

Specific indicators used to evaluate how well a person, organization, or a system is operating. 
	www.crfonline.org/orc/glossary/p.html

Qualitative or quantitative indicators for assessing the achievement of outcomes.www.wtcc.sa.gov.au/webdata/resources/files/sp_glossary.doc

1. Explicit statements that define how success is to be measured, and identify the criteria, or benchmarks, to be used in judging success. 2. Discrete and predetermined information collected in order to assess performance against targets.www.treasury.tas.gov.au/domino/dtf/dtf.nsf/03b2a1d6613ba1894a2566bf001d6c3f/3fd01dff12836ef64a2566dd000cd01b

_**Definitions of Performance Measures, cont'd.**_
- The indicators used to measure the performance of policies, programs and processes.www.fiu.edu/~pie/sec8appglossary.htm
- Performance measures can be either outcome or output measures. Program performance should be monitored and assessed._[ocsevalinitiative.com/resources/glossary.asp](http://www.google.com/url?sa=X&start=6&oi=define&q=http://ocsevalinitiative.com/resources/glossary.asp&usg=AFQjCNHF0lDcpCGdL281AtnhOnEGwKUdCQ)_
- Specific indicators used to evaluate how well a person, organization, or a system is operating.
- 	www.crfonline.org/orc/glossary/p.html
- Qualitative or quantitative indicators for assessing the achievement of outcomes.www.wtcc.sa.gov.au/webdata/resources/files/sp_glossary.doc
- 1. Explicit statements that define how success is to be measured, and identify the criteria, or benchmarks, to be used in judging success. 2. Discrete and predetermined information collected in order to assess performance against targets._[www.treasury.tas.gov.au/domino/dtf/dtf.nsf/03b2a1d6613ba1894a2566bf001d6c3f/3fd01dff12836ef64a2566dd000cd01b](http://www.treasury.tas.gov.au/domino/dtf/dtf.nsf/03b2a1d6613ba1894a2566bf001d6c3f/3fd01dff12836ef64a2566dd000cd01b)_

## 
What To Measure & Why: The Big Picture

_**What To Measure & Why: The Big Picture**_

**Notes:**
What to Measure and Why
Performance measurement allows librarians to gather data and draw conclusions about the overall success or impact of library services. Tracking library services provides data for senior management (CEOs, decision-makers, managers, etc.) with a clearer picture of the daily functioning of the library, its impact on the agency and the value of providing library services (also the value of retaining highly trained information professionals to provide these services). 
Librarians should approach performance measurement as though they are reporting to people who rarely use the library and don’t understand that it is a vital unit within their agency, as this is frequently the case. To present a clear picture of the library as a busy unit supporting the agency’s mission statement and the professionals whose work they do understand, librarians have to keep good statistics of the work we do. That is the quantitative component of the performance measurement task. There is a qualitative component, too, and using customer feedback or “success stories” has a real impact on senior managers. Library services don’t lend themselves to scientific measurements, but we can make assumptions based on solid facts. We use what we know factually and from experience to extrapolate certain conclusions. 

## 
What to Measure 

Quantitative Data :  Keep Good Statistics

ILL
Reference
Literature Searches
Circulation
Cataloging
Outreach Initiatives 
Website activity

_**What to Measure **_
- Quantitative Data :  Keep Good Statistics
- ILL
- Reference
- Literature Searches
- Circulation
- Cataloging
- Outreach Initiatives
- Website activity

**Notes:**
Many ILSs will track materials checked out, new items added to the database and ILL/document delivery activity. Librarians can create reports that are presented clearly, often with charts and graphs from these ILS modules. Many libraries track reference, literature searches and ILL statistics in Excel, which is available almost universally in any Microsoft Office environment. The pooled fund study has created a statistical tracking tool to record transactions quickly and easily. (See Appendix H.) There is no need to purchase additional software to accomplish accurate and well-presented statistical reports. Excel also has a fairly versatile charting and graphing function. It is a good idea to pick out some data sets and represent them graphically for both impact and visual interest. 

Display Tracking Tools:
Hank: MTKN
Maggie: Pooled Fund
Email Hank / Maggie for copies of these documents.

## 
What to Measure 

Networks: 
Identify the inputs / outputs in common or establish
baseline metrics for measurement.

ILL
Reference
Literature Searches
Circulation
Cataloging
Outreach Initiatives 
Website activity

Sources:
The following tools were developed by MTKN & the Pooled Fund.

_**What to Measure **_
- Networks:
- Identify the inputs / outputs in common or establish
- baseline metrics for measurement.
- ILL
- Reference
- Literature Searches
- Circulation
- Cataloging
- Outreach Initiatives
- Website activity
- Sources:
- The following tools were developed by MTKN & the Pooled Fund.

**Notes:**
Networks need to agree upon which metrics they are going to focus on for an apples-to-apples comparison and to identify trends (benefits) resulting from working collaboratively.

## 
What to Measure

Qualitative Data: What do the statistics tell us?

Return On Investment (ROI) calculations
	Value of library services? How much $$$ was saved by having trained staff provide these services?

Success stories
	How did the library’s services help the organization fulfill its mission?

Customer feedback
Customer testimonials speak volumes.

_**What to Measure**_
- Qualitative Data: What do the statistics tell us?
**Return On Investment (ROI) calculations**
- 	Value of library services? How much $$$ was saved by having trained staff provide these services?
**Success stories**
- 	How did the library’s services help the organization fulfill its mission?
**Customer feedback**
- Customer testimonials speak volumes.

## What to Measure

Qualitative Data: Return On Investment (ROI)

What is the value of library services?

How much $$$ was saved by having trained staff provide these services?

Sources:
ROI for Libraries Remains High, Roger Strouse,  http://www.outsellinc.com/store/insights/3538.
Mn/DOT Library Accomplishments,  Jerry Baldwin,
http://www.dot.state.mn.us/library/mndot_library_benefits.html.

- Qualitative Data: Return On Investment (ROI)
- What is the value of library services?
- How much $$$ was saved by having trained staff provide these services?
- Sources:
- ROI for Libraries Remains High, Roger Strouse,  _[http://www.outsellinc.com/store/insights/3538](http://www.outsellinc.com/store/insights/3538)__._
- Mn/DOT Library Accomplishments,  Jerry Baldwin,
- _[http://www.dot.state.mn.us/library/mndot_library_benefits.html](http://www.dot.state.mn.us/library/mndot_library_benefits.html)__._

**Notes:**
Qualitative data is also important when demonstrating the ROI of library services to senior managers. Several studies claim that libraries return at least four to six times the investment 
Outsell 2007 Research
Important Details: Libraries looking to justify their existence with quantitative return-on-investment (ROI) numbers need look no further than their own end-users’ feedback. In Outsell research from Spring of 2007, users reported substantial, quantifiable benefits from involving the library in their information gathering. Library ROI was highest for the corporate and government sectors, but also positive for healthcare and education.
Benefits of library use included:

Time Saved: Government sector users topped this metric, reporting that each interaction with the library saved them 12.2 hours on average. 

Dollars Saved:  Government users report $2,575 in savings per use. 
. 
Decision Support: Two-thirds (65%) of users across sectors used library-provided information to support decisions, actions, and strategies. In this regard, government end-users were the most reliant on library information (72%)

Implications: Given the time that end-users spend on information tasks (now around 11.6 hours per week) and the increasing costs of content, libraries can use the data above to make a powerful argument for their own value. In light of Outsell’s newest ROI data, it seems penny-wise and pound-foolish for enterprises to eliminate library budgets in a time of increased information challenges. Outsell recommends that libraries, publishers, and information providers alike collect and publish ROI data gathered from their own users as one method of illustrating their contribution to the bottom line.

Display Jerry Baldwin’s ROI, Toolkit version
Jerry Baldwin (Mn/DOT) has developed an ROI formula that many people have adapted for their own use that demonstrates reduced costs and added-value ratios. 

## What to Measure

Qualitative Data: Success Stories

“An internal customer at WisDOT was about to solicit a Request for Proposal
on a $50,000 project to determine the demerit point/administrative license
withdrawal system used by other states. The library not only had the NHTSA
related study on the shelf but was able to deliver it to the general counsel’s 
Office within 15 minutes of receiving the request.”

Source:
Example provided by John Cherney, Head Librarian, Wisconsin DOT.

- Qualitative Data: Success Stories
- “*An internal customer at WisDOT was about to solicit a Request for Proposal*
*on a $50,000 project to determine the demerit point/administrative license*
*withdrawal system used by other states. The library not only had the NHTSA*
*related study on the shelf but was able to deliver it to the general counsel’s *
*Office within 15 minutes of receiving the request.”*
- Source:
- Example provided by John Cherney, Head Librarian, Wisconsin DOT.

**Notes:**
The pooled fund study has been collecting Success Stories as Qualitative Data. Our libraries have submitted a number of high-impact accounts of how the librarians help customers avoid duplication of research, provide quick turn-around on requests for vital information, save customers’ time, enhance productivity and save their departments’ $$$. 

## What to Measure

Qualitative Data: Customer Feedback

“The KDOT Library has been an indispensable resource that I have used
often over the past few years. Without it, I would have spent hours looking 
or articles, visiting local libraries, making phone calls and waiting on faxes
or mail. This has meant not only better informed decisions by staff engineers,
but an actual savings to the taxpayers of the state of Kansas.” 

Source:
Excerpt from letter of thanks from a bridge hydraulics engineer at Kansas DOT. Example provided by Marie
Manthe, Librarian, Kansas DOT.

- Qualitative Data: Customer Feedback
*“**The KDOT Library has been an indispensable resource that I have used*
*often over the past few years. Without it, I would have spent hours looking *
*or articles, visiting local libraries, making phone calls and waiting on faxes*
*or mail. This has meant not only better informed decisions by staff engineers,*
*but an actual savings to the taxpayers of the state of Kansas.”*
- Source:
- Excerpt from letter of thanks from a bridge hydraulics engineer at Kansas DOT. Example provided by Marie
- Manthe, Librarian, Kansas DOT.

**Notes:**
The pooled fund librarians feel that customer feedback speaks volumes to management.

## Why?

Reporting: Communicating Value To Management

Value of the library to the organization
Value of investing time and resources in a consortium or network

Statistics

Expenditures

ROI

Success Stories

Customer Feedback

Source:
Transportation Librarians Toolkit (forthcoming)

- Reporting: Communicating Value To Management
- Value of the library to the organization
- Value of investing time and resources in a consortium or network
- Statistics
- Expenditures
- ROI
- Success Stories
- Customer Feedback
- Source:
- Transportation Librarians Toolkit (forthcoming)

**Notes:**
Statistics, expenditures, ROI and success stories should all be part of the reports to management. Reporting informs the decision-makers of the role of the library within the organization and allows library staff members to step back and review their progress. A good rule of thumb for reports is to include the mission statement up front, to inform the reader of the library’s purpose and follow with statistics and narrative illustrating how the mission was fulfilled and what factors may have interfered with the library’s objectives. The report should also include a summary of customer populations (percentage of civil engineering staff, planning, research), which parts of the collection get the most use, most frequently used services and the results of a customer survey.

Show WSDOT Library Use Charts

## Why?

Strategic Planning: The Library in Context 

Performance Measurement

Demonstrate Value 

Align the Library’s Vision with the Department’s Mission

Source:
Transportation Librarians Toolkit (forthcoming).

- Strategic Planning: The Library in Context
- Performance Measurement
- Demonstrate Value
- Align the Library’s Vision with the Department’s Mission
- Source:
- Transportation Librarians Toolkit (forthcoming).

**Notes:**
Strategic planning for libraries
Performance measures and reporting provide context for effective library strategic planning. It also provides a framework for ensuring that the library's vision and strategies are aligned to support the mission of the departments they serve. 

## 

Hank Zaletel
MTKN/Iowa DOT Library/CTRE
Hank.Zaletel@dot.iowa.gov  

Maggie Sacco
Transportation Library Connectivity Pooled Fund Study
msacco@ctcandassociates.com 

www.mtkn.org     www.libraryconnectivity.org 

- Hank Zaletel
- MTKN/Iowa DOT Library/CTRE
- _[Hank.Zaletel@dot.iowa.gov](mailto:Hank.Zaletel@dot.iowa.gov)__  _
- Maggie Sacco
- Transportation Library Connectivity Pooled Fund Study
- _[msacco@ctcandassociates.com](mailto:msacco@ctcandassociates.com)_
- _[www.mtkn.org](http://www.mtkn.org/)__     __[www.libraryconnectivity.org](http://www.libraryconnectivity.org/)__ _

- THANK YOU!