# Presentation: 662812

## Results from the Performance Management Survey

**Results from the Performance Management Survey**
- Prepared by Janice Rouiller and Laura Stouffer, SAIC
- and
- Joe Wolski, OQM
- 9 May 2005

## Table of Contents

- Survey Background...............................................................................3
- Respondent Information.........................................................................7
- Perceptions of Cycle 3 PM Implementation...............................................15
- Perceptions of Needs for Cycle 4............................................................52
- Organizational Climate Perceptions.........................................................68
- Organizational Climate Perceptions by Program Area.................................78
- Organizational Climate Perceptions by Role.............................................84
- Customer Scorecard Ratings.................................................................88
- Summary of Comments.......................................................................101
- Summary..........................................................................................105
- Appendix - Individual Comments from Survey..........................................111

## Survey Background

## Purpose

- Administer a customized version of the ORS Customer Scorecard to:
- Gather feedback on past Performance Management (PM) implementation efforts
- Collect needs assessment information to tailor the on-going implementation of PM
- Gather data on the required Customer Scorecard dimensions
- Collect organizational culture change information for the Evaluation Study

## Methodology

- Administer the survey at the completion of the third cycle of Performance Management implementation on an organization-wide basis
- Which occurred in February 2005
- Designed and tested web survey during February 2005
- Survey went “live” on 3 February, 2005
- Email requests sent to all PM Team Leaders, Members, and additional senior ORS/ORF managers
- First reminder email sent  17 February 2005
- Closed survey field period  24 February 2005
- Data file sent to contractor for analysis

## Survey Distribution 

- Number of email recipients				186
- Number of  respondents			             41
- Response Rate					  22%

## Respondent Information

## Respondents by Role

**N = 45**

- Note:  Multiple responses allowed.

**Number of Respondents**

## Respondents by Organization

**N = 41**

**Number of Respondents**

## Respondents by Program Area

**N =41**

**Number of Respondents**

## Experience with the PM Process

**Percent “Yes”**

**N = 41**

**N = 40**

**N = 40**

**N = 36**

**N = 35**

## Approximately what percent of time did you spend on PM-related activities during FY04?

**N = 41**

**Percent of time Spent**

**Number of Respondents**

## On average, how often did your PM team meet during FY04?  

**N = 41**

**How Often PM Team Met**

**Number of Respondents**

## To what extent did your team members work between meetings to implement PM?  

**To what extent did your team members work between meetings to implement PM?  **

- To No Extent

- To a Great Extent

**N = 41**
**Mean = 6.71**
**Median = 6.5**

## Perceptions of Cycle 3 PM Implementation

**(January 2004 - January 2005)**

**How helpful did you find the following in implementing PM?  **

- Not at all Helpful

- Extremely Helpful

**Mean Ratings**

## How helpful did you find the following tools in implementing PM?

**N = 41**

- Not at all Helpful

- Extremely Helpful

**Mean Ratings**

**N = 41**

**N = 41**

**N = 41**

**N = 38**

**N = 40**

**N = 41**

## How helpful did you find the following tools in implementing PM? 

**How helpful did you find the following tools in implementing PM? **
_**Listings of the Services Hierarchy**_

- Not at all Helpful

- Extremely Helpful

**N = 41**
**Mean = 5.46**
**Median = 6**

## How helpful did you find the following tools in implementing PM? 

**How helpful did you find the following tools in implementing PM? **
_**Performance Management Plan (PMP) Template**_

- Not at all Helpful

- Extremely Helpful

**N = 41**
**Mean = 7.15**
**Median = 8**

## How helpful did you find the following tools in implementing PM? 

**How helpful did you find the following tools in implementing PM? **
_**Strategy Mapping**_

- Not at all Helpful

- Extremely Helpful

**N =40**
**Mean = 6.29**
**Median = 7**

## How helpful did you find the following tools in implementing PM? 

**How helpful did you find the following tools in implementing PM? **
_**Measures Roadmap Template**_

- Not at all Helpful

- Extremely Helpful

**N = 41**
**Mean = 6.18**
**Median = 7**

## How helpful did you find the following tools in implementing PM? 

**How helpful did you find the following tools in implementing PM? **
_**Initiative Roadmap Template**_

- Not at all Helpful

- Extremely Helpful

**N = 41**
**Mean = 6.25**
**Median = 7**

## How helpful did you find the following tools in implementing PM? 

**How helpful did you find the following tools in implementing PM? **
_**Measures Reporting Form**_

- Not at all Helpful

- Extremely Helpful

**N = 41**
**Mean = 5.88**
**Median = 7**

## How helpful did you find the following tools in implementing PM? 

**How helpful did you find the following tools in implementing PM? **
_**PM Presentation Template for Conference**_

- Not at all Helpful

- Extremely Helpful

**N = 38**
**Mean = 6.52**
**Median = 8**

## How helpful did you find the following data analysis services in implementing PM?

- Not at all Helpful

- Extremely Helpful

**Mean Ratings**

**N = 41**

**N = 41**

**N = 40**

**N = 41**

**N = 41**

## How helpful did you find the following data analysis services in implementing PM:

**How helpful did you find the following data analysis services in implementing PM:**
_**Design of Data Collection Forms**_

- Not at all Helpful

- Extremely Helpful

**N = 41**
**Mean = 5.38**
**Median = 5.5**

## How helpful did you find the following data analysis services in implementing PM:

**How helpful did you find the following data analysis services in implementing PM:**
** **_**Establishing Data Collection Methodologies**_

- Not at all Helpful

- Extremely Helpful

**N = 41**
**Mean = 6.00**
**Median = 7**

## How helpful did you find the following data analysis services in implementing PM:

**How helpful did you find the following data analysis services in implementing PM:**
_**Graphing Data**_

- Not at all Helpful

- Extremely Helpful

**N = 40**
**Mean = 6.06**
**Median = 7**

## How helpful did you find the following data analysis services in implementing PM:

**How helpful did you find the following data analysis services in implementing PM:**
_**Process Behavior Chart Analysis**_

- Not at all Helpful

- Extremely Helpful

**N = 41**
**Mean = 5.86**
**Median = 7**

## How helpful did you find the following data analysis services in implementing PM:

**How helpful did you find the following data analysis services in implementing PM:**
_**ORS Customer Scorecards**_

- Not at all Helpful

- Extremely Helpful

**N = 41**
**Mean = 6.45**
**Median = 7**

## How helpful did you find the following communication vehicles in implementing PM?

**N = 40**

- Not at all Helpful

- Extremely Helpful

**Mean Ratings**

**N = 40**

**N = 40**

## How helpful did you find the following communication vehicles in implementing PM:

**How helpful did you find the following communication vehicles in implementing PM:**
_**Performance Management Website**_

- Not at all Helpful

- Extremely Helpful

**N = 40**
**Mean = 4.78**
**Median = 4**

## How helpful did you find the following communication vehicles in implementing PM:

**How helpful did you find the following communication vehicles in implementing PM:**
_**PM Related Emails from OQM**_

- Not at all Helpful

- Extremely Helpful

**N = 40**
**Mean = 5.74**
**Median = 6**

## How helpful did you find the following communication vehicles in implementing PM:

**How helpful did you find the following communication vehicles in implementing PM:**
_**PM Conference**_

- Not at all Helpful

- Extremely Helpful

**N = 40**
**Mean = 5.44**
**Median = 6**

## How helpful did you find assistance from the following groups in implementing PM?

**N = 39**

- Not at all Helpful

- Extremely Helpful

**Mean Ratings**

**N = 40**

**N = 38**

**N = 38**

## How helpful did you find assistance from the following groups in implementing PM? 

**How helpful did you find assistance from the following groups in implementing PM? **
_**OQM Staff**_

- Not at all Helpful

- Extremely Helpful

**N = 39**
**Mean = 6.38**
**Median = 8**

## How helpful did you find assistance from the following groups in implementing PM? 

**How helpful did you find assistance from the following groups in implementing PM? **
_**PM Consultants**_

- Not at all Helpful

- Extremely Helpful

**N = 40**
**Mean = 7.97**
**Median = 9**

## How helpful did you find assistance from the following groups in implementing PM? 

**How helpful did you find assistance from the following groups in implementing PM? **
_**Other PM Teams**_

- Not at all Helpful

- Extremely Helpful

**N = 38**
**Mean = 4.96**
**Median = 6**

## How helpful did you find assistance from the following groups in implementing PM? 

**How helpful did you find assistance from the following groups in implementing PM? **
_**Your Supervisor**_

- Not at all Helpful

- Extremely Helpful

**N = 38**
**Mean = 7.82**
**Median = 8**

## Immediate Supervisor Involvement in PM   

- To no Extent

- To a Great Extent

- Not at all Helpful

- Extremely Helpful

**Mean Ratings**

**N = 38**

**N = 39**

## What extent was your immediate supervisor involved in your PM process during this PM cycle?

**What extent was your immediate supervisor involved in your PM process during this PM cycle?**

- To No Extent

- To a Great Extent

**N = 38**
**Mean = 7.53**
**Median = 8**

## How helpful was the involvement of your immediate supervisor in PM related matters during this cycle?

**How helpful was the involvement of your immediate supervisor in PM related matters during this cycle?**

- Not at all Helpful

- Extremely Helpful

**N = 39**
**Mean = 7.60**
**Median = 8**

## To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle?

**N = 38**

- To No Extent

- To a Great Extent

**Mean Ratings**

**N = 40**

**N = 40**

**N = 40**

**N = 39**

**N = 38**

**N = 39**

**N = 38**

## To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle? 

**To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle? **
_**Balanced Scorecard Approach to PM**_

- To No Extent

- To a Great Extent

**N = 38**
**Mean = 6.03**
**Median = 7**

## To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle? 

**To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle? **
_**Methods to Gather Data**_

- To No Extent

- To a Great Extent

**N = 40**
**Mean = 6.18**
**Median = 7**

## To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle? 

**To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle? **
_**Methods to Analyze Data**_

- To No Extent

- To a Great Extent

**N = 40**
**Mean = 5.68**
**Median = 7**

## To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle? 

**To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle? **
_**Presentation Preparation and Delivery**_

- To No Extent

- To a Great Extent

**N = 40**
**Mean = 5.54**
**Median = 7**

## To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle? 

**To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle? **
_**Working in a Team**_

- To No Extent

- To a Great Extent

**N = 39**
**Mean = 5.54**
**Median = 6**

## To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle? 

**To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle? **
_**Using Excel**_

- To No Extent

- To a Great Extent

**N = 38**
**Mean = 5.03**
**Median = 6**

## To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle? 

**To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle? **
_**Using PowerPoint**_

- To No Extent

- To a Great Extent

**N = 39**
**Mean = 4.79**
**Median = 5**

## To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle? 

**To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle? **
_**Using Visio**_

- To No Extent

- To a Great Extent

**N = 38**
**Mean = 4.19**
**Median = 5**

## Perceptions of Needs for Cycle 4

**In planning for the next PM cycle, how helpful would each of the following be to your PM Team?**

- Not at all Helpful

- Extremely Helpful

**Mean Ratings**

## In planning for the next PM cycle, how helpful would each of the following tools/resources be to your PM Team?

**N = 40**

- Not at all Helpful

- Extremely Helpful

**Mean Ratings**

**N = 40**

**N = 37**

**N = 39**

**N = 38**

**N = 37**

## In planning for the next PM cycle, how helpful would each of the following tools/resources be to your PM Team: 

**In planning for the next PM cycle, how helpful would each of the following tools/resources be to your PM Team: **
_**PM Consultants Provided by OQM**_

- Not at all Helpful

- Extremely Helpful

**N = 40**
**Mean = 7.84**
**Median = 9**

## In planning for the next PM cycle, how helpful would each of the following tools/resources be to your PM Team: 

**In planning for the next PM cycle, how helpful would each of the following tools/resources be to your PM Team: **
_**OQM Staff**_

- Not at all Helpful

- Extremely Helpful

**N = 40**
**Mean = 6.57**
**Median = 7**

## In planning for the next PM cycle, how helpful would each of the following tools/resources be to your PM Team: 

**In planning for the next PM cycle, how helpful would each of the following tools/resources be to your PM Team: **
_**Regular Meetings with your Management**_

- Not at all Helpful

- Extremely Helpful

**N = 37**
**Mean = 7.03**
**Median = 8**

## In planning for the next PM cycle, how helpful would each of the following tools/resources be to your PM Team: 

**In planning for the next PM cycle, how helpful would each of the following tools/resources be to your PM Team: **
_**Rotating Quarterly Presentation Meetings**_

- Not at all Helpful

- Extremely Helpful

**N = 39**
**Mean = 5.94**
**Median = 7**

## In planning for the next PM cycle, how helpful would each of the following tools/resources be to your PM Team: 

**In planning for the next PM cycle, how helpful would each of the following tools/resources be to your PM Team: **
_**Yearly Performance Management Conference**_

- Not at all Helpful

- Extremely Helpful

**N = 38**
**Mean = 5.59**
**Median = 7**

## In planning for the next PM cycle, how helpful would each of the following tools/resources be to your PM Team: 

**In planning for the next PM cycle, how helpful would each of the following tools/resources be to your PM Team: **
_**IT Support to Establish Data Collection Systems**_

- Not at all Helpful

- Extremely Helpful

**N = 37**
**Mean = 7.47**
**Median = 8**

## In planning for the next PM cycle, how helpful would each of the following training sessions be to your PM Team?

**N = 38**

- Not at all Helpful

- Extremely Helpful

**Mean Ratings**

**N = 39**

**N = 40**

**N = 39**

**N = 40**

**N = 40**

## In planning for the next PM cycle, how helpful would each of the following training sessions be to your PM Team? 

**In planning for the next PM cycle, how helpful would each of the following training sessions be to your PM Team? **
_**Balanced Scorecard Methodology**_

- Not at all Helpful

- Extremely Helpful

**N = 38**
**Mean = 5.97**
**Median = 6**

## In planning for the next PM cycle, how helpful would each of the following training sessions be to your PM Team? 

**In planning for the next PM cycle, how helpful would each of the following training sessions be to your PM Team? **
_**Setting up Data Collection Systems and Forms**_

- Not at all Helpful

- Extremely Helpful

**N = 39**
**Mean = 6.45**
**Median = 7**

## In planning for the next PM cycle, how helpful would each of the following training sessions be to your PM Team? 

**In planning for the next PM cycle, how helpful would each of the following training sessions be to your PM Team? **
_**Analyzing Data Through Graphs**_

- Not at all Helpful

- Extremely Helpful

**N = 40**
**Mean = 6.58**
**Median = 8**

## In planning for the next PM cycle, how helpful would each of the following training sessions be to your PM Team? 

**In planning for the next PM cycle, how helpful would each of the following training sessions be to your PM Team? **
_**Analyzing Processes with Process Behavior Charts**_

- Not at all Helpful

- Extremely Helpful

**N = 39**
**Mean = 6.54**
**Median = 8**

## In planning for the next PM cycle, how helpful would each of the following training sessions be to your PM Team? 

**In planning for the next PM cycle, how helpful would each of the following training sessions be to your PM Team? **
_**Customer Assessment through Interviews, Focus Groups and Surveys**_

- Not at all Helpful

- Extremely Helpful

**N = 40**
**Mean = 6.86**
**Median = 8**

## In planning for the next PM cycle, how helpful would each of the following training sessions be to your PM Team? 

**In planning for the next PM cycle, how helpful would each of the following training sessions be to your PM Team? **
_**ORS ABC Cost Accounting Model**_

- Not at all Helpful

- Extremely Helpful

**N = 40**
**Mean = 6.56**
**Median = 8**

## Organizational Climate Perceptions

## Organizational Climate PerceptionsSummary

**Organizational Climate Perceptions****Summary**

- Strongly Disagree

- Strongly Agree

**N = 40**

**N = 39**

**N = 40**

**N = 40**

**N = 39**

**N = 40**

**N = 40**

**N = 40**

- Neither Agree nor Disagree

## I see the value of performance management for improving business operations in my area.

**I see the value of performance management for improving business operations in my area.**

- Strongly Disagree

- Strongly Agree

**N = 40**
**Mean = 3.59**
**Median = 4**

- Neither Agree nor Disagree

## My organization (ORS or ORF) is committed to the performance management effort.  

**My organization (ORS or ORF) is committed to the performance management effort.  **

- Strongly Disagree

- Strongly Agree

**N = 39**
**Mean = 4.03**
**Median = 4**

- Neither Agree nor Disagree

## I understand what the Services Hierarchy is and its purpose.  

**I understand what the Services Hierarchy is and its purpose.  **

- Strongly Disagree

- Strongly Agree

**N = 40**
**Mean = 3.50**
**Median = 4**

- Neither Agree nor Disagree

## The culture of my organization (ORS or ORF) is changing to be more results-oriented.  

**The culture of my organization (ORS or ORF) is changing to be more results-oriented.  **

- Strongly Disagree

- Strongly Agree

**N = 40**
**Mean = 3.70**
**Median = 4**

- Neither Agree nor Disagree

## Managers in my organization (ORS or ORF) believe accountability is an important organizational value. 

**Managers in my organization (ORS or ORF) believe accountability is an important organizational value. **

- Strongly Disagree

- Strongly Agree

**N = 39**
**Mean = 3.84**
**Median = 4**

- Neither Agree nor Disagree

## Performance management has contributed to improvements in my area.  

**Performance management has contributed to improvements in my area.  **

- Strongly Disagree

- Strongly Agree

**N = 40**
**Mean = 3.21**
**Median = 4**

- Neither Agree nor Disagree

## Performance management will assist my organization if we have to go through an A-76 competition. 

**Performance management will assist my organization if we have to go through an A-76 competition. **

- Strongly Disagree

- Strongly Agree

**N = 40**
**Mean = 3.56**
**Median = 4**

- Neither Agree nor Disagree

## My PM Team (or the groups I am responsible for) are actively involved in data collection and analysis.

**My PM Team (or the groups I am responsible for) are actively involved in data collection and analysis.**

- Strongly Disagree

- Strongly Agree

**N = 40**
**Mean = 4.15**
**Median = 5**

- Neither Agree nor Disagree

## Organizational Climate Perceptions by Program Area

## Program and Employee Services Climate PerceptionsPositive Responses

**Percent Agree or Strongly Agree**

**N = 7**

**N = 7**

**N = 7**

**N = 7**

**N = 7**

**N = 7**

**N = 7**

**N = 7**

## Scientific Resources Climate PerceptionsPositive Responses

**Percent Agree or Strongly Agree**

**N = 6**

**N = 6**

**N = 6**

**N = 6**

**N = 6**

**N = 6**

**N = 6**

**N = 6**

## Security and Emergency Response Services Climate PerceptionsPositive Responses

**Percent Agree or Strongly Agree**

**N = 8**

**N = 8**

**N = 8**

**N = 8**

**N = 8**

**N = 8**

**N = 8**

**N = 8**

## Management Services Climate PerceptionsPositive Responses

**Percent Agree or Strongly Agree**

**N = 3**

**N = 3**

**N = 3**

**N = 3**

**N = 3**

**N = 3**

**N = 3**

**N = 3**

## Real Estate and Facilities Climate PerceptionsPositive Responses

**Percent Agree or Strongly Agree**

**N = 4**

**N = 4**

**N = 4**

**N = 4**

**N = 4**

**N = 4**

**N = 4**

**N = 4**

## Organizational Climate Perceptions by Role

**Division Directors’/Office Directors’ Climate Perceptions**

**Percent Agree or Strongly Agree**

**N = 5**

**N = 5**

**N = 5**

**N = 5**

**N = 5**

**N = 5**

**N =5**

**N = 5**

## PM Team Leaders’ Climate Perceptions

**Percent Agree or Strongly Agree**

**N = 7**

**N = 7**

**N = 7**

**N = 7**

**N = 7**

**N = 7**

**N = 7**

**N = 7**

## PM Team Members’ Perceptions

**Percent Agree or Strongly Agree**

**N = 27**

**N = 27**

**N = 27**

**N = 27**

**N = 26**

**N = 27**

**N = 27**

**N = 27**

## Customer Scorecard Ratings

## Customer Scorecard Ratings for PMOverview for Cycle 3

**N = 40**

- Unsatisfactory

- Outstanding

**Mean Ratings**

**N = 40**

**N = 39**

**N = 38**

**N = 40**

**N = 40**

**N = 39**

**N = 40**

**N = 40**

## Cost Satisfaction Ratings

**Cost Satisfaction Ratings**

- Unsatisfactory

- Outstanding

**N = 40**
**Mean = 5.92**
**Median = 7**

## Quality Satisfaction Ratings

**Quality Satisfaction Ratings**

- Unsatisfactory

- Outstanding

**N = 40**
**Mean = 6.77**
**Median = 8**

## Timeliness Satisfaction Ratings

**Timeliness Satisfaction Ratings**

- Unsatisfactory

- Outstanding

**N = 39**
**Mean = 6.67**
**Median = 7**

## Reliability Satisfaction Ratings

**Reliability Satisfaction Ratings**

- Unsatisfactory

- Outstanding

**N = 38**
**Mean = 6.82**
**Median = 8**

## Availability Satisfaction Ratings

**Availability Satisfaction Ratings**

- Unsatisfactory

- Outstanding

**N = 40**
**Mean = 7.16**
**Median = 8**

## Responsiveness Satisfaction Ratings

**Responsiveness Satisfaction Ratings**

- Unsatisfactory

- Outstanding

**N = 40**
**Mean = 7.16**
**Median = 8**

## Convenience Satisfaction Ratings

**Convenience Satisfaction Ratings**

- Unsatisfactory

- Outstanding

**N = 39**
**Mean = 7.23**
**Median = 8**

## Competence Satisfaction Ratings

**Competence Satisfaction Ratings**

- Unsatisfactory

- Outstanding

**N = 40**
**Mean = 7.71**
**Median = 8**

## Handling of Problems Satisfaction Ratings

**Handling of Problems Satisfaction Ratings**

- Unsatisfactory

- Outstanding

**N = 40**
**Mean = 7.18**
**Median = 8**

## Customer Scorecard Ratings for PMTrend Analysis

- Unsatisfactory

- Outstanding

**Mean Ratings**

**N = 37**

**N = 70**

**N = 67**

**N = 67**

**N = 40**

**N = 40**

**N = 39**

**N = 38**

**N = 68**

**N = 68**

**N = 67**

**N = 68**

## Customer Scorecard Ratings for PMTrend Analysis (cont.)

- Unsatisfactory

- Outstanding

**Mean Ratings**

**N = 68**

**N = 66**

**N = 69**

**N = 65**

**N = 40**

**N = 40**

**N = 39**

**N = 40**

**N = 40**

**N = 68**

**N = 69**

**N = 68**

**N = 68**

**N = 68**

**N = 65**

## Summary of Comments

## What did you value most about this cycle of performance management?

- Knowledge and Experience Gained (4)
- Results show that we have improved since last year.
- Surveys have provided an opportunity to find out how customers perceive quality of service.
- Data gathering has helped quantify quality of service.
- Assistance provided by consultants and OQM staff  (3)
- Interaction with OQM staff & consultants in analyzing performance.
- Knowledgeable and helpful consultants.
- Measurement Tools (3)
- The use of data reporting template and the initiatives template.
- The Measures Reporting Form allowed us to look at the objectives and measures in a more relevant perspective.

- Note: Comments were sorted into categories. Most frequently mentioned categories listed first.

## What obstacles were most challenging to you during this cycle of performance management?

- Note: Comments were sorted into categories. Most frequently mentioned categories listed first.

- Lack of Time and/or Resources (6)
- Lack of time to devote to PMP and keep up with workload
- Short notice
- OQM website is out of date
- Miscellaneous (4)
- Learning to use Visio and Excel
- Preparing the report
- Lack of Management Involvement (3)
- Not having discussions about PM in staff meetings
- Managers need to use data in decision making
- Data Collection and Measurement (2)
- Obtaining data
- Lack of commitment from team members (2)
- Getting all team members to contribute

## How can we help integrate PM into your daily business activities?

- Increase support currently provided  (6)
- Need help in learning how to distribute data collection throughout the division.
- Need help collecting and interpreting data.
- Miscellaneous (3)
- Work to establish timelines so that we see the year at a glance.
- Continue emphasis on the importance of performance management.

## Summary

## SummarySurvey Respondents

- Response rate of 22%
- Respondents were primarily from ORS (73%)
- The majority of respondents were team members (68%)
- About 75% of respondents were experienced with PM (i.e., participated in FY03)
- About 60% of respondents attended FY04 training
- About 63% of respondents spent 20% or less of their time during the past year on PM
- Team meetings typically occurred once a month (41%), biweekly (22%), and quarterly (20%)

## Summary (cont.)Feedback on Past Cycle

- Most Helpful Tools/Resources
- PM consultants
- Service group supervisors
- PM template
- PM presentation template
- ORS Customer Scorecards
- OQM Staff.
- These are the same tools/resources receiving the highest ratings in FY03.
- KSA Improvements
- Just above mid-scale improvements noted on all topics with the following exceptions:  PowerPoint and Visio.
- Perhaps respondent familiarity with these tools has leveled off.

## Summary (cont.)Planning for Next PM Cycle

- Future needs
- Most helpful to teams for the future would be
- PM consultants
- IT support for data collection systems
- Regular meetings with management to discuss PM data
- Training on customer assessment
- Training on analyzing data
- These requests are similar to previous years

## Summary (cont.)Organizational Climate

- Majority of respondents either agree or strongly agree that:
- PM Teams are actively involved in data collection (85%)
- Their organization is committed to PM (77%)
- Managers in their organization believe accountability is important (64%)
- They see the value of PM for improving business operations (60%)
- The culture of their organization is changing to be more results-oriented (58%)

## Summary (cont.)Customer Scorecard Trend Analysis

- Ratings on dimensions show minor variance from cycle to cycle but none of the differences are statistically significant.
- Current ratings:
- Highest on competence, convenience, and handling of problems,
- Lowest on cost

## Appendix

**Individual Comments From Survey**

## What did you value most about this cycle of performance management? Individual Comments

- The opportunity to find out how my customers perceive the quality of the job I'm doing.
- Data gathering enabled us to quantify that construction inspection was decreasing our submittal review response time.
- Our consultant Janice Rouiller.
- The survey provided verification of the quality of service DMCS is providing the NIH community.
- The numbers show that last year's improvements have not only been sustained, but improved upon.
- Honestly, I must state NOTHING! This ploy that the PMP process is management tool is hog-wash, the true intent of the PMP is to tie it to how much time staff spend in each function and to cut staff.
- Interaction with OQM staff & consultants in analyzing performance.

## What did you value most about this cycle of performance management?  (cont.) Individual Comments

- Patrick  Patterson, is the consultant with whom our team works on PMP.  His skills & knowledge of all aspect of the program are excellent.  He ensures that we ‘see’ the process & purpose rather than ‘just do it’.
- The willingness of the ORS Scientific resources to be first in the rotation of PM conferences.
- The use of data reporting template and the initiatives template.
- Meeting as a group to plan ahead.
- the introduction of the use of the Business Framework and the concept of the Balanced Scorecard.
- It was helpful to focus just on the high impact objectives.
- The Measures Reporting Form allowed us to look at the objectives and measures in a more relevant perspective.

## What obstacles were most challenging to you during this cycle of performance management?Individual Comments

- Preparing the report is painful.
- Being taken away from my primary duties created undue stress and hardship when OQM wanted their deadlines met, yet they couldn't seem to get their parts done in a timely fashion.
- We did not have the time to do the customer survey focus groups we had planned on doing.
- OQM website is out of date and needs updating.
- The challenge of continuing performance gains without losing any quality.
- learning to use Excel and Visio.
- Lack of time to devote to PMP AND perform ones own work.   PMP is almost a full time job.   This is an OQM function and should be performed by that staff.
- Supervisors and managers should use data more often in their decision-making.

## What obstacles were most challenging to you during this cycle of performance management?  (cont.) Individual Comments

- Commitment from all the team members.
- obtaining data.
- Getting all team members to contribute to the tasks of identifying relevant data and analyzing it.
- The way the PMP was created in Excel with an entire page in one row of cells made it extremely time consuming & tedious to update.
- Switching the rules in the middle of the game!  We went from PMP, to score card, then to the dash board.  These aberrations made staff wonder what value all of this has to our the ultimate duty to provide the service to the Institutes.
- Not having discussions about the performance of service groups/discrete services in staff meetings.
- Lack of a program area manager who accepts PM plans and manage their implementation.

## What obstacles were most challenging to you during this cycle of performance management?  (cont.) Individual Comments

- Concentration on the Measures Reporting Form throughout the year caused the PMP to be neglected.  At the end of the cycle, it took a lot of time to update the PMP to match the Measures Reporting Form.
- The short notice and scheduling a report due for the week of January 17 was inane!

## How can we help integrate PM into your daily business activities? Individual Comments

- We have always been collecting and analyzing data to improve our program--no value added from this formal process.
- In my opinion performance management might be applicable for some businesses in private industry but not for the US Gov,t.
- It already is, and was before I was tasked with being a part of this initiative.
- Work to establish timelines so that we see the year at a glance.  Several times my consultant worked with me to make deadlines at the last minute.
- Implement the ORF design review and inspection processes
- Already done.
- Have the ORS Office of Quality Management (OQM) staff do the work.  In many agencies and in private sector companies this function is not put on the backs of the line staff but is performed by either consultants or an office such as OQM.

## How can we help integrate PM into your daily business activities?  (cont.) Individual Comments

- Need help in learning how to distribute data collection throughout the division.
- Follow up with Quarterly team meetings.
- Continue emphasis on the importance of performance management to efficient 7 effective business operations.
- Need help collecting observation type data.
- Need help interpreting data.