# Presentation: 682736

## A JOURNEY FROM DATA COLLECTION TO ACTION

## Ken Kizer’s Revolutionary Transformation of VHA

**Decentralized organizational structure**
**22 Networks as basic business units & seedbeds for experimentation**
**Transition from inpatient to outpatient care**
**National protocols for patient care based on evidence based medicine**
**Multiple measures of the quality of care**
**Measure, measure, measure everything that is important!**

## 1997 One VA Survey

**161 questions**
**50% response rate**
**Facilities received printed descriptive reports prepared by an outside contractor**
**6 months after survey closed!**
**No access to data sets**
**Reports reproduced poorly **
**Facilities designed & implemented interventions**
**Few targeted interventions as opposed to many interventions most effective**
**Didn’t matter what you did!**

## 7,600 employees who voluntarily left VHA during 8 months prior to the survey 
38 diagnostic & 5 demographic questions
some from 1997 One VA Survey
others proven to be important to employee recruitment & retention
IVR & web at Sirota Consulting
Written invitation, survey mailed, & post card reminder 
23% response rate
Sirota suggested adding
Are you interested in returning to VHA?
Recruitment cost avoidance paid for survey!

**38 diagnostic & 5 demographic questions**
**some from 1997 One VA Survey**
**others proven to be important to employee recruitment & retention**
**IVR & web at Sirota Consulting**
**Written invitation, survey mailed, & post card reminder **
**23% response rate**
**Sirota suggested adding**
**Are you interested in returning to VHA?**
**Recruitment cost avoidance paid for survey!**

**2001 – Prior Employee Summary**

## 2001 – Prior Employee Summary

**Some differences among job functions but generally very little variance on “the issues”**
**Areas of strength, attraction:**
**Benefits package (and pay in comparison to norms)**
**Systems and equipment enabling effective performance**
**Sense of accomplishment from serving Veterans**
**Areas for attention, to improve retention:**
**Managers who communicate openly, get employee input and provide employees with coaching and feedback to improve**
**Capitalizing on employee ideas and opinions to improve effectiveness**
**Operating as a team**
**Treating employees with respect, and fairness**

## 2001 - Prior Employee SurveyThe “Oh my god!” slide

***Could your manager (or anyone else) at VHA have done something to persuade you to remain at VHA? ***

## 2001 VHA Employee Survey

**Attempt to reduce number of questions failed**
**173 questions!**
**Paper survey**
**35% response rate**
**Contractor prepared a printed report and performed some analysis**
**VHA received the data sets 2 months after the survey closed**
**Analysis & feedback to facilities by VHA National Center for Organizational Development with researchers in VHA & academic partners**

## 2004 - VHA All Employee Survey Values

**Collect and use survey data**
**To better understand employee satisfaction & perceptions of organizational culture, policies & practices**
**To improve the workplace environment, employee satisfaction, & organizational culture**
**Protect the confidentiality of individual responses & of aggregate data for groups of less than ten respondents**
**Ensure the transparency of aggregate data for groups of ten or more respondents**
**Committed to the rigorous analysis of survey data using sound statistical techniques**
**Share the analyses of the data in fair, honest, objective manner in order to guide effective action**
**Not use survey data or the results of the analysis to punish, demean or otherwise hurt any individual, any group of individuals, any work unit or any organization**

## 2004 – VHA All Employee Survey

**59 questions examining satisfaction at individual, group & organizational culture levels **
**29 questions can be tracked back to 1997**
**3 modalities – web, paper & IVR**
**109,000 respondents -- 51% response rate**
**VHA received datasets 10 days after survey closed**
**Internal analysis with academic collaboration**
**Feedback to facilities began 30 days after the survey closed**
**Rule of 10 for reporting aggregate data**

## Slide 10

## Current Survey Intervention Model

## Slide 12

## VHA Performance Measures

**September 2004**
**Action plan to address lowest scores within 120 days after survey closed**
**September 2005**
**Are planned activities on-going?**
**September 2006**
**.5 improvement in STD DEV on issues addressed in 2004 action plan**

## A Business Case for Employee Satisfaction

**Employee satisfaction travels with**
**Patient satisfaction**
**Lower costs of providing patient care after adjusting for economies of scale**
**Higher employee perceptions of the quality of care provided veterans**
**Higher levels of measurable quality of care provided to veterans**
**e.g. Chronic disease index**

## Civility Scale from VHA AES

**People treat each other with respect**
**A spirit of cooperation/teamwork exists**
**People I work with can be relied on when I need help**
**Disputes resolved fairly**
**People I work with take a personal interest in me**
**Differences among individuals respected and valued**
**Managers work well with employees of different backgrounds**
**This organization does not tolerate discrimination**

## Slide 16

## Relationship of Workplace Civility to Overall Employee Satisfaction (Black) and Intent to Stay (Blue)

**Notes:**
To examine the practical impact of civility, we performed what is called a utility analysis.  A utility analysis can take several specific forms, but the general idea is to group your cases – whether these be individuals or organizations – according to their level on a certain variable of interest, and to then compare those “dosage groups” on some relevant performance measure(s).

In this case, our variable of interest is civility.  Accordingly, we divided the 2004 AES respondents into four roughly equal groups (quartiles) based on their scores on the civility scale (10 items).  Thus we created a high civility group, a low civility group -- and two groups in between.  In round-number terms, there were about 27,000 employees in each of the four groups.

We then examined the scores of these four groups on two relevant criterion measures: overall employee satisfaction (as measured by JSI Q12), and intent to remain in the VA (as measured by OAI Q22).  As you may recall, OAI Q22 is actually phrased in terms of intent to leave the VA, so we reverse-coded this item so that a “higher score is better,” to be consistent with all the other JSI and OAI items.

This graph displays the results.  As you can see, those employees in the higher civility groups had higher satisfaction and higher “intent to stay” scores than those in the lower civility quartiles.  Specifically, with regard to overall job satisfaction, there was a 1.5 point difference (4.52 vs. 3.01) between the highest and lowest civility groups.  With regard to “intent to stay,” there was about a 1.6 point difference (4.23 vs. 2.65) between the highest and lowest civility groups.  Remember that the scale for these items is 5 points.  Differences of the magnitude seen here on a 5-point attitude scale are huge and are highly statistically significant.  For the statisticians in the audience, the one-way analysis of variance (ANOVA) results were, for overall satisfaction: F(3,107231) = 12663.5, p<.0001; for intent to stay: F(3,103927) = 8058.61, p<.0001

## Relationship of Workplace Civility to Perceptions of Outpatient (Green) and Inpatient (Red) Overall Quality Ratings

**Notes:**
We also examined the relationship between civility and another important performance measure – patient satisfaction, as measured by the VA SHEP surveys.  We conducted this analysis at the facility (station or STA3N) level, because our patient satisfaction data is not linked to particular individuals but is an average for each facility as a whole.  Accordingly, we averaged the civility scores of all AES respondents at each facility to get a civility score for each facility.  

We then created four groups (quartiles) of facilities based on those civility scores: a high civility group, a low civility group, and two groups in between.  As our measure of patient satisfaction, we used the overall quality question from both the outpatient and inpatient SHEP surveys.  This is the question that asks veterans to rate the overall quality of the care they have received at the facility in question during the past two months on a 5-point scale from poor to excellent.  OQP then translates this to a 100-point scale.  As before, we then compared the scores of the four civility groups on these two patient satisfaction items.

This graph displays the results, and they were again dramatic.  Veterans making outpatient visits to high civility facilities rated overall quality 5.52 points higher than did those who received their care at facilities in the lowest civility group (77.37 vs. 71.85).  Veterans who were inpatients at high civility facilities rated overall quality a whopping 8.4 points higher than did veterans who stayed at facilities in the lowest civility group (79.5 vs. 71.1).  Differences of this magnitude on these patient satisfaction measures huge and are highly statistically significant by one-way analysis of variance ANOVA.  For outpatient quality: F(3,126) = 7.87, p<.0001; for outpatient quality: F(3,121) = 9.02, p<.0001

## Emerging ideasEngagement

**Preconditions for empowerment**

## Engagement

**Possible additional questions to 2006 VHA AES**
**I feel a strong personal connection with the mission of VA**
**What VA stands for is important to me**
**I am proud to tell people I work for VA**
**I feel a sense of “ownership” for VA...**
**VA cares about my general satisfaction at work**
**VA cares about my opinions**
**VA takes pride in my accomplishments at work**

## Empowerment

**Possible additional questions to 2006 VHA AES**
**Members of my workgroup are able to bring up problems and tough issues**
**In my workgroup, when a problem is identified, we try to learn from it and change things for the better**
**It is safe to take a risk in this workgroup**

## 2006 – VHA All Employee Survey

**April 24 – May 15, 2006**
**Web, paper & IVR**
**Network or facility mapping of work group identifiers**
**Hierarchical – official supervisor**
**Functional team (recommended for clinical teams)**
**Goal – 65% response rate**
**Additional questions to measure**
**Engagement & empowerment**
**VANOD questions for RNs**

## What matters?

**VHA mission**
**One VA Survey 1997 and 2001, Prior Employee Survey, VA Workplace Stress & Aggression Project (WSA)**
**Relationship with supervisor**
**One VA Survey 1997, 2001 Prior Employee Survey, WSA, 2004 VHA AES, NCOD interviews, & anecdotal ADR data**
**Relationships with coworkers**
**Prior Employee Survey, WSA, 2004 VHA AES & NCOD interviews**

## Are Civility, ADR, Employee Satisfaction and Organizational Outcomes Connected?

**People treat each other with respect**
**A spirit of cooperation/teamwork exists**
**People I work with can be relied on when I need help**
**Disputes resolved fairly**
**People I work with take a personal interest in me**
**Differences among individuals respected and valued**
**Managers work well with employees of different backgrounds**
**This organization does not tolerate discrimination**

**Is the ability to resolve minor differences at the work group level an important pre-condition or enabler of civility?**