NRC Independent Evaluation 

INPO’s Safety Culture 

Traits Validation Study

Valerie Barnes, PhD

Office of Nuclear Regulatory Researc

28 September 2010
Purposes of the Study

• Independently evaluate INPO’s approac

data analysis decisions

• Assess whether the factors from INPO’s 

culture (SC) survey correlate with otherculture (SC) survey correlate with other

measures of SC and equipment performthe NRC has available 
INPO/NEI/NRC Roles

• Nuclear Energy Institute funded the data 

collection

• INPO developed the survey, oversaw 

administration, performed majority of the analyses

• NRC reviewed/commented on survey item

study design

• Idaho National Lab, under contract to NRC

independently verified INPO’s analyses, conducted additional analyses
Examples of NRC Measure

• Number, source and type of allegations• Performance indicators maintained under 

Reactor Oversight Process (ROP)

• Ins

pectors’ assi

gnment of SC as

pects to 

pg p

inspection findings

• Location and movement in the ROP Action • Cross-cutting and substantive cross-cuttin

identified during mid-year and year-end performance assessments
Overview of Criterion 

Validity Results

• Correlations between the factors/subfa

and NRC 2009 measures were satisfactoin the expected direction

• Correlations between factors/subfactor

Correlations between factors/subfactor

NRC 2008 measures weaker but also in expected direction
Example SC Validities

*

Factor Variable Co

Mgt Responsibility HP Aspects Raising Concerns Substantiated Allegations Decision Making PI&R Aspects Supv Responsibility Total Aspects 

Questioning Attitude HP Cross-cutting Issues Safety Communication Total Aspects Personal Responsibility HFIS Communication Issues Prioritizing Safety HFIS Work Practices/Procedures Training Quality Total Aspects 

* Correlations a
Example Equipment Validit

Factor Variable Co

Mgt Responsibility Power Changes/7000 hrs Raising Concerns Alert Notification Sys Unavailability 

Decision Making EDG Actuations Supv Responsibility Findings related to Initiating Events 

Questioning Attitude Forced Outage Rate Safety Communication Forced Outage Rate Personal Responsibility Unplanned auto scrams Prioritizing Safety No sig correlations Training Quality EDG Actuations 

* Correlations a
NRC Conclusions

• INPO methods, data analyses and 

interpretations appropriate

• Workshop traits supported by either a f

or subfactor from INPO survey

from INPO survey

• Correlations of Questioning Attitude, 

Supervisor Responsibility, Decision MakTraining Quality and Prioritizing Safety fwith other measures suggest they shoube ignored
Factors from Other Domain

• Analysis techniques applied by INPO ar

standard in survey research

• Used to identify factors in many other s

H d i ld f l f

•

Han

dout inclu

des a 

few examples 

from 

healthcare, construction, manufacturinsmall business settings
Example of Healthcare Survey Facto

Agency for Healthcare Research and Qu

Factors Definition

Communication openness Important patient care information is trans

hospital units and during shift changes

Feedback & communication about error

Management provides a work climate that patient safety and shows that patient safetpriority

Frequency of events reported Staff feel that their mistakes and event rep

held against them, and that mistakes are ntheir personnel file

Handoffs & transitions There is a learning culture in which mistake

positive changes and changes are evaluateeffectiveness

Management support for patient safety

Procedures and systems are good at preveand there is a lack of patient safety problem
Agency for Healthcare Resear

and Quality (con’t)

Factors Definition

Non-punitive response to error Staff feel that their mistakes a

reports are not held against tmistakes are not kept in their file

Organizational learning

—Continuous There is a learnin

g culture in w

Organizational learning Continuous

improvement

g

mistakes lead to positive chanchanges are evaluated for eff

Overall perceptions of patient safety Procedures and systems are g

preventing errors and there is patient safety problems

Staffing There are enough staff to han

workload and work hours are to provide the best care for p
Agency for Healthcare Resear

and Quality (con’t)

Factors Definition

Supervisor/manager expectations and actions promoting safety

Supervisors/managers considsuggestions for improving patpraise staff for following patieprocedures, and do not overlsafety problems

Teamwork across units Hospital units cooperate and 

with one another to provide for patients

Teamwork within units Staff support one another, tr

another with respect, and woas a team

http://www.ahrq.gov/qual/hospsurvey10/
Possible Options for Discuss

Option Pros C

No change to workshop traits Stakeholder agreement Future res

w/ differen

Adopt survey traits Probably align w/ future 

research

Does not 

stake

No traits in policy statement Limits communication; Lack of No traits in policy statement Limits communication;

Allows future research

Lack of 

Only traits that agree w/factors EDG Actuations (NRC) Omits QProgram offices decide Limits communication;

Allows future research

No high-lev

across st

Modified traits, some discussion of subtraits

More future flexibility;Better align w/ research; Close to traits

Not perfec

with 

Mgt factor