# Presentation: 616002

## Site Tracking, Analysis and Reporting System (STAR)
 Lynda Blystone

** ****Lynda Blystone**

**Notes:**
A wise person once said...”Numbers are like people, torture them enough, and they’ll tell you anything.”

So what happens when you have a whole lot of numbers, or in our case, probably findings, and you’re trying to figure out what they’re telling you?
If those findings are strewn all over the place, it can be hard to figure out if they’re telling you anything at all.

Making sense of things is what STAR is all about.

## What is STAR?

- “Numbers are like people, torture them enough, and they’ll tell you anything.” – Anonymous
- Site Tracking, Analysis and Reporting System
- Sitewide, web-based tracking system for problems and ongoing work
- ES implemented in June 2004, replacing and consolidating existing local department tracking systems

**Notes:**
First off, what is STAR?  

It stands for Site Tracking, Analysis and Reporting System.  STAR is a  web-based  system that all of SRS uses to track problems and ongoing work.  And it does just that.  It tracks yours problems, helps you analyze them, and helps you report on them.

The Emergency Services Department implemented STAR in June of 2004, a little less than a year ago, consolidating and replacing the systems that were in use at the time.

Today, I’m going to provide you a brief overview of STAR and its usefulness in the tracking and trending of emergency management related issues.

## Why STAR?

- Cost Savings
- Quality Assurance Management Assessment Criterion in 10CFR830 to:
- Establish and implement processes to detect and **prevent** problems
- Identify, control and correct  - items, services and processes not meeting requirements
- Identify **causes** of problems and work to prevent **recurrence**

**Notes:**
Why did we go to STAR?

Let’s talk about the motivators for going to the STAR system.

Cost savings was one driver.  It’s cost effective to maintain one system versus multiple systems.  STAR is accessible to anyone with access to the site’s intranet.  With everyone on the same system, the need for duplicate tracking is eliminated.

Another driver was an OA finding. The OA’s issue (across the board, not just in EP) was that although we were identifying problems, we weren’t identifying the root causes of the problems, and therefore, we were not successfully preventing recurrence of problems.

## Why STAR?

- OA Report (February 2004)
- “WSRC has not established and implemented a fully effective issues management process that consistently evaluates performance, identifies adverse **trends** and root **causes**, and **prevents recurrence** through appropriate actions.”

**Notes:**
Why did we go to STAR?

Let’s talk about the motivators for going to the STAR system.

Cost savings was one driver.  It’s cost effective to maintain one system versus multiple systems.  STAR is accessible to anyone with access to the site’s intranet.  With everyone on the same system, the need for duplicate tracking is eliminated.

Another driver was an OA finding. The OA’s issue (across the board, not just in EP) was that although we were identifying problems, we weren’t identifying the root causes of the problems, and therefore, we were not successfully preventing recurrence of problems.

## Why STAR?

- Consistency
- Multiple processes (drills, self assessment, audits, critiques, etc.) result in issues being counted multiple times or not at all
- Provides a consistent binning structure, making meaningful analysis possible
- Availability

**Notes:**
For EP, the need for consistency was a primary driver for using STAR.  Lack of data is not a problem in EP.  With all the drills we run, the problem is too much data. And not only from drills.  There’s also findings from programmatic self assessments, critiques, actual events, audits.  The binning structure in STAR helps us organize the wealth of data available and make some sense of it.

As I said before, STAR is web-based, and available to anyone with access to the site’s intranet.  In the past, gathering data from the field was time-consuming, if not difficult.  Now the same data is available by running one report.  This is especially important now that the site’s EP group is no longer consolidated and Points of Contact are constantly changing as people are reassigned or leave the site.

With STAR, EP-related data is readily available, whether it originated in the field or the site program.

## STAR Bins

- Drill/Exercise Objectives
- Safety				Recovery
- Protective Actions		Notifications
- Facilities/Equipment		Mitigation
- Offsite Interactions		Drill/Exercise Conduct
- Consequence Assessment	Staff and Activate
- Public Information		Medical
- Classification			Radiological/Chemical 					Monitoring

**Notes:**
So let’s talk about those consistent bins.  For EP, drill issues are binned at the drill objective level.

The bins I mentioned earlier are separated into two sections.  The first, for issues identified during drills are our drill and exercise objectives.  Recovery, Classification, Medical, Public Information, etc.

## STAR Bins

- Other Program Elements
- 	Administration/Organization
- 	Drills and Exercises
- 	Emergency Response Training
- 	Facilities/Equipment
- 	Readiness Assurance
- 	Hazard Surveys/Assessments
- 	Actual Events
- 	Procedures

**Notes:**

The second set of bins is for issues identified outside of drills.  This could me training, hazards assessments, procedures, etc.

## “An undefined problem has an infinite number of solutions.”

- “An undefined problem has an infinite number of solutions.”
- 				- Robert A. Humphrey

**Notes:**
Robert A. Humphrey once said:  “An undefined problem has an infinite number of solutions.”

In my experience, we frequently band-aid drill findings with procedure revisions or lesson plan modifications, only to find the same issue pops ups again in later drills.

The causal analysis tree shown here helps us identify what really needs to be fixed.  In addition to the EP-specific bins we just covered, causal factors can  also be determined and entered directly into STAR.  The tree is integrated directly into STAR.

## Causal Analysis Tree

- DOE M 231.1-2, “Occurrence Reporting and Processing of Operations Information”
- Developed from a benchmarking study of Causal Analysis systems in the nuclear power industry
- Causal analysis process described in DOE G 231.1-2, “Causal Analysis Guide”

**Notes:**
The Causal Analysis Tree contained in  DOE G 231.1-2 was developed as a  result of a benchmarking study of Causal Analysis systems in the nuclear power industry.  

The causal analysis process is described in DOE G 231.1-2.

Between the binning structure and causal factors, multiple data points are available with drilldown capability at the touch of a button.

## Data Points

- Binning/causal factors provide multiple data points
- Functional Area
- Singular bin
- Causal factors

- DRILL DOWN REPORT

**Notes:**
This an example of a drill down report.  You can’t see the entire report here.  But as an example, say I wanted to look at Facility and Equipment  issues over the past 12 months.  

I can define the timeframe, drill down to the individual findings related to Facilities and Equipment, and even down to corrective actions and supporting closure documentation. 

By selecting this particular finding (2005-1111), I found that it relates to procedures lacking direction to consider habitability surveys.  

I’ve highlighted  the CAT code here.  That’s the Causal Analysis Tree (CAT) code determined for this issue.  If we looked back at our CAT tree, we would see that the causes for this issue were determined to be:
Written Communication Less Than Adequate
Incomplete/Situation Not Covered.

By drilling down even further, I was able to reach copies of revised procedures attached as closure documentation.

Attempt to superimpose a screenshot of the STAR tree here.  Consider adding more drilldown slides.

## 	STAR Problems / Solutions

- Inconsistent significance determination
- Classification Issues
- Inconsistent Binning/ Level of Usage

**Notes:**
As you can see, there are many benefits to everyone being on the same system.  There have also been, of course, many difficulties in its implementation.

Primarily, these have been 
inconsistency in the determination of how severe a problem is
Classification issues with UCNI
inconsistent binning
Inconsistent level of usage

## Problems / Solutions

- Problem: Inconsistent significance determination
- Solution: Guidance to promote consistency
- Deficiencies, Weaknesses, Improvement Items and Strengths
- Significance Categories 1-4
- Deficiencies and Weaknesses = minimum of 3
- Improvement Items

**Notes:**

While SRS has a site policy for determining the significance of identified problems, it is somewhat open to interpretation.  The Site EP Program has issued guidance on how EP-related issues should be categorized. 
We use a Finding structure of Deficiency, Weakness, Improvement Item and Strength. The Significance scale for identified problems is 1-4, with 1 being the most significant. 
At a minimum, Deficiencies and Weaknesses are rated as 3’s.
Improvement Items are a different animal.  Since what we call Improvement Items don’t always meet the true definition of a Problem, there are inconsistencies in how they are categorized. 
For example, if an II documents a deficient condition, such as out-of date rally point signs, site policy would dictate it be categorized as at least a 4.  On the other hand, if the II item documents a player performance issue, it may not be required to be categorized and tracked.  We encourage that all issues be entered into STAR, regardless of their significance level,  to insure no repetitive issues are being overlooked.

## Problems / Solutions

- Problem:	Classification
- Solution:	Restricted Access

**Notes:**
Since everyone can access STAR and view its contents, it is only approved for unclassified, non-sensitive information. 

UCNI and other sensitive unclassified information can be entered, provided approved access controls are in place. If you were to login to STAR without an UCNI login code, this is what you would see.

## Problems / Solutions

- Problem: Inconsistent Binning/Level of Usage
- Binning required for Categories 1-3 only
- Solution:	Review and discuss with facility 			assessment coordinators once use of 		the system is stabilized

**Notes:**
These inconsistencies do impact the trending data provided by STAR, and so they need to be addressed.  STAR has been implemented in stages across the site, so different organizations have been using it for different periods of time.  As I said before, we’ve been using it for a little less than a year.  Once use of the system has stabilized across the site, we’ll tackle this one, and try to convince those involved of the benefits of binning all the issues.

## Summary

- “For every problem, there is one solution which is simple, neat and wrong.” – Henry Louis Mencken
- No more band-aids
-  Logically assembled information
- Element
- Cause
- Significance
- Trending identification

**Notes:**
To get real fixes to problems, instead of just band-aid fixes, you have to determine what led to the problem in the first place.

STAR aids that process by assembling information in a logical manner.  First by element, then cause, and finally, by significance.

Once information is organized, trends can be identified.   Once identified,  they can be addressed.

Does anyone have any questions?

Thank you for your time.

Potential question:
Will STAR be available to the complex?