Root Cause Analysis (RCA)
=========================

*March 4, 2009*

*J. Bakken, S. Fuess, T. Levshina, G. Oleynik*

Definition of Root Cause Analysis: 
==================================

> An RCA is distinct from incident or problem resolution. Various
> definitions are available, but these three seem most appropriate:

-   An Activity that defines the underlying or original cause of an
    Incident or Problem. (OGC: ITIL Service Operation)

-   A technique used to identify the conditions that initiate the
    occurrence of an undesired activity or state.
    (http://www.gao.gov/special.pubs/bprag/bprgloss.htm)

-   A process improvement and error or defect prevention tool that
    examines the individual processes within a system, identifies the
    control or decision points, and uses a series of why? questions to
    determine the reasons for variations in the process paths.
    (http://www.bridgefieldgroup.com/bridgefieldgroup/glos8.htm)

Why:
====

-   To identify and determine the cause of underlying failures that lead
    to significant service degradation or outages

-   To identify and determine the cause of recurring problems that are
    indicative of short-sighted solutions or lack of sufficient
    cross-group communication

-   To bring resources to the solution of root cause issues

When:
=====

-   

-   

-   In case of a major incident without a known cause (following a
    reasonable period of investigation), or

-   In case of a major incident where the discovered explanation
    indicates a potential for more incidents, or

-   When the view is that a recurring problem has the potential for
    causing future major incidents but is not being adequately
    addressed, or

-   When there is recognition of an underlying problem resulting in
    otherwise disparate incidents,

> [AND]{.underline}

-   When agreed upon by more than one person in quadrant and department
    leadership, or

-   When requested by a major stakeholder.

> The expectation is that there will be a small number of RCAs necessary
> per year for the quadrant.

-   
-   
-   

How: 
====

-   The quadrant head, deputy, or delegate shall draft an RCA charge,
    including recommendations for:

    -   RCA lead

    -   

    -   RCA membership, to include topic expert, service stakeholder,
        individuals significant to any incidents, and department level
        management relevant to the topic

    -   timescale for RCA report

-   

-   The RCA team shall follow \"Kepner and Tregoe\" method of problem
    analysis:

    -   Define the problem

    -   Describe the problem in terms of identity, location, time, and
        service impact

    -   Establish possible causes

    -   Test the most probable cause (either actual or gedanken
        experiment)

    -   Verify the true cause

-   The RCA lead investigator shall produce a succinct RCA memo to:

    -   Give a concise description of the initiating incident(s)

    -   Summarize the analysis steps

    -   Describe the desired change

    -   Propose a course of action and schedule

    -   Note the consequences of inaction or alternative workarounds

-   The RCA report shall be sent to quadrant and department leadership
    and involved parties as appropriate.

    -   

-   The RCA report shall be published (DocDB). Personnel, vendor
    sensitive or similar issues may be inappropriate for DocDB; such
    references should be expunged from the published version.

-   Outstanding RCAs and associated action items shall be reviewed
    during quadrant/department meetings.

    -   

Process Flow Charts:
====================

Decision Process:

RCA Process: