# Presentation: 092682

## SafeStat

**SafeStat**

- Measurement System
- (Version 8.6)

- 7-12-99

## What is SafeStat?

- SafeStat is a **data-driven analysis system** that determines the **current**** ****relative safety status** of individual motor carriers
- SafeStat was developed at the Volpe Center for the Federal Motor Carrier Safety Administration (FMCSA)
- Data used are maintained and managed at the Federal level by the FMCSA

- 7-12-99

## Uses of SafeStat

- FMCSA Compliance Reviews (CRs):
- Semiannually identifies and prioritizes carriers for on-site FMCSA compliance reviews
- PRISM Program:
- Identifies and monitors poorly performing carriers for the PRISM Federal/State safety improvement process (MCSIP)
- Inspection Selection System (ISS):
- Supports recommendation of evaluated carriers’ drivers and vehicles for roadside inspections
- A&I Online (www.ai.volpe.dot.gov)
- Makes SafeStat results available via the internet to industry and the public to promote safety awareness and self-improvement

- 7-12-99

## SafeStat Methodology

**SafeStat Methodology**

- Involves **analytically assessing** a motor carrier in four **Safety Evaluation Areas** (SEAs):
**Accident** SEA
**Driver **SEA
**Vehicle**** **SEA
**Safety Management **SEA
**Each SEA** is based on **two or more indicators** supported by different data sources

- 7-12-99

## SafeStat Design

**Event and exposure data **are used to calculate** **normalized** measures **for each carrier
**Measures** are ranked and assigned percentile values (**Indicators**) from 0-100 - with 100 being highest or worst
**Indicators** are combined into **SEA values**
**SEA values** are weighted and summed to derive the **SafeStat score**

- 7-12-99

## SafeStat Hierarchy and Features

- The **SafeStat **algorithm** **design** **is computationally  _**hierarchical **_beginning with** safety event data **building to a** SafeStat Score**
- Algorithm _**features**_** data sufficiency tests, normalization **and** weighting**

- 7-12-99

## SafeStat Hierarchy and Features

_**Features**_

_**Features**_

- 7-12-99

## SafeStat Overview

**SafeStat Overview**

**MOTOR CARRIER SAFESTAT SCORE**

**Data Sources by Carrier**

- 7-12-99

## SafeStat Data

- At the foundation of SafeStat are **safety event data** and **exposure data**
- Carrier specific** safety event data** reflect the carrier’s safety compliance and performance and include:
- State Reported Crashes (last 30 mos.)
- Recordable Crashes from CRs (last 12 mos. CRs)
- Roadside Inspection Violations (last 30 mos.)
- Moving Violations (last 30 mos.)
- Compliance Review Violations (last 18 mos. CRs)
- Closed Enforcement Cases (last 6 years)
**Exposure data** **normalize** a carrier’s safety event data:
- Number of drivers/vehicles and VMT
- Number of inspections

- 7-12-99

## SafeStat Measures

- Safety **measures **are the result of normalizing safety event data
- Example:  **accident event data** are converted to **accident rates** which take into account differences in **exposure**

- 7-12-99

## SafeStat Indicators

**Indicators** rank carriers by their safety **measures** converted to a percentile (0-100) scale.
- When determining **indicators**, SafeStat may employ peer groupings that reflect differences in operations to assure appropriate comparisons among carrier types and size classes

- 7-12-99

## SafeStat SEA Values

**Safety Evaluation Areas (SEAs)** represent the four major criteria used to evaluate carriers’ safety status
- A **SEA Value,** also on a 0-100 percentile scale, is derived from the **Indicators** related to that SEA
- For example a* SEA Value of ****85**** *means*:*
***85% ****of the carriers (that have sufficient data) have a better safety status in that SEA*
*15%** **have a worse safety status*

- 7-12-99

## SafeStat SEAs

- Accident
- Driver
- Vehicle
- Safety Management

- 7-12-99

## Accident SEA Summary

- 7-12-99

## Accident SEA

**	****	****	****	**
- Data & Measures
**State reported crashes** **(using NGA standard) **over past 30 months normalized by **# of power units** from FMCSA carrier registration data yield the **Accident Involvement Measure (AIM)**
**Recordable crashes** from Compliance Reviews (CRs) conducted during the previous 12 months normalized by **vehicle miles traveled (VMT)** yield the **Recordable Accident Rate (RAR) Measure **

**Crash!**

- 7-12-99

## Accident SEAAccident Involvement Indicator (AII)

-  Accident Involvement Measure (AIM) applies time weighting (most recent crashes have greatest weight) and severity weighting (crashes involving injury/fatality and/or HM release have more weight)
-  AIM is calculated by dividing weighted-crashes by average # of power units
-  AII is percentile ranking of weighted AIM

-  Carriers are peer-grouped by similar # of crashes and ranked on a percentile basis to obtain an AII

- 7-12-99

## Accident SEA - Recordable Accident Indicator (RAI)

-  # of Recordable Crashes and last 12 months VMT data are gathered during CR
-  Recordable crashes are divided by VMT and multiplied by 1 million to obtain a Recordable Accident Rate (RAR) per million miles traveled.
-  Carriers are peer-grouped by similar # of crashes and then ranked on a percentile basis to obtain the RAI

- 7-12-99

## Accident SEA Value Calculation

- For Carriers with no CR within past 12 months:
- Accident SEA = AII
- Carriers with CR within past 12 months and no state-reported crashes since the CR
- Accident SEA = RAI
- Carriers with CR within past 12 months and at least one state-reported crash since the CR
- Accident SEA = highest of (AII, RAI)

- 7-12-99

## Driver SEA Summary

- 7-12-99

## Driver SEA

**Driver SEA**

**Data and Measures:**
**Driver OOS Violations** normalized by number of roadside inspections over past 30 months yield the **Driver Inspection Measure (DIM)**
**Driver-Related Critical and Acute Violations** from CRs completed within past 18 months yield the **Driver Review Measure (DRM)**
**Moving Violations** over past 30 months normalized by number of drivers yield the **Moving Violation Measure (MVM)**

- 7-12-99

## Driver SEA Driver Inspection Indicator (DII)

**Driver SEA ****Driver Inspection Indicator (DII)**

- DIM is based on OOS inspections that are severity-weighted (based on # of DOOS viol.) and time-weighted:
- 	0-6 months old # of inspections (3x)	7-18 months old # or inspections (2x)	19-30 months old # of inspections (1x)
- DIM is adjusted up for violations of OOS orders (aka jumping OOS orders)
- Carriers are peer-grouped by similar # of driver inspections
- DII is percentile ranking of DIM

- 7-12-99

## Driver SEA - Driver Review Indicator (DRI)

**Driver SEA - Driver Review Indicator (DRI)**

- Driver Review Measure (DRM) is based on violations of driver-related Critical and Acute Regulations from CRs
- DRM accounts for the number and severity of violations
- DRI is percentile ranking of DRM

- Carriers with CR and no violations are given a DRI of 0

- 7-12-99

## Driver SEA - Moving Violations Indicator (MVI)

**Driver SEA - Moving Violations Indicator (MVI)**

-  MVM is based on serious Moving Violations (MVs) issued in conjunction with roadside inspections normalized by the number of drivers
- Time weighting is applied to violations:
- 	0-6 months old # of MVs (3x)	7-18 months old # or MVs (2x)	19-30 months old # of MVs (1x)
- Carriers are peer-grouped by similar # of MVs
- MVI is percentile ranking of MVM

- 7-12-99

## Driver SEA Value Calculation

- Driver SEA Value is the **highest** of the DII and DRI and uses the MVI when its value is greater than the DII and DRI
- When the MVI is greater than the DII and DRI, Driver SEA is equal to the weighted average of MVI and the highest of the DII and DRI.

- 7-12-99

## Vehicle SEA Summary

- 7-12-99

## Data and Measures:
Vehicle OOS Violations normalized by number of roadside inspections over past 30 months yield the Vehicle Inspection Measure (VIM)
Vehicle-related Critical and Acute Violations from CRs completed within past 18 months yield the Vehicle Review Measure (VRM)

**Vehicle OOS Violations** normalized by number of roadside inspections over past 30 months yield the **Vehicle Inspection Measure (VIM)**
**Vehicle-related Critical and Acute Violations** from CRs completed within past 18 months yield the **Vehicle Review Measure (VRM)**

**   ****Vehicle SEA**

- 7-12-99

## Vehicle SEA - Indicators VII & VRI

**Vehicle SEA - Indicators VII & VRI**

-  VII is similar to DII but uses Vehicle OOS violations instead of Driver OOS violations

-  VRI is similar to DRI but uses Vehicle-related violations of Acute/Critical regulations

- 7-12-99

## Vehicle SEA Value Calculation

**Vehicle SEA Value Calculation**

- Vehicle SEA Value is the **highest** of the VII and VRI.

- 7-12-99

## Safety Management SEA Summary

- 7-12-99

## Safety Management SEA

**Data and Measures:**
- Safety Management/HM **Critical and Acute Violations** from CRs completed within past 18 months yield the **Safety Mgmt Review Measure (SMRM) **and the** HM Review Measure (HMRM)**
- SMRM and HMRM account for the number and severity of violations
**Closed Enforcement Cases** are used to determine the **Enforcement Severity Measure (ESM)**

- 7-12-99

## Safety Mgmt SEA - Indicators SMRI & HMRI

- SMRI is similar to DRI but uses Safety Management-related violations of Acute/Critical regulations
- HMRM is similar to DRI but uses Hazardous Material-related violations of Acute/Critical regulations

**Safety Mgmt SEA - Indicators SMRI & HMRI**

- 7-12-99

## Safety Mgmt SEA - Enforcement History Indicator (EHI)

**Safety Mgmt SEA - Enforcement History Indicator (EHI)**

-  Enforcement Severity Measure (ESM) is calculated for carriers with closed enforcement cases in the past 6 years
-  ESM includes severity weighting (based on # of violations cited) and time weighting of each enforcement case:
- 	0-12 months old cases (4x)	13-30 months (3x)	31-50 months (2x)	51-72 months (1x)
-  EHI is percentile based on ESM rank
-  EHI of 75-100 assigned to carriers w/CR -initiated enforcements within past 30 months with either
- 	1) no subsequent CR or
- 	2) a subsequent CR resulting in acute/critical 	violations.
-  EHI of 50-74 assigned to all other carriers w/  enforcements.

- 7-12-99

## Safety Mgmt SEA Value Calculation

**Safety Mgmt SEA Value Calculation**

- Safety Management SEA Value is the **highest** of the EHI, SMRI & HMRI

- 7-12-99

## SafeStat Detailed Summary

- 7-12-99

## SafeStat Results

- The SafeStat Score
- Assignment of Carriers to Categories
- Scored carriers
- Single SEA carriers
- Example of SafeStat Results
- Effectiveness Study

- 7-12-99

## The SafeStat Score

- The **SafeStat score** only applies to carriers with safety deficiencies
- Only carriers that have deficient SEA values of 75 and higher (the worst 25th percentile)** **in **two or more** of the four **SEAs **receive a **SafeStat Score**

- 7-12-99

## SafeStat Score Calculation

- A carrier must have **two or more SEAs **with a value **of 75** **or greater **(worst 25th percentile)
- SafeStat **sums only the SEAs with values of 75** **or greater** to determine the **SafeStat Score**
- More emphasis is put on the **Accident SEA** (twice the weight) and **Driver SEA** (1.5 greater weight) than Vehicle and Safety Mgmt SEAs

- =

- 2 X

**SafeStat**
**Score**

**Acc.**
** ****SEA**

- +

- +

**Driver**
** ****SEA**
** **

**Vehicle**
** ****SEA**
** **

**Safety**
**Mgmt.**
** ****SEA**

- +1.5 X

- 7-12-99

## SafeStat Categoriesfor Scored Carriers

**SafeStat Categories****for Scored Carriers**

- SafeStat evaluated carriers are assigned to **Categories (A-G)** based on their SafeStat Scores and SEA Values

_**SafeStat Categories for Scored Carriers**_

- 7-12-99

- Categories of carriers deficient in one SEA (SEA Value of 75 or higher)

_**SafeStat Categories for Carriers with One SEA Value**_

**Single-SEA ****SafeStat Categories **

- 7-12-99

## Example of SafeStat Results

**Example of SafeStat Results**

- SafeStat Score:	383.02
- Overall Rank: 	19
- State Rank:	3
- Previous  Status:	Warning 			letter
- Current Status:
- 	*Category A ( At Risk)  CR Recommended*

* **Name: Rollemover Express*
- DOT # 12345 Physical Address		Mailing Address		Power units:         **35 Launch Pad Road ****	****	****P.O... BOX 1234 ****	****	**Hazmat Carrier:    **yes Yourtown, Ourstate 12345****	****Yourtown, Ourstate 12345 ****	**Passenger Carrier: **No****	****	**

- 7-12-99

## SafeStat Effectiveness Study

- 4/04/2002

**SafeStat Effectiveness Study**

- 	A study was conducted to confirm SafeStat effectiveness by comparing post-run crash rates for scored vs. unscored carriers:
- (1)	SafeStat was run 18 months in the past with data available at that time to identify and score carriers
- (2)	Carriers with sufficient data were assigned to 3 groups based on the SafeStat run results: At-Risk (Categories A&B), Other Scored (Category C) and unscored
- (3)	Post-run crash rates for each group were observed and compared

## Effectiveness Study Results

- Carriers identified by SafeStat have higher crash rates than carriers not identified by SafeStat

- 7-12-99