NREL is a national laboratory of the U.S. Department of Energy, Office of Energy Efficiency and Renewable Energy, operated by the Alliance for Sustainable Energy, LLC.

Map Matching and Real World 

Integrated Sensor Data Warehousing 

www.nrel.gov/tsdc

www.nrel.gov/fleet_dna 

Evan Burton – Data Engineer (Presenter)

Jeff Gonder – Vehicle System Analysis Team Lead

Adam Duran – Engineer/Analyst

Eric Wood – Engineer/Analyst

2013 Federal Committee on Statistical Methodology 

(FCSM) Research Conference

November 4-6, 2013

Washington, DC

NREL/PR-5400-60893
2

NREL–Real World Vehicle Drive Cycle Database

Uses a significant sample group of 

both personal and commercial 

vehicles to provide a resource for 

engineers and researchers focusing 

on:

• Regional vehicle emissions modeling

• Alternative vehicle application 

identification

• Vehicle-to-grid electrification

• Regional transportation demand 

modeling

• Green routing

• Intelligent route-based control 

strategies for hybrid electric vehicles

• Long-term battery degradation 

analysis 

Map matching sensor data appends 

additional variables at second-bysecond increments :

• Allows for scalability of analysis 

• Incorporates route choice and 

geography into results

Quantifies otherwise overlooked variables in vehicle analysis.
3

The Data – Location/Speed/Time 

Points by Speed
4

Location and Time

• All data sets are linked using an XY coordinate system (Earth)

• Links established by passing primary keys through records 

o Trip 1 ended at tract 3

• Time provides the final link for moving objects 

o Vehicle 1 enters road 1 at time A and exits at time B 

Spatial Reference Data:

• 2010 Census

• 2011 ACS

• USGS 1/3 DEM

• NAVTEQ Streets

State

County

Tract

Block-groups

Streets

Zip 

Codes

Land Use

Elevation

Deployment

Points

Trips

Vehicles
Map Matching
6

Vehicle Processing – End Product

• Second-by-second data are 

summarized at the trip level

o Trips are also categorized by end and start 

type (home/work/school)

• Vehicle processing of the second-bysecond data includes:

o Filtering the recorded data

o Appending data from additional data layers 

to second-by-second data

• Summary statistics are generated at 

the vehicle, day, and trip levels

• Trip classifiers are grouped to vehicle 

and day levels

Vehicle Processing (Next Slide)

1. Speed – Filtered

2. Location – Filtered

3. Elevation – Appended

4. Grade – Appended

5. Street ID – Appended

6. Street Functional Class – Appended

7. Street Speed Category – Appended
7

Vehicle Processing – End Product

Speed – Filtered

Location – Filtered

Elevation – Appended Grade – Appended Street ID – Appended

Street Functional Class 

– Appended

Street Speed Category 

– Appended
8

Speed & Location Filters

Severe outlier filter

• Removes points falling outside 

a data defined buffer

• Radius = Hours of travel day X 

70 mph

Mean Center

• The mean latitude 

and longitude 

independently

Speed Filtration

• Flags unrealistic speed

• Backfills missing points

• Interpolates missing and 

flagged data

GPS Jumps Filter

• Calculates a rough speed value 

from distance and time

• Points where the calculated 

speed is > 100 mph are flagged 

until calculated speed falls under 

100 mph

1 

2 

Typically less than 2% of GPS points require these processing adjustments 3 
9

Map Matching: Dynamic Node Selection

Line Buffer Street Select

Unique Node 

Options

1 2 

3 4 
10

Map Matching: Solve

Point-to-node links are 

established using time

• Each point is linked to the nearest node 

within 500 feet of a road

• Link is established by appending the node 

identifier to the point data (time)

• Points are down sampled to a discrete time 

at which the vehicle passed the node

• Vehicle x passed this node at this second

Solving connections logically (topology)

• The segments connected to the node and the possible exit node options for each 

entry node are identified independent of previous or next known results

• Options are then tested logically in order using multi-step decision tree

o Identifies and removes latent links

o Fill gaps in connection of consecutive segment links

• Appends attributes from the linked street segment to the data points falling within 

the entry and exit time of the matched segment
11

Map Matching: Accuracy

Complex overpasses

• Connectivity can become 

ambiguous when so many 

options are available

• 95% of distance matched 

across all data sets

• Cleaned up post processing 

during road based analysis

Points by Speed

Matched Link Results

Point/Link Overlay
12

Elevation/Grade 

USGS DEM

• The elevation for each location 

recorded is looked up from the USGS 

Digital Elevation Model (DEM)

• The elevation profile returned is 

smoothed

• Down sampled to normalize for 

distance to calculate grade

• The processing accuracy has been 

validated against roads with available 

ADAS grade measurements
Integrated Sensor Data 

Warehousing 
14

NREL - Real World Vehicle Data Warehousing

• Transportation Secure Data Center (TSDC) – DOT and DOE 

o Securely archives and provides public access to detailed

regional transportation survey data

o Speed/time/location data for personal vehicle samples

• Reporting is specific to project using the same source data, but different 

ways of looking at it

o TSDC – Trip starts and ends are classified relative to the home, work, school 

location 

o Fleet DNA – Trip starts and ends are classified relative to the depot location

• Fleet DNA – DOE 

o Warehouses medium- and heavy-duty fleet vehicle 

operational data across vocations, drivetrains, fuel type, 

and vehicle types

– Has the ability to crowd source data collection, allowing for

large-scale comparative analysis

o Removes identifying information and provides data to 

public for vehicle performance comparisons
15

Geography: Sample Groupings

Fleet DNA categorizations 

(vehicle specs):

• Make/Model/Year

• Class

• Vocation

• Type

• Drivetrain

• Fuel Type

• City

• State

TSDC categorizations (geographic)

• Regions for data collection

• Make/Model/Year

Region of Deployment (Atlanta) Region of Deployment (Chicago)

Deployment (Caltrans)
16

Privacy & Data Security

• Both TSDC and Fleet DNA data sets contain extreme 

detail regarding the travel of the vehicles

o Vehicles equipped with GPS sensors require certain 

precautions to be taken to ensure privacy and data security

• Each public distribution portal has specific controls in 

place
17

Data Center Infrastructure

TSDC Website Fleet DNA Website

TSDC Virtual Machines

Data Center Servers

Processing Server Visualization and Reporting 

Server

File sharing over 

secure network

Secure file 

transfer

Secure Connection

NREL Firewall

Data Server

VM Server

Virtual Desktop User’s PC

Cleansed 

TSDC data

Anonymous 

fleet data
Analysis Examples
19

Vehicle Drive Cycle Characterization

Analysis Level: Vehicle

Drive Cycle Visualization

• Generate graphs of speed vs. 

time to observe vehicle 

operation

• Display histograms of useful 

statistics such as acceleration for 

analysis of performance 

envelopes

• Output latitude and longitude 

data for route visualization and 

mapping to ensure route/data 

consistency 
20

Fleet Drive Cycle Characterization

Analysis Level: Deployment

Delivery Van Vocational Database:

• Class 3–6 delivery van vehicles

o Parcel, Linen, Food and Product 

vocations

• 92 unique vehicleso Two conventional fuels (diesel/gas)

o Four hybrid configurations (PHEV, 

GHEV, HEV, HHV)

o ~600 EVs to be added

• 1,282 individual operating days

• Over a dozen unique geographic 

operating locations Strong exponential relationship 

between speed and intensity

Wide range of operating 

speeds depending on location

Common performance threshold
21

Charger Utilization: PHEV

Vehicle state profile (% time by 

hour of week)

State of operation by day of operation

Vehicle statistics relative to population

Analysis Level: Vehicle relative 

to population

Vehicle statistics 
22

Regional Road Usage Analysis Level: Regional 

Infrastructure Utilization
23

Daily Road Utilization

Cumulative Daily distance vs. 

Distance on Functional Class

• X axis is cumulative distance

• Y axis is cumulative distance 

functional class 1

Vehicle Sub-sample Road Use

• A small sample of vehicle days where 

the daily distance traveled exceeds 50 

miles

Analysis Level: High resolution daily 

vehicle road use

Videos not available in this PDF
24

High speed 

cruising Hard acceleration

eceleration

Near exclusive existence of 

data at positive acceleration

Hard d

Drive Cycle Prediction

Low-speed data on FC1 

assumed to be 

symptomatic of real-time 

road conditions 

(congestion, weather, etc.)

Analysis Level: Vehicles and 

infrastructure interaction
25

Road Use by Lane Analysis Level: High resolution infrastructure 

utilization by time of day (5 a.m. to 10 a.m.)

Street Segments Driver 

Profiles

• All point data collected for a 

grouping of roads ordered by 

trip visualized where speed is 

the z axis (left)

Road Grid Segments 

Lane Summary

• Road segment is divided into 

grid cells by lane

• Speed data are summarized 

by grid cell

• STDDEV of speed of travel by 

grid cell is visualized (right)
26

Conclusion

• By organizing sensitive, high-resolution data appropriately and 

linking the data to additional data sets using location and time, 

the quality and quantity of data available for researchers can 

be greatly improved while still maintaining restrictions placed 

upon the data

• Benefits 

o Results in an infinite number of ways to group and sort the data for 

reporting

o Increases the number of applications for the data, and the amount of 

data distributed without violating restrictions

• Why never before?

o Location data are very restricted and have a high value, so very little 

data of this kind are available publicly 

– In-vehicle navigation systems utilize similar approach to analysis, but do not 

provide it publicly

o Programming languages and databases now incorporate spatial 

analysis capability, providing free and open source tools with which to 

perform analysis