GEM Validation

Greenhouse [Gas] Emission Model

Southwest Research Institute

11 December 2014

11-Dec-2014 1
Outline

• Introduction

• Test vehicles, test platforms, and drive cycles

• Test configurations

• GEM validation

• Understanding test variation

• GEM use enhancements

• Summary

11-Dec-2014 2
Introduction

• GEM is a vehicle simulation tool

– Meant for showing compliance to US FE and GHG standards; 

not designed for research and development

– Used by all vehicle OEMs

• GEM capable of simulating

– Engine: All technologies tested on engine stand – fuel map, 

accessory load, friction, pumping, parasitic loss etc.

– Transmission: MT, AMT, AT

– Driveline: Axles, shafts

– Vehicle: Rolling resistance, aerodynamics

11-Dec-2014 3
Role of SwRI in GEM Development

• In earlier Work Assignments, SwRI worked on 

adding certain component models to GEM

– Current version developed by the EPA

• In recent Work Assignments, SwRI role is as a 

test house validating GEM results by

– Comparing test data to GEM

– Analyzing and understanding differences

11-Dec-2014 4
Test Vehicles and Test Platforms

• Test vehicles

– Ford F650 – Class-6 Vocational Tow Truck with AT

– Kenworth T270 – Class-6 Vocational Box Truck with AT

– Kenworth T700 – Class-8 Tractor with AMT

– Refuse Truck with AT

– Freightliner Cascadia – Class-8 Tractor with MT

• Test platforms

– Heavy-duty Chassis Dynamometer (HDC)

– Medium-duty hardware-in-the-loop stand (HIL or C22)

11-Dec-2014 5
Drive Cycles

• Six drive cycles

– 55 MPH (with grade option)

– 65 MPH (with grade option)

– CARB HHDDT Transient (CARB)

– HTUF Class-6 Parcel (Parcel)

– NREL CILCC (Utility)

– UNECE WHVC (WHVC)

• Characteristics later in the presentation

11-Dec-2014 6
Test Configurations

• Seven configurations

– Baseline (“as received” parameters)

– ± Cd

– ± Crr

– + Mass

– Optimized

• 6 cycles x 7 configurations = 42 cases for parametric 

study (F650 HIL, T270 HDC, and T700 HDC)

• 6 cycles x 1 baseline configuration = 6 cases (F650 

HDC, Refuse HDC, Cascadia HDC)

11-Dec-2014 7
Criterion for GEM Validation

• GEM simulation is deterministic and produces a single 

MPG number for a given vehicle configuration over a 

given drive cycle

• Testing is not so. A given vehicle configuration over a 

given drive cycle likely produces a different MPG 

number for each run

• EPA has chosen to compare test mean to GEM result –

match within 5% considered good

• But, comparison to test mean should not overlook 

standard error of test mean and test variance

11-Dec-2014 8
Examining Test Data and GEM Results

• More than 130 combinations tested

– Vehicles – F650, T270, T700, Cascadia, Refuse

– Drive cycles – 55 MPH, 65 MPH, CARB, Parcel, Utility, WHVC

– Parameter configurations – Cd, Crr, mass

– Platforms – Chassis Dynamometer, HIL

• Time-traces useful for detailed comparison and 

understanding of HDC and GEM data

• Box-plot, density-plot useful to summarize; see 

Appendix for details on box-plot as used here

11-Dec-2014 9
Baseline

10.14

9.855

-10%Cd

10.19

9.966

-10%Crr

 10.3

10.05

+10%Cd

10.05

9.787

+10%Crr

9.998

9.644

+363kg

9.862

9.773

Optim

10.37

10.12

9.6

9.8

10.0

10.2

10.4

F650; Test Platform HIL; Drive Cycle CARB MPG

Baseline

10.14

9.855

-10%Cd

10.19

9.966

-10%Crr

 10.3

10.05

+10%Cd

10.05

9.787

+10%Crr

9.998

9.644

+363kg

9.862

9.773

Optim

10.37

10.12

9.6

9.8

10.0

10.2

10.4

F650; Test Platform HIL; Drive Cycle CARB MPG

mean(HIL)

GEM

SwRI 2014-12-08 13:07

GEM: filled triangle, HIL: unfilled square, HIL outlier: unfilled circle

11-Dec-2014 10
F650; Test Platform HIL; grouped by drive cycle

mean(HIL) [MPG]

GEM [MPG]

8

10

12

14

8 10 12 14

SwRI 2014-12-08 13:07

gray lines mark the +-5% region around 1:1

55 MPH

65 MPH

CARB

Parcel

Utility

WHVC

11-Dec-2014 11
-15%Cd

5.386

5.359

-15%Crr

5.358

5.398

+15%Cd

5.149

 5.11

+15%Crr

 5.08

5.077

+907kg

5.153

5.203

Baseline

5.247

5.234

Optim

5.526

 5.53

5.0

5.1

5.2

5.3

5.4

5.5

5.6

T700; Test Platform HDC; Drive Cycle WHVC MPG

-15%Cd

5.386

5.359

-15%Crr

5.358

5.398

+15%Cd

5.149

 5.11

+15%Crr

 5.08

5.077

+907kg

5.153

5.203

Baseline

5.247

5.234

Optim

5.526

 5.53

5.0

5.1

5.2

5.3

5.4

5.5

5.6

T700; Test Platform HDC; Drive Cycle WHVC MPG

mean(HDC)

GEM

SwRI 2014-12-08 13:07

GEM: filled triangle, HDC: unfilled square, HDC outlier: unfilled circle

11-Dec-2014 12
T700; Test Platform HDC; grouped by drive cycle

mean(HDC) [MPG]

GEM [MPG]

4

6

8

10

4 6 8 10

SwRI 2014-12-08 13:07

gray lines mark the +-5% region around 1:1

55 MPH

65 MPH

CARB

Parcel

Utility

WHVC

11-Dec-2014 13
-10%Cd

6.343

6.493

-10%Crr

6.434

 6.52

+10%Cd

6.345

6.402

+10%Crr

6.366

6.375

+363kg

6.094

6.417

Baseline

 6.33

6.447

Optim

6.504

6.566

6.0

6.1

6.2

6.3

6.4

6.5

6.6

T270; Test Platform HDC; Drive Cycle Parcel MPG

-10%Cd

6.343

6.493

-10%Crr

6.434

 6.52

+10%Cd

6.345

6.402

+10%Crr

6.366

6.375

+363kg

6.094

6.417

Baseline

 6.33

6.447

Optim

6.504

6.566

6.0

6.1

6.2

6.3

6.4

6.5

6.6

T270; Test Platform HDC; Drive Cycle Parcel MPG

mean(HDC)

GEM

SwRI 2014-12-08 13:07

GEM: filled triangle, HDC: unfilled square, HDC outlier: unfilled circle

11-Dec-2014 14
T270; Test Platform HDC; grouped by drive cycle

mean(HDC) [MPG]

GEM [MPG]

6

8

10

12

6 8 10 12

SwRI 2014-12-08 13:07

gray lines mark the +-5% region around 1:1

55 MPH

65 MPH

CARB

Parcel

Utility

WHVC

11-Dec-2014 15
55 MPH

6.562

6.906

65 MPH

5.158

5.477

CARB

4.278

4.418

Utility

4.697

5.132

WHVC

 5.44

5.627

4.5

5.0

5.5

6.0

6.5

7.0

Refuse Truck; Test Platform HDC; All Drive Cycles MPG

55 MPH

6.562

6.906

65 MPH

5.158

5.477

CARB

4.278

4.418

Utility

4.697

5.132

WHVC

 5.44

5.627

4.5

5.0

5.5

6.0

6.5

7.0

Refuse Truck; Test Platform HDC; All Drive Cycles MPG

SwRI 2014-12-08 13:07

GEM: filled triangle, HDC: unfilled square, HDC outlier: unfilled circle

11-Dec-2014 16
Refuse Truck; Test Platform HDC; grouped by drive cycle

mean(HDC) [MPG]

GEM [MPG]

4

5

6

7

4 5 6 7

SwRI 2014-12-08 13:07

gray lines mark the +-5% region around 1:1

55 MPH

65 MPH

CARB

Utility

WHVC

11-Dec-2014 17
©Copyright Southwest Research Institute ® 2014

Summarizing All Data

Composite Plots – All Vehicles All Tests

• A compact way to display MPG difference 

between mean test data and GEM results for 

all data

• More elaborate analysis of MPG difference 

possible via constructed kernel density

11-Dec-2014 18
Test Data; grouped by vehicle

mean(Test Data) [MPG]

GEM [MPG]

4

6

8

10

12

14

4 6 8 10 12 14

SwRI 2014-12-08 13:26

gray lines mark the +-5% region around 1:1

F650

Refuse

T270

T700

11-Dec-2014 19
Test Data; grouped by drive cycle

mean(Test Data) [MPG]

GEM [MPG]

4

6

8

10

12

14

4 6 8 10 12 14

SwRI 2014-12-08 13:26

gray lines mark the +-5% region around 1:1

55 MPH

65 MPH

CARB

Parcel

Utility

WHVC

11-Dec-2014 20
Understanding MPG Difference

Drive Cycle Characteristics

• See next two slides for time-chart and 2-d density 

plot of vehicle speed “v” and acceleration “a”

• CARB: Short duration, yet fairly transient

• Parcel: Long duration; lower speeds; higher 

accelerations; heavy idle time

• Utility: Long duration; lower speeds; lower 

accelerations; steady-state cruise at various speeds

• WHVC: Medium duration; balanced mix of transient 

operation and highway cruise (higher speeds)

11-Dec-2014 21
0 1000 2000 3000 4000

0 5 10 15 20 25

CARB

t [s]

v [m/s]

0 1000 2000 3000 4000

0 5 10 15 20 25

Parcel

t [s]

v [m/s]

0 1000 2000 3000 4000

0 5 10 15 20 25

Utility

t [s]

v [m/s]

0 1000 2000 3000 4000

0 5 10 15 20 25

WHVC

t [s]

v [m/s]

11-Dec-2014 22
0 5 10 15 20 25 -2 -1 0 1

CARB

v [m/s]

a [m/s^2]

0 5 10 15 20 25 -2 -1 0 1

Parcel

v [m/s]

a [m/s^2]

0 5 10 15 20 25 -2 -1 0 1

Utility

v [m/s]

a [m/s^2]

0 5 10 15 20 25 -2 -1 0 1

WHVC

v [m/s]

a [m/s^2]

11-Dec-2014 23
©Copyright Southwest Research Institute ® 2014

Distribution of Test Statistic

Composite Plots – All Vehicles All Tests

• Need a compact way to assess quality of test 

data for all vehicles over all test cycles

– Constructed kernel density function of coefficient of 

variation of test data defined as cov=mean/std

 WHVC most repeatable of all transient drive cycles

 Refuse truck (and F650) most repeatable of all vehicles

– Constructed kernel density better than histogram

 Histogram: strong dependence on bin boundaries

 Kernel density: choose “bandwidth” carefully

 Area under constructed kernel density is 1

11-Dec-2014 24
Kernel Density: HDC Coef of Variation by Cycle

cov=sigma/mu

Density

0

50

100

150

200

250

0.00 0.01 0.02 0.03

data at the bottom (unfilled circles) was used

to construct the density function (solid lines)

55 MPH

65 MPH

CARB

Parcel

Utility

WHVC

11-Dec-2014 25
Kernel Density: HDC Coef of Variation by Vehicle

cov=sigma/mu

Density

0

100

200

300

0.00 0.01 0.02 0.03

data at the bottom (unfilled circles) was used

to construct the density function (solid lines)

F650

Refuse

T270

T700

11-Dec-2014 26
©Copyright Southwest Research Institute ® 2014

Understanding Test Variation

Dynamometer Control and Vehicle Operation

• F650: Mix of HDC and HIL data; HDC using roll 

torque meter; HIL using transmission output 

shaft torque meter; robotic driver; AT

• T270: HDC using roll torque meter; human 

driver; AT

• T700: HDC using roll torque meter; human 

driver; AMT

• Refuse Truck: HDC using driveshaft torque 

meter; robotic driver; AT

11-Dec-2014 27
©Copyright Southwest Research Institute ® 2014

Understanding Test Variation

Additional Factors

• [Human] driver behavior

– Pedal aggressiveness

– Transmission shift schedule and shift duration

 Shift times affected by HDC speed behavior

• Transmission temperature

• Tire temperature

• After-treatment thermal management

• Accessory load

11-Dec-2014 28
GEM: Use Enhancements

• Perform vehicle coast-down simulation in GEM 

and compare to on-road coast-down

– Adjust A, B, C parameters to match coast-down

– GEM-set A, B, C akin to dynamometer-set A, B, C?

11-Dec-2014 29
©Copyright Southwest Research Institute ® 2014 ©Copyright Southwest Research Institute ® 2014

GEM as a Screening Tool

11-Dec-2014 30
GEM as a Screening Tool

Outside the Designed Use!

• Parametric study on F650 HIL-stand about to 

begin for additional GEM validation

• Parameters of interest: final drive ratio, drag 

coefficient, vehicle mass

• Conducted parametric study using GEM to get 

a sense of operating range and corners

• HIL testing will be conducted for the corner 

combinations marked A-I

• Legend identifies final drive ratios

11-Dec-2014 31
27 28 29 30 31 32

13

14

15

16

17

18

19

20

21

22

N/V [RPM/mph]

Work [hp-hr]

F650: 55 MPH GR; BSFC[g/hp-hr]=f(N/V,Work);brake

A B C

D

E F

G

H I

175

175

175

180

185

190

195

SwRI 05-Dec-2014 13:18:41

3.36

4.33

4.86

5.29

11-Dec-2014 32
67 68 69 70 71 72 73 74 75

3.8

4

4.2

4.4

4.6

4.8

5

5.2

5.4

5.6

5.8

N/V [RPM/mph]

Work [hp-hr]

F650: CARB; BSFC[g/hp-hr]=f(N/V,Work);brake

A B C

D E F

G H I

200

205

205

210

215

220

225

SwRI 05-Dec-2014 13:18:42

3.36

4.33

4.86

5.29

11-Dec-2014 33
Summary

• Development version of GEM incorporates several 

engine, transmission, driveline, and vehicle 

technologies being considered to meet FE and 

GHG standards for MD/HD vehicles

• GEM validated against 130+ test cases (multiple 

runs) of different size vehicles

• Difference between GEM FE and test data 

generally less than 5%

– Mismatch explainable

• Sources of variation in test data must be 

understood and considered during GEM validation

11-Dec-2014 34
©Copyright Southwest Research Institute ® 2014 ©Copyright Southwest Research Institute ® 2014

Appendix

11-Dec-2014 35
Visual Data Analysis: Box and Whisker Plot

• Thick line at median of the sample population (halfway mark)

• Bottom edge of the box at first quartile Q1 (i.e., 25% below it)

• Top edge of the box at third quartile Q3 (i.e., 25% above it)

• Half of sample population within the box

• Inter-Quartile Range = Q3 – Q1

• UF = third quartile + 1.5*IQR. UF stands for “upper fence”

• Data above UF is labeled “outlier” (circle). Upward whisker 

ends at the largest value of the sample population below UF

• LF = first quartile – 1.5*IQR. LF stands for “lower fence”

• Data below LF is labeled “outlier” (circle). Downward whisker 

ends at the smallest value of the sample population above LF

• Width of the box proportional to the sample size 11-Dec-2014 36
Visual Data Analysis: Box and Whisker Plot

• Non-parametric: No assumed underlying distribution

• Spread and skew: Spacing between different parts of the plot

• Outliers: Uses a generally accepted criterion to mark outliers

11-Dec-2014 37
Box and Whisker Plot: A Familiar Example

• For normal distribution ~ N(0,1):

– μ = 0; σ= 1

– Q1= -0.6745; Q3 = 0.6745; IQR = 1.349; 

– LF=Q1-1.5*IQR = -2.698; UF=Q3+1.5*IQR = 2.698

– 99.3% population between fences [LF,UF]

– 99.73% population between ±3; i.e., 3σ

11-Dec-2014 38