Using Sensitivity Analysis to Identify Sig

P t i N t k Si l ti Parame

ters in a 

N

e

twor

k Simulati

image generated using http://wwimage generated using http://ww

Kevin Mills & Jim Filliben, NIST Kevin Mills

& Jim Filliben, NIST

Winter Simulation Conference – Dec 6
Outline

• Goal – Problem – Solution• Scale Reduction: Theory and Practice• O i f th 20 P t i Overview of the 20 Parameters in M N

eso 

a TCP\IP Network Simulator

• 2-Level Per Parameter Experimental Des

– Theory

– Application to MesoNet

• Selected Analysis Techniques

– M i Eff t A l i Main Effects Analysis

– Two Factor Interaction Analysis– Tabular Summary Analysis

• Relati e Importance of Relative Importance of MesoNet MesoNet Parame

Parame

• Findings and Conclusions
Goal – Problem – Solution

• Goal

– compare proposed Internet congestion 

algorithms under a wide range of controlled, 

repeatable conditions as simulated by selectirepeatable conditions, as simulated by selecticombinations of parameter values for MesoNemesoscopic network model

• Problem

– how to determine key parameters 

influencing behavior in MesoNet, a 20-paramenetwork d l mo

d

el?

• Solution Solution

– apply 2 apply 2

-level

-per

-factor orthogona

fractional factorial (OFF) experimental desigrelated data analysis techniques to identify trelative importance of model parameters
Simulating large, fast networks across many conditions and cong

algorithms requires scale reduction in both model parameters 

Scale Reduction: Theory & Practice

y

1, ...,

yz

= f(

x1|[1,...,

l] ...,

xp|[1,...,

l]

)

ResponseState‐Space Stimulus State‐Space

(232)1000 O(109633) [ 1080 = atomsin visible universe]

Multidimensional Response Parameter 

Reduction (2 )

Discardparameters not germane to study – reduce by 944 parameters

O(10 ) [ ]

(232)56 O(10539)

20

Group related remainingparameters– reduce by 36 param

22 Responses

CorrelationAnalysis &Clusterin

g

PrincipalComponentsAnalysis

Talk on We

Use experimentdesign the

bi i 

220

(232)20 O(10192)

O(10

6)

Model Reduction

Select only 2 valuesfor each parameter

Level Reduction

g

7 Responses 4 Responses

Domain

Analysis

parameter combinationsto 

Use sensitto identity 

significant

220‐12 Experiment 2

Design Theory

Sensitivity

7 Responses Analysis

This Talk

Next Talk

Use experimentdesign theo

parameter combinationsto 3

This Talk
2-Level Per Parameter 

Orthogonal Fractional 

Factorial Ex

p 

erimental De

Theory
Sample & Population

Y = f(X1

, X2, X3, ..., Xk)

h f i( ) li ?

f( , , 3, ...,

?

Key: What 

factors are 

in 

(my

) popu

lation

?

What scope (robustness) do I want my conclusio

to be valid over?

Sample {...}

n =

Populati

N =

Representative

1. random sampling2. blockin

g

3. proportional sampling4. orthogonal fractionation

S iiSummar

ization

Inference
What is a 2-Level Per Parameter Desig

Each experimental parameter 

p is assigned only 2 of its po

What is a

2

-Level Full Factorial Desig

Each experimental parameter, p, is assigned only 2 of its po

What is

a

2 Level Full Factorial Desig

An experiment is conducted for each of the 2

p parameter 

What is a 2-Level Fractional Factorial (FF) 

An experiment is conducted for a 2

p-m subset of parameter 

What is a 2-Level Orthogonal FF (OFF) D

The choice of the 2

p-m subset of parameter combinations for 

is made in a fashion that achieves balance and orthogo

minimizing confounding of interactions between main eff

also between main effects and 2-term interactions and min

variance in the estimation of effects 
Why 2 Levels Per Factor?

Pros• Requires relatively few runs per factor• F ilit t s i t t ti f s s d t Facilit

a

t

es i

n

terpre

t

ation

of re

spon

s

e d

a

t

a

• Identifies promising directions for future 

exper ments, and may be augmented w th tho

iments, and may be augmented with tho

local exploration

• Forms basis for 2-level fractional factorial de• Fits naturally into a sequential strategy, whic

supports the scientific method

Cons• Limited exploration of parameter values• Assu

mes 

m y onotonicit

y in ran

ge between chose
Why Orthogonal Fractional Factorial D

2-Level Design for MesoNet requires 220 = 1 048 576 runs

At 28 processor hours per run and with 48 available processors, 

runs would require about 612 000 hours (70 years)

Adopting a 220-12 OFF experimental design would reduce the resourc

to only 256 runs, which could be completed in about 150 hours 

Cost: misses 220 - 28 parameter combinations

OFF Benefit #1: Superior Coverage & Robustness when compared with1-Factor-at-a-Ti

+ + + +

X4

X5

+

X1

X2

X3

+

+_

_

_

_

+ X4

X5

+

X1

X2

X3

+

+_

_

_

_

+ X4

X5

+

X1

X2

X3

+

+_

_

_

_

+ X4

X5

+

X1

X2

X3

+

+_

_

_

_

+

OFF Design 1-FAT Design

X7

X4

X5

+

X1

X2

X3

+

+_

_

_

_+

+ X4

X5

+

X1

X2

X3

+

+_

_

_

_

+

+ X4

X5

+

X1

X2

X3

+

+

_ _

_

_

_+

+ X4

X5

+

X1

X2

X3

+

+

_ _

_

_

_+

+

X5

X4

X6
What is the minimum number of require

Minimally strive for a resolution IV design, i.e., a design whis no confounding among parameters and between parame2

-parameter interactions

Requires sufficient runs, 

n, to resolve a leading constant, the pa

2-parameter interactions: 

n = 1 + 

p + C(p, 2)

MesoNet example – parameters, 

p = 20

Minimum runs 

n = 1 + 20 + C(20, 2) = 1 + 20 + 190 = 211

Given 2-levels 

per factor, we can choose the first 

power of 2 ab

n = 256 = 220-12 – this is a resolution IV design 

n = 2

p-r , where the 

factor is 

r
2**(20-12) Orthogonal Fractional Factorial Design (k=20, n=256, l=2) Resolution = 4 Specifying Parameter Combinatio

 2to20m12.xls

x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 x11 x12 x13 x14 x15 x16 x17 x18 x19 x

1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 

2 1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 1 -1 -1 -1 -1 -1 3 -1 1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 1 1 1 1 1 1 4 1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 5 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 1 1 1 1 6 1 -1 1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 1 1 1 1 1 7 -1 1 1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 -1 -1 -1 -1 -1 8 1 1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 9 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 1 -1 -1 -1 -1 

10 1 -1 -1 1 -1 -1 -1 -1 1 -1 1 1 1 1 1 -1 -1 -1 -1 11 -1 1 -1 1 -1 -1 -1 -1 1 -1 1 1 1 1 -1 1 1 1 1 12 1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 1 1 1 13 -1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 -1 -1 -1 1 1 1 1 14 1 -1 1 1 -1 -1 -1 -1 -1 -1 1 1 1 1 -1 1 1 1 1 15 -1 1 1 1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 -1 -1 -1 -1 16 1 1 1 1 -1 -1 -1 -1 1 1 -1 -1 -1 -1 1 -1 -1 -1 -1 17 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 1 -1 -1 -1 18 1 -1 -1 -1 1 -1 -1 -1 1 1 -1 1 1 1 -1 1 -1 -1 -1 19 -1 1 -1 -1 1 -1 -1 -1 1 1 -1 1 1 1 1 -1 1 1 1 20 1 1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 1 1 1 21 -1 -1 1 -1 1 -1 -1 -1 1 -1 1 -1 -1 -1 1 -1 1 1 1 22 1 -1 1 -1 1 -1 -1 -1 -1 1 -1 1 1 1 1 -1 1 1 1 23 -1 1 1 -1 1 -1 -1 -1 -1 1 -1 1 1 1 -1 1 -1 -1 -1 24 1 1 1 -1 1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 1 -1 -1 -1 25 -1 -1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 -1 1 1 -1 -1 -1 26 1 -1 -1 1 1 -1 -1 -1 1 -1 -1 1 1 1 1 1 -1 -1 -1 27 -1 1 -1 1 1 -1 -1 -1 1 -1 -1 1 1 1 -1 -1 1 1 1 28 1 1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 -1 -1 -1 1 1 1 29 -1 -1 1 1 1 -1 -1 -1 1 1 1 -1 -1 -1 -1 -1 1 1 1 30 1 -1 1 1 1 -1 -1 -1 -1 -1 -1 1 1 1 -1 -1 1 1 1 31 -1 1 1 1 1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 -1 -1 -1 32 1 1 1 1 1 -1 -1 -1 1 1 1 -1 -1 -1 1 1 -1 -1 -1 

220-12 Experiment Design Template

256

32 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 
Design Properties: Balance & Orthog

(p = 20, n = 256)

- +

128 128

X

Balance All 20:

i

+ 64 64

All : Xj 202 Orthogonality

- +

- 64

Xi

64
(p = 20 

FAT = 1.4142...

OFF Design Benefit #2: Minimizes Variation in Effec

(p = 20

1 1

1 2 ( ) n n SD Bi = σ +

)

0.125

128
2-Level Per Factor OFF De

Applied to

MesoNet Sensitivity Analy
Overview of the 20 Parameters

* in MesoNet, a TCP\

Simulator

Network Para

x1 Network Speed

x2 Propagation Delay

x3 Buffer Provisioning

x4 Topology

User Behavio

x5 Web Browsing File Sizes

x6 Larger File Download Probability

& Sizes

x7 User Think Time

x8 User Patience

S &R 

x9 Spatiotemporal Congestion on Very Fast Paths

x10 Number, Location and Start Time for Long‐Lived Flows

x11 Speed of Interfaces Connecting Sources

& Receivers to Network

x12 Number of Sources

& Receivers

Sources 

&

R

Protocols

x13 Distribution of Sourcesx14 Distribution of Receiversx15 Probability of Source using

a specific Congestion Control Algorithm

x16 Initial Size of Congestion Window (cwn

d )

Simulation & Control

x17 Initial Slow Start Threshold (sst)x18 Measurement Interval Size

x19 Simulation Durationx20 Startup Pattern for Sources

*For more details, attend related talk on Wed. 8:30 AM
Abilene-based Topology: (-) Lev

A1a

A2a A2b

I2c

I2a I2b K1a K1b

I1e I1f I1g A2c K1c

A

I

A1

A2

A1b

A1c

A1d

A1e C2

C2f

C2e

C2d C2c

C2b

C2a E1

E1b

E1a G2e

G2c

G2b

G2a

G2

G2d

G2f

I1c

I1b

I1a

I1

I1d

I2

I2c

I2d

K1

I2f

K2

G2g K2a

I2e E0a

I1e

I2g

A2d E1c E1d K

C

G

K

J

A1e

E

A1f A2g

B0a

B1b

B1a

C1

C2

C2g

E2a E2

E2b

E2c

G1b

G1c G1d

G2a

C1G

G1a

G1f J2

J

I0a

J2g

K0a

G1g

C0a

B

D

H

J B1

B2g

B2

B2a

B2bB2c

C1a

C1b

C1c C1d

C1e

C1f E2f

E2c

E2d E2e

G1c G1dG1e

J1

J1e

J1c J1d

J1b

J1a

J1f

J

J1g

B1c C1g E2g

B1d

D

F

B2f

B2

B2d

B2e D1

D1b

D2g

D1a

D2

D2a

D2b D2e

F0a

F1

F1bF1a F2

H2

H2aH2b

H2c H2d

H2e

H1

H1b

H2g

H1a

F2f

H2f

D2f

F2g

F1g H1c H1d

Backbone RBackbone LBackbone RPoP Routers 

 l

Access Router n x Normal SpeedAccess Router m x Normal Speed

F1d

D2c D2d F1bF1c

F2a

F2b F2c F2d

F2e

g

F1e F1f

D1c D1d

Access Router Normal Speed

102 to 103 Sources (not shown) and Receivers (n

under each Access Router 

PoP Routers Access Rou

Directly Fast Typical 

Routes are shortest-path based on propagation delay

yp
Commercial ISP-based Topology: (+) 

A2a A2b A2c K1b K1c K1d

K2a K2b

I1b I2a I1c I1d I2d I2b I2c

A

C

K

A1

A1b

A1c

A1a

C1

C1b

C1c

C1a

H1

H1c H1b

H1a H1d

H1eH1f

K1

K2

K0a

K1a

K1b K2a K2b

K2c

K2d

L1

J1a

J1b J1c J1d

J1e J2a

I2

I1

I I0a

I1a

I2g

I2f

I2e

B2c B2d B2e

B2f H

A2

J

B

L

C2

C2b

C2c

C2a

H2bH2c

H2a

H2f

H2e

H2d

L2

L1

L1a L1b

L1c

L1d

L2aL2b

L2c

L0a

L0b

J1

J2

J1f

J2a

J2b

J2c

J2d

J2e J2f

M2d M2e M2fM2g

B2

B2a

B2b B2g

H

H2

D

F

M

B

N

O

L2c

L2d N1a

N1b

N1c

N1d

N1e

N1f N2

N2a N2f

M2b

M2c

M2

g

M1a M2a

M1b M1

M1c

M1dM1e M1f M1g

F2a

F1

F1a

F1b F1c F1d

F2c

F2b

D2

D2a

D1a D1

D2g

D2f

D2d D2b D2c

B0a D2e

B1a B1

B1b

B1c B1d

N1

B kb 

E

G

O

E1

E1b

E1a

G1b

G1a

N2a

N2b

N2c N2d N2e

N2f O0a

O1a

O1b

O1c

O1c

O1

O2a

O2

O2b O2c O2d

O2e

O2f

O2g

F2

F2g F2f

F0a F2d

F2e

D1a D1

D1b

D1c D1d

D0a

E2

BackboneBackbone Backbone PoP RoutAccess R

P E1c

E2b E2a E2c

G1bG1c

G1e

G1d

G2a G2b G2c

G2e

G1f G2d

G2f

P1 P2

P1a

P1b

P1c

P1d

P2a

P2b P2c P2d

P2e

P2f

P2g G1 G2 Direct

Fast Typica

Routes are shortest-path based on traffic engineering goals
2 Levels Per Factor Used in Sensitivity 

Category Category ID Name Minus ( Minus (-1) Level ) Level Plus (+ Plus (+1Network Configuration 

X1 Network Speed 800 packets/ms 1600 packetX2 Propagation Delay 1 2 X3 Buffer Provisioning RTT

x

C/sqrt(

n) RTT

x

C

X4 Topology Abilene - SPF prop. 

dela

y

ISP - SPF costs

User Behavior 

X5 Web Object Size (

 = 1.5) 75 packets 150 packets 

X6 Proportion/Size of Larger Files 

(Fx = 10 Sx = 1000 Mx = 10000) 

Fp = 0.02 Sp = 0.002 Mp = 0.0002 

Fp = 0.04 SMp = 0.0004 

X7 User Think Time 2 s 5 s X8 User Patience 0.0 (Infinite) 1.0 (Finite) X9 Selected Spatiotemporal

4th X9 Selected Spatiotemporal Time Period None

Congestion 

4 Time Period None

X10 Long-lived Flows 3 Start in 3rd Time 

Period 

None 

Sources &

X11 Probability of Fast Interface 0.2 0.8 X12 Number of Sources & 2 3 Sources

&

Receivers Receivers X13 Distribution of Sources Web Centric Peer-to-Pee

X14 Distribution of Receivers Web Centric Peer-to-Pee

Protocols 

X15 Probability of Algorithms TCP = 0.8 CTCP = 0.2 TCP = 0.2 CX16 Initial Congestion Window Size 2 packets 8 packets X17 Initial Slow Start Threshold Initial Slow Start Threshold 43 packets 43 packets 1 073 741 82

1 073 741 82

Simulation & Measurement Control 

X18 Measurement Interval Size 200 ms 1 s X19 Simulation Duration 25 minutes 50 minutes X20 Source Startup Pattern Exponential (mean = 

X7) 

50 % start e
Traffic Scenario(s)
M iR Fl G f Th h t A

18 Macroscopic Response Variables Throughput in each of 2

102 Response Variables

Macroscopic Responses Flow Groups for Throughput Aver

Category Identity Definition Number File Size Path Class M

Flow

Y1 Average # sources connecting 1 VF Y2 Average # sources sending 2 VF Y3 % sending flows in initial slow start 3 F Y4 % di fl i d d i F 

Flow State Movie Y4 % sending flows in standard congestion avoidance 4 F 

Y5 % sending flows in alternate congestion 

avoidance 5 T 

6 T 

Congesti Y6 Retransmission rate 7 VF 

Y7 A i id i ( k) 8 VF 

Congesti on Software 

Service 

Pack 

Y7 Average congestion window size (packets) 8 VF Y8 Aggregate # connection failures 9 F 

10 F 

Delay Y9 Average round-trip time (ms) 11 T 

Y10 Average queuing delay (ms) 12 T 

13 VF 

Docume

nt 

Work Y11 Average # flows completed per second 14 VF 

Y12 Average # packets output per second 15 F 

16 F 

LongLived 

Flows 

Y13 Average throughput on long-lived flow #1 17 T Y14 Average throughput on long-lived flow #2 18 T Y15 Averag gp g e throughput on long-lived flow #3 19 VF 

Web 

Object 

20 VF 

Flows by 

Path 

Class 

Y16 Average throughput on flows transiting VF paths 21 F Y17 Average throughput on flows transiting F paths 22 F Y18 Average throughput on flows transiting T paths 23 T 

24 T 

T 

Averaged over each of three time periods (3 x 18 = 54 responses)

Average Computed Separately foand CTCP Flows ( 2 x 24 = 48 re
Selected Analysis Techniq

1. Main Effects Analysis2. Two Factor Interaction Analysis3. Tabular Summary Analysis
Sample: 1 of 102 Main Effects Anal

Larger InitialSlow Start 

Larger ThresholdFil

FasterNetwork

ShorterPropagationDelay

Files

Throughput (pps) for Movies transferred over Very Fast Paths with Fast Interfa
Another Sample: 1 of 102 Main Effects A

Shorter Think Time More Sources Distributed in a Peer-to-Peer PatternLarger Files

Bigger

SlowerNetwork

Topology

Y2 – Average Number of Sending Sources in Time Perio
Sample: 1 of 102 Two Factor Interaction A

Two Factor Interaction Plot for Y2 

– Avg. Number of Sending Sources in 

(not much in the way of significant 2 factor interactions) 
Sample: 1 of 5 Tabular Summary Ana

Metric

Class Y# Name

Network User Behavior Source/Receiver P

X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13 X14 X15 

NSp PrD Buf Top FS LFS ThT UP CVF LLF SSR NSR DiS DiR CCA 

Y1

# Connecting +** +** +* ‐** +** +** 

Y2

# Active +** +** +** ‐** +** +**

Flows

Y2

# Active + + + ‐ + +

Y3

% ISS +** +** +** ‐** +** ‐** ‐** 

Y4

% NCA ‐** ‐** ‐** +** ‐** +** +** ‐*

Y5

% ACA +** +** +** ‐** ‐** +** 

Y6 Retrans. Rate ‐** ‐** ‐** +** ‐** +** +** 

Congestion Y7 cwn

d Size +*

Y8

# conn. fails ‐** ‐** ‐** +** +** +** 

Delay

Y9 SRT

T ‐** +** +** +* +*

Y10 Queue Delay ‐** +** +** ‐* +** +**

Aggregate Aggregate Y11 Flows/sec +** * +**   +**  +**

TP

Y11 Flows/sec +** ‐* +** ‐** ‐** +** +**Y12 Packets/sec +** +** +** +** ‐** ‐** +**

Long‐Lived

Flow TP

Y13 LLF 1 +** +* +* +** ‐**

Y14 LLF 2 +* +** +** ‐**Y15 LLF 3 +** +** ‐* ‐*

Other

Flow TP

Y16 VF Paths +** ‐** ‐** +* +** ‐** ‐* 

Y17 F Paths +** ‐** +* +** +** ‐** +** ‐** Y18

N Paths +** ‐** ‐** +** ‐** ‐** 

Significant Influence of each Factor on each Macroscopic Response in 
Network User Behavior Source/Receiver Pro

Another Sample: 1 of 5 Tabular Summary A

File

Type

Path

Class

Connection

Speed

X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13 X14 X15 

NSp PrD Buf Top FS LFS ThT UP CVF LLF SSR NSR DiS DiR CCA 

Movies

VF Fast +** ‐** +** VF Normal +** ‐** +** ‐* 

F Fast +** +** ‐** ‐** +** ‐* +** ‐* Movies F Normal +** +** +** ‐** +** ‐** +** ‐*

T Fast +** +** ‐** +** ‐** ‐**T Normal +** +** ‐** +** ‐** ‐**

VF Fast +** ‐** +** VF Normal ‐** ‐* +** 

Service

Packs

F Fast +** ‐** +** +** +* +** ‐* 

F Normal +** ‐** +** +** +** ‐** +** ‐* 

T Fast +** +** ‐** +** ‐* ‐**T Normal +** +** ‐** +** ‐* ‐**

VF Fast ‐** +* 

l ** ** 

Documents

VF Normal ‐** +** 

F Fast +** ‐** +** +* +* ‐* +** F Normal +** ‐** +** +** +** ‐* +** 

T Fast +** ‐** +** +** ‐** ‐** T Normal +** ‐** +** +** ‐** ‐** 

VF Fast ** +** +** 

Web

Objects

VF Fast ‐** +** +** VF Normal ‐** +* +** 

F Fast +** ‐** ‐* +** +** +* +** F Normal +** ‐** ‐* +** +** +** +** 

T Fast +** ‐** +** ‐** ‐** T Normal +** ‐** +** ‐** ‐** 

Significant Influence of Each of 20 Factors on Throughput for Each of 2

when using CTCP 
Relative Importance o

MesoNet Parameters
Summary of Influence of Each Factor on All R

Protocol

T‐test

Statistic

Network User Behavior Source/Receiver Protoc

X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13 X14 X15 X16 

NSp PrD Buf Top FS LFS ThT UP CVF LLF SSR NSR DiS DiR CCA ICW 

Time

P id #1

>0.99 17 9 10 8 8 0 11 0 0 3 0 12 11 2 1 7 

>0.95<0.99 1 1 3 2 2 0 2 0 0 0 0 3 2 1 0 2 

Period #1 Total 18 10 13 10 10 0 13 0 0 3 0 15 13 3 1 9 

Time

Period #2

>0.99 16 9 9 6 7 0 10 0 6 2 0 13 11 1 1 5 

>0.95<0.99 2 1 1 2 0 0 2 0 0 1 0 1 1 2 1 1 

Total 18 10 10 8 7 0 12 0 6 3 0 14 12 3 2 6 

>0 99 17 9 11 6 9 0 12 0 4 3 0 12 11 2 1 5 

Time

Period #3

>0.99 17 9 11 6 9 0 12 0 4 3 0 12 11 2 1 5 

>0.95<0.99 1 2 0 3 1 0 1 0 1 0 0 3 2 0 0 0 

Total 18 11 11 9 10 0 13 0 5 3 0 15 13 2 1 5 

TCP

>0.99 19 16 12 8 11 0 10 0 1 0 0 4 16 0 0 8 

>0.95<0.99 0 2 3 3 5 0 4 0 0 0 0 2 0 0 0 0 

Total 19 18 15 11 16 0 14 0 1 0 0 6 16 0 0 8 

CTCP

>0.99 19 18 10 9 15 0 13 0 1 0 0 8 16 0 0 7 

>0.95<0.99 0 0 2 3 1 0 3 0 0 0 0 6 0 4 0 1 

Total 19 18 12 12 16 0 16 0 1 0 0 14 16

4 0 8 

Total

>0.99 88 61 52 37 50 0 56 0 12 8 0 49 65 5 3 32 

Total >0 95 . <0 99 . 4 6 9 13 9 0 12 0 1 1 0 15 5 7 1 4 

Total 92 67 61 50 59 0 68 0 13 9 0 64 70 12

4 36 

 l E E 

% of responses influenced 90% 66% 60% 49% 58% 67% 63% 69% 13% 12% 4% 35

9%

Significant Influence of Each of 20 Factors on Each of 18 Macrosc
Findings: 7 main factors drive MesoNet Re

• Capacity (network speed)• Demand (number, distribution and activity of • Physics (propagation delay)• Buffer sizing• File sizes
Conclusions

• 2-Level-per-Factor Orthogonal Fractional Fac

(OFF) experimental designs can reveal signifiinf m u mu m ormation about simulation models

• MesoNet simulation appears to be driven by t

key factors that influence behavior in real ne

• App p p p ears feasible to compare proposed Intern

congestion control algorithms while varying onMesoNet parameters

Entire Study Report NIST Special Publication 500Study of Proposed Internet Congestion Control MAvailable online at h // i /i l/ d/C i C 

http://www.nist.gov/itl/antd/Congestion_Cont