An Efficient Sensitivity Analysis Meth

f M i N t kM d l for 

Mesoscopic 

N

e

twor

k

M

o

d

els

image generated using http://wwimage generated using http://ww

Kevin Mills & Jim Filliben, NIST Kevin Mills

& Jim Filliben, NIST

CxS Study Group – Feb 2, 2010
Outline

• Goal – Problem – Solution• Scale Reduction: Theory and Practice• O i f th 20 Overview of the 20 M Nt eso e P t arame ers

• 2-Level Per Parameter Experimental Des

– Theory– Application to MesoNet

• Selected Analysis Techniques

– Main Effects Analysis– Two Factor Interaction Analysis

– Tabular Summary Analysis

• Relative Importance of MesoNet Parame• Conclusions• Future Work
Goal – Problem – Solution

• Goal – compare proposed Internet congestion c

algorithms under a wide range of controlled, repconditions as simulated by selecting combinatioconditions, as simulated by selecting combinatio

parameter values for MesoNet, a mesoscopic nemodel

• Problem – how to determine key parameters inf

behavior in MesoNet, a 20-parameter network m

• Solution – apply 2-level-per-factor orthogonal fr

factorial (OFF) experimental design and related factorial (OFF) experimental design and related 

analysis techniques to identify the relative imporof model parameters
Scale Reduction: Scale Reduction:

Theor

y & Practice
The Function ☺ of a Simulation Mo

y1, ..., ym = f( x1|[1,...,k], ..., xn|[1,...

Model Response Space Model Parameter Space +

+ Determining which responses to examine is an interesting problem in its own right.

Though not addressed in this presentation, we used correlation and principal

components analyses to reduce the response space.
Theory – Scale Reduction in Two P

k

n

model

t i ti

SCIENTIFI

k

n

model

t i ti

SCIENTIFI

A Previous Talk

k(n

‐

r1) res

t

riction

factor

DOMAIN

EXPERTIS

k(n

‐

r1) res

t

riction

factor

k(n

‐

r1) res

t

riction

factor

DOMAIN

EXPERTIS

k(n

‐

r

1

‐

r2)

2

‐level

clustering

k(n

‐

r

1

‐

r2)

2

‐level

clustering

k(n

‐

r

1

‐

r2)

2

‐level

clustering

2(n

‐

r

1

‐

r2) experiment design

OFF (orthogonal fract

STATISTICAL

EXPERIMENT

2(n

‐

r

1

‐

r2) experiment design

OFF (orthogonal fract

2(n

‐

r

1

‐

r2) experiment design

OFF (orthogonal fract

STATISTICAL

EXPERIMENT

2

DESIGN experiment d

EXPERTISE

2

experiment d

2

DESIGN experiment d

EXPERTISE

T d ’ T lk To

day’s 

Talk
Practice – Scale Reduction in Two P

A Previous 

(232

)1000

1 944

O(109633

) [ 1080

= atoms in visible univ

(232

)1000

1 944

O(109633

) [ 1080

= atoms in visible univ

(232

)1000

1 936

O(109633

) [ 1080

= atoms in visible univ

r

1

= 944

(232

)56 O(10539

)

r

1

= 944

(232

)56 O(10539

)

r

1

= 936

(232

)64 O(10616

)

Today’

(232

)20

r

2

= 36

O(10192

)

Domain

(232

)20

r

2

= 36

O(10192 (2

) 32

)20

r

2

= 44

O(10192

)

Domain Today 

Kevin Mills

2

k

=

2 20 O(1

Analyst

2

k

=

2 20 O(1

2

k

=

2 20 O(1

Analyst

Jim Filliben

2

r

3

= 12

8 

Statistician

2

r

3

= 12

8

2 

r

3

= 12

8 

Statistician
Brief Review of Brief Review of

MesoNet Parameters
MesoNet – a TCP/IP network mode

using only 20 parameterusing only 20 parameter

Network Para

x1 Network Speed

x2 Propagation Delay Network Para

x3 Buffer Provisioningx4 Topologyx5 Web Browsing File Sizes

x6 Larger File Download Probability & Sizes

hi k i User Behavio

x7 User Think Time

x8 User Patience

x9 Spatiotemporal Congestion on Very Fast Paths

x10 Number, Location and Start Time for Long‐Lived Flows

x11 Speed of Interfaces Connecting Sources & Receivers to Network

Sources & R

x11 Speed of Interfaces Connecting Sources & Receivers to Networkx12 Number of Sources & Receivers

x13 Distribution of Sourcesx14 Distribution of Receivers

x15 Probability of Source using a specific Congestion Control Algorithm

ProtocolsSimulation & C tl

x15 Probability of Source using a specific Congestion Control Algorithmx16 Initial Size of Congestion Window (cwnd )

x17 Initial Slow Start Threshold (sst)x18 Measurement Interval Size

x19 Simulation Duration Control x19 Simulation Duration x20 Startup Pattern for Sources
2-Level Per Parameter 

Orthogonal Fractional Facto

Experimental Design TheoExperimental Design Theo
What is a 2-Level Per Parameter Desig

Each experimental parameter

p is assigned only 2 of its po

What is a

2

-Level Factorial Design?

Each experimental parameter, p, is assigned only

2 of its po

What is

a

2 Level Factorial Design?

An experiment is conducted for each of the 2

p parameter com

What is a 2-Level Fractional Factorial (FF) 

An experiment is conducted for a 2

p-m subset of parameter co

What is a 2-Level Orthogonal FF (OFF) D

The choice of the 2

p-m The choice of the

2 subset of parameter combinations for 

subset of parameter combinations for 

is made in a fashion that achieves balance and orthog

minimizing confounding of interactions between main eff

also between main effects and 2-term interactions and min

variance in the estimation of effects 
Why 2 Levels Per Factor?

Pros• Requires relatively few runs per factor• F ilit t i t t ti f d t Facilit

a

tes in

terpre

t

ation o

f response 

d

a

t

a

• Identifies promising directions for future experim

and may be augmented with thorough local expand may be augmented with thorough local exp

• Forms basis for 2-level fractional factorial design• Fits naturally into a sequential strategy, which su

the scientific method

Cons• Li it d l ti f t l Li

mit

e

d exploration o

f parame

ter values

• Assumes linear behavior in range between chos

values
Why Orthogonal Fractional Factorial D

2-Level Design for MesoNet requires 220 = 1 048 576 runs

At 28 processor hours per run and with 48 available processors, th

runs would require about 612 000 hours (70 years)

Adopting a 220-12 OFF experimental design would reduce the resource 

to only 256 runs, which could be completed in about 150 hours 

Cost: misses 212 parameter combinations

OFF Benefit #1: Superior Coverage & Robustness when compared with1-Factor-at-a-T

+ + + +

X4

X5

+

X1

X2

X3

+

+_

_

_

_

+ X4

X5

+

X1

X2

X3

+

+_

_

_

_

+ X4

X5

+

X1

X2

X3

+

+_

_

_

_

+ X4

X5

+

X1

X2

X3

+

+_

_

_

_

+

OFF Design 1-FAT Design

X7

X4

X5

+

X1

X2

X3

+

+_

_

_

_+

+ X4

X5

+

X1

X2

X3

+

+_

_

_

_

+

+ X4

X5

+

X1

X2

X3

+

+

_ _

_

_

_+

+ X4

X5

+

X1

X2

X3

+

+

_ _

_

_

_+

+

X5

X4

X6
What is the minimum number of require

Minimally strive for a resolution IV design, i.e., a design wheris no confounding among parameters and between paramete2

-parameter interactions and where any confounding among 

parameter interactions and where any confounding among 

pairs of 2-parameter interactions is known

Requires sufficient runs, 

n, to resolve a leading constant, the param

2-parameter interactions: 

n = 1 + 

p + C(

p, 2)

MesoNet example – parameters, 

p = 20

Minimum runs 

n = 1 + 20 + C(20, 2) = 1 + 20 + 190 = 211

Given 2-levels per factor, we can choose the first power of 2 above 

n = 256 = 220-12 – this is a resolution IV design 

n = 2

p-r, where the re

factor is 

r
2**(20-12) Orthogonal Fractional Factorial Design (k=20, n=256, l=2) Resolution = 4 Specifying Parameter Combinatio

 2to20m12.xls

x1 x2 x3 x4 x5 x6 x7 x8 x9 x10 x11 x12 x13 x14 x15 x16 x17 x18 x19 x

1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 

2 1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 1 -1 -1 -1 -1 -1 3 -1 1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 1 1 1 1 1 1 4 1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 5 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 1 1 1 1 6 1 -1 1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 1 1 1 1 1 7 -1 1 1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 -1 -1 -1 -1 -1 8 1 1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 9 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 1 -1 -1 -1 -1 

10 1 -1 -1 1 -1 -1 -1 -1 1 -1 1 1 1 1 1 -1 -1 -1 -1 11 -1 1 -1 1 -1 -1 -1 -1 1 -1 1 1 1 1 -1 1 1 1 1 12 1 1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 1 1 1 13 -1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 -1 -1 -1 1 1 1 1 14 1 -1 1 1 -1 -1 -1 -1 -1 -1 1 1 1 1 -1 1 1 1 1 15 -1 1 1 1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 -1 -1 -1 -1 16 1 1 1 1 -1 -1 -1 -1 1 1 -1 -1 -1 -1 1 -1 -1 -1 -1 17 -1 -1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 -1 1 -1 -1 -1 18 1 -1 -1 -1 1 -1 -1 -1 1 1 -1 1 1 1 -1 1 -1 -1 -1 19 -1 1 -1 -1 1 -1 -1 -1 1 1 -1 1 1 1 1 -1 1 1 1 20 1 1 -1 -1 1 -1 -1 -1 -1 -1 1 -1 -1 -1 1 -1 1 1 1 21 -1 -1 1 -1 1 -1 -1 -1 1 -1 1 -1 -1 -1 1 -1 1 1 1 22 1 -1 1 -1 1 -1 -1 -1 -1 1 -1 1 1 1 1 -1 1 1 1 23 -1 1 1 -1 1 -1 -1 -1 -1 1 -1 1 1 1 -1 1 -1 -1 -1 24 1 1 1 -1 1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 1 -1 -1 -1 25 -1 -1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 -1 1 1 -1 -1 -1 26 1 -1 -1 1 1 -1 -1 -1 1 -1 -1 1 1 1 1 1 -1 -1 -1 27 -1 1 -1 1 1 -1 -1 -1 1 -1 -1 1 1 1 -1 -1 1 1 1 28 1 1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 -1 -1 -1 1 1 1 29 -1 -1 1 1 1 -1 -1 -1 1 1 1 -1 -1 -1 -1 -1 1 1 1 30 1 -1 1 1 1 -1 -1 -1 -1 -1 -1 1 1 1 -1 -1 1 1 1 31 -1 1 1 1 1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 -1 -1 -1 32 1 1 1 1 1 -1 -1 -1 1 1 1 -1 -1 -1 1 1 -1 -1 -1 

220-12 Experiment Design Template

256

32 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 
Design Properties: Balance & Orthog

(p = 20, 

n = 256)

- +

128 128

X

Balance All 20:

i

+ 64 64

All :

Xj

20

2 Orthogonality

- +

- 64

Xi

64
(p = 20 

FAT = 1.4142...

OFF Design Benefit #2: Minimizes Variation in Effec

(p = 20

1 1

1 2 ( ) n n SD Bi = σ +

)

0.125

128
2-Level Per Factor OFF De

Applied to

MesoNet Sensitivity Analy

Sensitivity Analy
2 Levels Per Factor Used in Sensitivity 

Factor

Parameter

D fi i i efi

nition MINUS (‐) LEVEL 

x1 Network Speed BBspeedup

=

2 R1

= 800 packets/ms BBspeed

x2 Propagation Delay



X

= 1 

X

=

2

x3 Buffer Provisioning RTT

x

C /sqrt(

n ) RTT

x

C

x4 Topology Abilene ‐ SPF propagation delay ISP ‐ SPF 

x5 Web Browsing Browsing File Sizes

on

= 75



= 1 5. on

= 15

x6 Larger File Download Probability

& Sizes Fx

= 10

Sx

= 1000 Mx

= 1000

0 Fx

= 10

S

Fp

= 0.02

Sp

= 0.002 Mp

= 0.0002 Fp

= 0.04

x7 User Think Time 2 seconds 5 second

x8 User Patience NONE REACTIVE RFp

= 0.

0 ALL REAC

x9 Spatiotemporal Spatiotemporal Congestion Congestion on Very Fast Paths 4th Time Period NONE

Jon

= 0.6 Joff

= 0.8 Jx

= 100 Jon

= 1.0 

x10 Number, Location and Start Time for Long‐Lived Flows 3 Start 3rd Time Period with NONE

distances:short, medium, long

x11 Speed of Interfaces Connecting Sources

& Receivers to Network FastHostProb

= 0.2 FastHost

x12 Number of Sources

& Receivers



U

=

2



U

=

3

x13 Distribution of Sources WEB pNs

= 0.1 pNsf

= 0.6 pNsd

= 0.3 P2P pNs

x14 Distribution of Receivers WEB pNr

= 0.6 PNrf

= 0.2 pNfd

= 0.2 P2P pNr

x15 Probability of Source using a specific Congestion Control Algorithm prTCP

= 0.8 prCTCP

= 0.2 prTCP

= 

x16 Initial Size of Congestion Window (cwnd ) 2 packets 8 packet

x17 Initial Slow Start Threshold (sst) 43 packets 1 073 741 

x18 Measurement Interval Size 200 ms 1 second

x19 Simulation Duration 25 minutes 50 minut

x20 Startup Pattern for Sources prOn1st

= 0.0 prOn2nd

= 0.

0 prOn1st

prOn3rd

= 0.0 prRest

= 1.

0 prOn3rd
Abilene-based Topology: (-) Lev

A1a

A2a A2b

I2c

I2a I2b K1a K1b

I1e I1f I1g A2c K1c

A

I

A1

A2

A1b

A1c

A1d

A1e C2

C2f

C2e

C2d C2c

C2b

C2a E1

E1b

E1a G2e

G2c

G2b

G2a

G2

G2d

G2f

I1c

I1b

I1a

I1

I1d

I2

I2c

I2d

K1

I2f

K2

G2g K2a

I2e E0a

I1e

I2g

A2d E1c E1d K

C

G

K

J

A1e

E

A1f A2g

B0a

B1b

B1a

C1

C2

C2g

E2a E2

E2b

E2c

G1b

G1c G1d

G2a

C1G

G1a

G1f J2

J

I0a

J2g

K0a

G1g

C0a

B

D

H

J B1

B2g

B2

B2a

B2bB2c

C1a

C1b

C1c C1d

C1e

C1f E2f

E2c

E2d E2e

G1c G1dG1e

J1

J1e

J1c J1d

J1b

J1a

J1f

J

J1g

B1c C1g E2g

B1d

D

F

B2f

B2

B2d

B2e D1

D1b

D2g

D1a

D2

D2a

D2b D2e

F0a

F1

F1bF1a F2

H2

H2aH2b

H2c H2d

H2e

H1

H1b

H2g

H1a

F2f

H2f

D2f

F2g

F1g H1c H1d

Backbone Backbone Backbone 

Access Router n x Normal SpeedAccess Router m x Normal Speed

F1d

D2c D2d F1bF1c

F2a

F2b F2c F2d

F2e

g

F1e F1f

D1c D1d

Access Router Normal Speed

102 to 103 Sources (not shown) and Receivers (n

under each Access Router 

PoP RoutAccess Ro

DirectFast 

Routes are shortest-path based on propagation delay

Typica
Commercial ISP-based Topology: (+) 

A2a A2b A2c K1b K1c K1d

K2a K2b

I1b I2a I1c I1d I2d I2b I2c

A

C

K

A1

A1b

A1c

A1a

C1

C1b

C1c

C1a

H1

H1c H1b

H1a H1d

H1eH1f

K1

K2

K0a

K1a

K1b K2a K2b

K2c

K2d

L1

J1a

J1b J1c J1d

J1e J2a

I2

I1

I I0a

I1a

I2g

I2f

I2e

B2c B2d B2e

B2f H

A2

J

B

L

C2

C2b

C2c

C2a

H2bH2c

H2a

H2f

H2e

H2d

L2

L1

L1a L1b

L1c

L1d

L2aL2b

L2c

L0a

L0b

J1

J2

J1f

J2a

J2b

J2c

J2d

J2e J2f

M2d M2e M2fM2g

B2

B2a

B2b B2g

H

H2

Bac

kbo

n

e

R

D

F

M

B

N

O

L2c

L2d N1a

N1b

N1c

N1d

N1e

N1f N2

N2a N2f

M2b

M2c

M2

g

M1a M2a

M1b M1

M1c

M1dM1e M1f M1g

F2a

F1

F1a

F1b F1c F1d

F2c

F2b

D2

D2a

D1a D1

D2g

D2f

D2d D2b D2c

B0a D2e

B1a B1

B1b

B1c B1d

N1

ac bo e 

Backbone LBackbone RPoP Routers Access Rout

E

G

O

E1

E1b

E1a

G1b

G1a

N2a

N2b

N2c N2d N2e

N2f O0a

O1a

O1b

O1c

O1c

O1

O2a

O2

O2b O2c O2d

O2e

O2f

O2g

F2

F2g F2f

F0a F2d

F2e

D1a D1

D1b

D1c D1d

D0a

E2

Access Rout

Directly CFast Typical 

P E1c

E2b E2a E2c

G1bG1c

G1e

G1d

G2a G2b G2c

G2e

G1f G2d

G2f

P1 P2

P1a

P1b

P1c

P1d

P2a

P2b P2c P2d

P2e

P2f

P2g G1 G2

Routes are shortest-path based on traffic engineering goals
Traffic Scenario(s)
18 Macroscopic Response Variab

Averaged over each of three time periods (3 x 18 = 54 res

Network-wide 

Flow State

y1 Average number of sources connectingy2 Average number of sources sendingy3 Proportion of sending flows in initial slow-start

y4 Proportion of sending flows in normal congestion avoiy5 Proportion of sending flows in alternate congestion av

Network-wide

y6 Retransmission Rate

7 A C ti Wi d i Congestion

y

Average 

Congestion Wi

n

dow size

y8 Aggregate Connection Failures

Network

Dela

y

y9 Average Round-Trip Timey10 Average Queuing Delay Average Queuing Delay

Network

Throughput

y11 Average number of flows completed per secondy12 Average number of flows output per second

Throughput on

y13

Avera

ge throu

g

h

put on lon

g-lived flow #1 Throughput on

Long-Lived 

Flows

y g gp

g

y14 Average throughput on long-lived flow #2y15 Average throughput on long-lived flow #3

Throu

g

h

put for y16 Average throughput for flows transiting Very Fast (VF) 

g p

Flows on each 

Path Class

y17 Average throughput for flows transiting Fast (

F) Paths

y18 Average throughput for flows transiting Typical (

T) Pat
Average Throughput in each of 24 Flow 

Avera

g p py ( 

e Com

puted Se

paratel

y for TCP Flows and CTCP Flows ( 2 x 24 

File

Type

Path

Class

Connection

Speed

VF Fast

Movies

VF Normal

F FastF Normal

T FastT Normal

Service

Packs

VF Fast

VF Normal

F FastF Normal

T FastT Normal

Documents

VF Fast

VF Normal

F FastF Normal

T FastT Normal

VF FastVF Normal

F Fast Web Objects F FastF Normal

T FastT Normal 
Selected Analysis Techniq

1. Main Effects Analysis2. Two Factor Interaction Analysis3. Tabular Summary Analysis
Sample Main Effects Analysis

Larger InitialSlow Start 

Larger ThresholdFil

FasterNetwork

ShorterPropagationDelay

Files

Throughput (pps) for Movies transferred over Very Fast Paths with Fast Interfa
Another Sample Main Effects Anal

Shorter Think Time More Sources Distributed in a Peer-to-Peer PatternLarger Files

Bigger

SlowerNetwork

Topology

Y2 – Average Number of Sending Sources in Time Perio
Sample Two Factor Interaction Ana

Two Factor Interaction Plot for Y2 – Avg. Number of Sending Sources in 

(not much in the way of significant 2 factor interactions) 
Sample Tabular Summary Analys

Metric

Class Y# Name

Network User Behavior Source/Receiver P

X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13 X14 X15 

NSp PrD Buf Top FS LFS ThT UP CVF LLF SSR NSR DiS DiR CCA 

Y1

# Connecting +** +** +* ‐** +** +** 

Y2

# Active +** +** +** ‐** +** +**

Flows

Y2

# Active + + + ‐ + +

Y3

% ISS +** +** +** ‐** +** ‐** ‐** 

Y4

% NCA ‐** ‐** ‐** +** ‐** +** +** ‐*

Y5

% ACA +** +** +** ‐** ‐** +** 

Y6 Retrans. Rate ‐** ‐** ‐** +** ‐** +** +** 

Congestion Y7 cwn

d Size +*

Y8

# conn. fails ‐** ‐** ‐** +** +** +** 

Delay

Y9 SRT

T ‐** +** +** +* +*

Y10 Queue Delay ‐** +** +** ‐* +** +**

Aggregate Aggregate Y11 Flows/sec +** * +**   +**  +**

TP

Y11 Flows/sec +** ‐* +** ‐** ‐** +** +**Y12 Packets/sec +** +** +** +** ‐** ‐** +**

Long‐Lived

Flow TP

Y13 LLF 1 +** +* +* +** ‐**

Y14 LLF 2 +* +** +** ‐**Y15 LLF 3 +** +** ‐* ‐*

Other

Flow TP

Y16 VF Paths +** ‐** ‐** +* +** ‐** ‐* 

Y17 F Paths +** ‐** +* +** +** ‐** +** ‐** Y18

N Paths +** ‐** ‐** +** ‐** ‐** 

Significant Influence of each Factor on each Macroscopic Response in 
Another Sample Tabular Summary An

Network User Behavior Source/Receiver Pro

File

Type

Path

Class

Connection

Speed

X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13 X14 X15 

NSp PrD Buf Top FS LFS ThT UP CVF LLF SSR NSR DiS DiR CCA 

Movies

VF Fast +** ‐** +** VF Normal +** ‐** +** ‐* 

F Fast +** +** ‐** ‐** +** ‐* +** ‐* Movies F Normal +** +** +** ‐** +** ‐** +** ‐*

T Fast +** +** ‐** +** ‐** ‐**T Normal +** +** ‐** +** ‐** ‐**

VF Fast +** ‐** +** VF Normal ‐** ‐* +** 

Service

Packs

F Fast +** ‐** +** +** +* +** ‐* 

F Normal +** ‐** +** +** +** ‐** +** ‐* 

T Fast +** +** ‐** +** ‐* ‐**T Normal +** +** ‐** +** ‐* ‐**

VF Fast ‐** +* 

l ** ** 

Documents

VF Normal ‐** +** 

F Fast +** ‐** +** +* +* ‐* +** F Normal +** ‐** +** +** +** ‐* +** 

T Fast +** ‐** +** +** ‐** ‐** T Normal +** ‐** +** +** ‐** ‐** 

VF Fast ** +** +** 

Web

Objects

VF Fast ‐** +** +** VF Normal ‐** +* +** 

F Fast +** ‐** ‐* +** +** +* +** F Normal +** ‐** ‐* +** +** +** +** 

T Fast +** ‐** +** ‐** ‐** T Normal +** ‐** +** ‐** ‐** 

Significant Influence of Each of 20 Factors on Throughput for Each of 2

when using CTCP 
Relative Importance of Relative Importance of

MesoNet Parameters
Summary of Influence of Each Factor on All R

Protocol

T‐test

Statistic

Network User Behavior Source/Receiver Protoc

X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13 X14 X15 X16 

NSp PrD Buf Top FS LFS ThT UP CVF LLF SSR NSR DiS DiR CCA ICW 

Time

P id #1

>0.99 17 9 10 8 8 0 11 0 0 3 0 12 11 2 1 7 

>0.95<0.99 1 1 3 2 2 0 2 0 0 0 0 3 2 1 0 2 

Period #1 Total 18 10 13 10 10 0 13 0 0 3 0 15 13 3 1 9 

Time

Period #2

>0.99 16 9 9 6 7 0 10 0 6 2 0 13 11 1 1 5 

>0.95<0.99 2 1 1 2 0 0 2 0 0 1 0 1 1 2 1 1 

Total 18 10 10 8 7 0 12 0 6 3 0 14 12 3 2 6 

>0 99 17 9 11 6 9 0 12 0 4 3 0 12 11 2 1 5 

Time

Period #3

>0.99 17 9 11 6 9 0 12 0 4 3 0 12 11 2 1 5 

>0.95<0.99 1 2 0 3 1 0 1 0 1 0 0 3 2 0 0 0 

Total 18 11 11 9 10 0 13 0 5 3 0 15 13 2 1 5 

TCP

>0.99 19 16 12 8 11 0 10 0 1 0 0 4 16 0 0 8 

>0.95<0.99 0 2 3 3 5 0 4 0 0 0 0 2 0 0 0 0 

Total 19 18 15 11 16 0 14 0 1 0 0 6 16 0 0 8 

CTCP

>0.99 19 18 10 9 15 0 13 0 1 0 0 8 16 0 0 7 

>0.95<0.99 0 0 2 3 1 0 3 0 0 0 0 6 0 4 0 1 

Total 19 18 12 12 16 0 16 0 1 0 0 14 16

4 0 8 

Total

>0.99 88 61 52 37 50 0 56 0 12 8 0 49 65 5 3 32 

Total >0 95 . <0 99 . 4 6 9 13 9 0 12 0 1 1 0 15 5 7 1 4 

Total 92 67 61 50 59 0 68 0 13 9 0 64 70 12

4 36 

% of responses influenced 60% 49% 58% 67% 63% 69% 90% 66% 13% 12% 4% 35

9%

Significant Influence of Each of 20 Factors on Each of 18 Macroscop
What main factors drives MesoNet Res

• Ca

p y( p ) acit

y

(network s

peed

)

• Demand (number, distribution and activity of sou• Physics (propagation delay)• Buffer sizing
Conclusions

• 2-Level-per-Factor Orthogonal Fractional Factor

experimental designs can reveal significant inforabout mesoscopic simulation models about mesoscopic simulation models

• MesoNet simulation app y ears to be driven by the 

key factors that influence behavior in real netwo

• Appears feasible to compare proposed Internet Appears feasible to compare proposed Internet

congestion control algorithms while varying only MesoNet parameters
Future Work

• Apply insights from MesoNet sensitivity analysis 

compare proposed Internet congestion control algorithms [f t t ti future presentation]

JOINT WORK BETWEEN CxS and CNS Prog

• Develop a reduced scale simulation model for c

computing IaaS (infrastructure-as-a-service) [studliterature code and deployments literature, code and deployments]

• Conduct sensitivity analysis of IaaS model• Compare propose IaaS resource allocation algo

[studying literature and code]