How to model a TCP/IP network 

using only 20 parameters

Kevin Mills, NIST

CxS Study Group – November 17, 2

visual hash of MesoNet source code from http://www.wordle.net/
Outline

• Goal – Problem – Solution• Scale Reduction: Theory and Practice• Overview of the 20 MesoNet Parameter

• Parameter Explanations in 5 Categories

– Network (4 parameters)– Sources & Receivers (4 parameters)– User Behavior (6 parameters)– Protocols (3 parameters)– Simulation & Measurement Control (3 paramete

• Discuss Simulation Resource Requirem• Future Work
Goal – Problem – Solution

• Goal – compare proposed Internet congestion c

algorithms under a wide range of controlled, repconditions

• Problem – real network not controllable and rep

test beds currently too small; most network simu

models have large search space and require infmemory and processing resources for large, fasnetworks; more tractable fluid-flow simulators arcurrently inaccurate

• Solution – design a reduced scale network sim

model (MesoNet) that is easy to configure and t

to compute Joint work with Jian Yuan and Edward Schwartz
The Function ☺ of a Simulation Mo

y1, ..., ym = f( x1|[1,...,k], ..., xn|[1,...

Response State‐Space Stimulus State‐Space
Theory – Scale Reduction in Two P

k

n

k(n

‐

r1)

k(n

‐

r

1

‐

r2)

2(n

‐

r

1

‐

r2)

2

2

‐level

experiment

design

model

restriction

factor

clustering

OFF (orthogonal fract

experiment d

SCIENTIFI

DOMAIN

EXPERTIS

STATISTICAL

EXPERIMENT

DESIGN

EXPERTISE

k

n

k(n

‐

r1)

k(n

‐

r

1

‐

r2)

2(n

‐

r

1

‐

r2)

2

2

‐level

experiment

design

model

restriction

factor

clustering

OFF (orthogonal fract

experiment d

k

n

k(n

‐

r1)

k(n

‐

r

1

‐

r2)

2(n

‐

r

1

‐

r2)

2

2

‐level

experiment

design

model

restriction

factor

clustering

OFF (orthogonal fract

experiment d

SCIENTIFI

DOMAIN

EXPERTIS

STATISTICAL

EXPERIMENT

DESIGN

EXPERTISE

Today’s Talk

A Future Talk
Practice – Scale Reduction in Two P

Today

A Futu

Kevin Mills

Jim Filliben

(232

)1000

220

2

8

k

=

2

r

1

= 944

O(109633

) [ 1080

= atoms in visible univ

(232

)56 O(10539

)

(232

)20

r

2

= 36

O(10192

)

O(1

r

3

= 12 

DomainAnalyst

Statistician

(232

)1000

220

2

8

k

=

2

r

1

= 944

O(109633

) [ 1080

= atoms in visible univ

(232

)56 O(10539

)

(232

)20

r

2

= 36

O(10192

)

O(1

r

3

= 12 

(232

)1000

220

2

8

k

=

2

r

1

= 936

O(109633

) [ 1080

= atoms in visible univ

(232

)64 O(10616

)

(232

)20

r

2

= 44

O(10192

)

O(1

r

3

= 12 

DomainAnalyst

Statistician
MesoNet – a TCP/IP network mode

using only 20 parameter

Network ParameterUser BehaviorSources & ReceiverProtocolsSimulation & Measu

x1 Network Speedx2 Propagation Delayx3 Buffer Provisioningx4 Topologyx5 Source & Receiver Interface Speeds

x6 Number of Sources & Receivers

x7 Distribution of Sources

x8 Distribution of Receivers

x9 Web browsing File Sizes

x10 Larger download Proportion & File Sizes

x11 Think Time

x12 User Patience

x13 Temporal Congestion on Very Fast paths

x14 Long‐Lived Flows

x15 Assignment of Congestion‐Control Algorithm

x16 Initial Congestion Window

x17 Initial Slow‐Start Theshold

x18 Measurement Interval Size

x19 Simulation Duration

x20 Startup Pattern
Network Parameters
Parameter X4 is the Topology = 

Routers + Links + Routes + Propagation D

A

C

D

E

G

F

J

K

M

P

B

L

N

O

A1

A1b

A1c

A1a

A2a A2b A2c

C1

C1b

C1c

C1a

C2

C2b

C2c

C2a

H2bH2c

H2a

H1

H1c H1b

H1a

E1

E1b

E1c

E1a

E2b E2a E2c

H1d

H1eH1f

H2f

H2e

H2d

G1bG1c

G1a

G1e

G1d

G2a G2b G2c

G2e

G1f G2d

G2f

P1 P2

P1a

P1b

P1c

P1d

P2a

P2b P2c P2d

P2e

P2f

P2g

K1

K2

K0a

K1a

K1b

K1c K1d

K2a K2b

K2c

K2d

L2

L1

L1a L1b

L1c

L1d

L2L2

L2

L2d

L0a

L0b

J1

J2

J1a

J1b J1c J1d

J1e

J1f

J2a

J2b

J2c

J2d

J2e J2f

N1a

N1b

N1c

N1d

N1

N1f N2

N2a

N2b

N2c N2d N2e

N2f

M2b

M2c

M2d M2e M2f

M2

M2g

M1a M2a

M1b M1

M1c

M1dM1e M1f M1g

O0a

O1a

O1b

O1c

O1c

O1

O2a

O2

O2b O2c O2d

O2e

O2f

O2g

I2

I2a

I1

I I0a

I1a

I1b I1c I1d

I2g

I2f

I2e

I2d I2b I2c

F2

F2a

F1

F1a

F1b F1c F1d

F2g F2f

F2d

F2c

F2b

F0a

F2e

D2

D2a

D1a D1

D1b

D1c D1d

D2g

D2f

D2d D2b D2c

D2e

D0a

B0a

B1a B1

B1b

B1c B1d

B2

B2a

B2bB2c B2d B2e

B2f

B2g

H

G1 G2

E2

H2

A2

N1

3 Router Tiers: Backbone – Point of Presence (PoP) – Acc3 Access Router Classes: Typical

– Fast

– Directly Conne

1 ingress/egress path from access routers to backbone rou
Link Characteristics – Propagation 

and Cost Metric

A

C

D

E

G

F

J

K

M

P

B

L

N

O

A1

A1b

A1c

A1a

A2a A2b A2c

C1

C1b

C1c

C1a

C2

C2b

C2c

C2a

H2bH2c

H2a

H1

H1c H1b

H1a

E1

E1b

E1c

E1a

E2b E2a E2c

H1d

H1eH1f

H2f

H2e

H2d

G1bG1c

G1a

G1e

G1d

G2a G2b G2c

G2e

G1f G2d

G2f

P1 P2

P1a

P1b

P1c

P1d

P2a

P2b P2c P2d

P2e

P2f

P2g

K1

K2

K0a

K1a

K1b

K1c K1d

K2a K2b

K2c

K2d

L2

L1

L1a L1b

L1c

L1d

L2aL2b

L2c

L2d

L0a

L0b

J1

J2

J1a

J1b J1c J1d

J1e

J1f

J2a

J2b

J2c

J2d

J2e J2f

N1a

N1b

N1c

N1d

N1e

N1f N2

N2a

N2b

N2c N2d N2e

N2f

M2b

M2c

M2d M2e M2f

M2

M2g

M1a M2a

M1b M1

M1c

M1dM1e M1f M1g

O0a

O1a

O1b

O1c

O1c

O1

O2a

O2

O2b O2c O2d

O2e

O2f

O2g

I2

I2a

I1

I I0a

I1a

I1b I1c I1d

I2g

I2f

I2e

I2d I2b I2c

F2

F2a

F1

F1a

F1b F1c F1d

F2g F2f

F2d

F2c

F2b

F0a

F2e

D2

D2a

D1a D1

D1b

D1c D1d

D2g

D2f

D2d D2b D2c

D2e

D0a

B0a

B1a B1

B1b

B1c B1d

B2

B2a

B2bB2c B2d B2e

B2f

B2g

H

G1 G2

E2

H2

A2

N1

• Packets incur propagatio

when transiting a link

• Cost metric used to com

from source backbone ro

destination backbone rou

Link # Link Endpoints Delay (ms) Cost Metric

1 A‐B 21 502 B‐C 25 10

3 B‐D 8 504 B‐L 75 2235 C‐H 12 1006 D‐E 10 107 D‐F 33 1088 E‐G 33 100

9 F‐G 7 10

10 F‐H 12 50

11 F‐I 22 55

12 G‐O 23 10413 G‐P 19 11014 I‐H 14 10

15 I‐J 8 50

16 I‐K 22 14717 J‐L 20 6018 K‐L 7 5019 L‐M 12 50

20 L‐N 6 3921 L‐O 14 10

22 M‐O 6 10

23 N‐O 8 10

24 O‐P 14 10
Dijkstra’s SPF Backbone Routes Co

from Cost Metrics

A

C

D

E

G

F

J

K

M

P

B

L

N

O

A1

A1b

A1c

A1a

A2a A2b A2c

C1

C1b

C1c

C1a

C2

C2b

C2c

C2a

H2bH2c

H2a

H1

H1c H1b

H1a

E1

E1b

E1c

E1a

E2b E2a E2c

H1d

H1eH1f

H2f

H2e

H2d

G1bG1c

G1a

G1e

G1d

G2a G2b G2c

G2e

G1f G2d

G2f

P1 P2

P1a

P1b

P1c

P1d

P2a

P2b P2c P2d

P2e

P2f

P2g

K1

K2

K0a

K1a

K1b

K1c K1d

K2a K2b

K2c

K2d

L2

L1

L1a L1b

L1c

L1d

L2aL2b

L2c

L2d

L0a

L0b

J1

J2

J1a

J1b J1c J1d

J1e

J1f

J2a

J2b

J2c

J2d

J2e J2f

N1a

N1b

N1c

N1d

N1e

N1f N2

N2a

N2b

N2c N2d N2e

N2f

M2b

M2c

M2d M2e M2f

M2

M2g

M1a M2a

M1b M1

M1c

M1dM1e M1f M1g

O0a

O1a

O1b

O1c

O1c

O1

O2a

O2

O2b O2c O2d

O2e

O2f

O2g

I2

I2a

I1

I I0a

I1a

I1b I1c I1d

I2g

I2f

I2e

I2d I2b I2c

F2

F2a

F1

F1a

F1b F1c F1d

F2g F2f

F2d

F2c

F2b

F0a

F2e

D2

D2a

D1a D1

D1b

D1c D1d

D2g

D2f

D2d D2b D2c

D2e

D0a

B0a

B1a B1

B1b

B1c B1d

B2

B2a

B2bB2c B2d B2e

B2f

B2g

H

G1 G2

E2

H2

A2

N1

PATHS FOR NODE A PATHS FOR NODE B PATHS FOR NODE C PATHS FOR NODE D

A:B

@ A‐>B $ 50 B:A

@ B‐>A $ 50 C:A

@ C‐>B‐>A $ 60 D:A

@ D‐>B‐>A $ 100

A:C

@ A‐>B‐>C $ 60 B:C

@ B‐>C $ 10 C:B

@ C‐>B $ 10 D:B

@ D‐>B $ 50

A:D

@ A‐>B‐>D $ 100 B:D

@ B‐>D $ 50 C:D

@ C‐>B‐>D $ 60 D:C

@ D‐>B‐>C $ 60

A:E

@ A‐>B‐>D‐>E $ 110 B:E

@ B‐>D‐>E $ 60 C:E

@ C‐>B‐>D‐>E $ 70 D:E

@ D‐>E $ 10

A:F

@ A‐>B‐>D‐>F $ 208 B:F

@ B‐>D‐>F $ 158 C:F

@ C‐>H‐>F $ 150 D:F

@ D‐>F $ 108

A:G

@ A‐>B‐>D‐>E‐>G $ 210 B:G

@ B‐>D‐>E‐>G $ 160 C:G

@ C‐>H‐>F‐>G $ 160 D:G

@ D‐>E‐>G $ 110

A:H

@ A‐>B‐>C‐>H $ 160 B:H

@ B‐>C‐>H $ 110 C:H

@ C‐>H $ 100 D:H

@ D‐>F‐>H $ 158

A:I

@ A ‐> B ‐> C ‐> H ‐> I $ 170 B:I

@ B ‐> C ‐> H ‐> I $ 120 C:I

@ C ‐> H ‐> I $ 110 D:I

@ D ‐> F ‐> I 

A:J

@ A‐>B $220 B:J

@ B‐>C $170 C:J

@ C‐>H $160 D:J

@ D‐>F $213

A:K

@ A‐>B‐>C‐>H‐>I‐>K $ 317 B:K

@ B‐>C‐>H‐>I‐>K $ 267 C:K

@ C‐>H‐>I‐>K $ 257 D:K

@ D‐>F‐>I‐>K $ 310

A:L

@ A‐>B‐>L $ 273 B:L

@ B‐>L $223 C:L

@ C‐>H $220 D:L

@ D‐>E‐>G‐>O‐>N‐>L 

A:M

@ A‐>B‐>L‐>M $ 323 B:M

@ B‐>L‐>M $ 273 C:M

@ C‐>H‐>I‐>J‐>L‐>M $ 270 D:M

@ D‐>E‐>G‐>O‐>M $ 

A:N

@ A‐>B‐>L‐>N $ 312 B:N

@ B‐>L‐>N $ 262 C:N

@ C‐>H‐>I‐>J‐>L‐>N $ 259 D:N

@ D‐>E‐>G‐>O‐>N $ 2

A:O

@ A‐>B‐>D‐>E‐>G‐>O $ 314 B:O

@ B‐>D‐>E‐>G‐>O $ 264 C:O

@ C‐>H‐>F‐>G‐>O $ 264 D:O

@ D‐>E‐>G‐>O $ 214

A:P

@ A‐>B‐>D‐>E‐>G‐>P $ 320 B:P

@ B‐>D‐>E‐>G‐>P $ 270 C:P

@ C‐>H‐>F‐>G‐>P $ 270 D:P

@ D‐>E‐>G‐>P $ 220

PATHS FOR NODE E PATHS FOR NODE F PATHS FOR NODE G PATHS FOR NODE H

E:A

@ E‐>D‐>B‐>A $ 110 F:A

@ F‐>D‐>B‐>A $ 208 G:A

@ G‐>E‐>D‐>B‐>A $ 210 H:A

@ H‐>C‐>B‐>A $ 160

E:B

@ E‐>D‐>B $ 60 F:B

@ F‐>D‐>B $ 158 G:B

@ G‐>E‐>D‐>B $ 160 H:B

@ H‐>C‐>B $ 110

E:C

@ E‐>D‐>B‐>C $ 70 F:C

@ F‐>H‐>C $ 150 G:C

@ G‐>F‐>H‐>C $ 160 H:C

@ H‐>C $ 100

E:D

@ E‐>D $ 10 F:D

@ F‐>D $ 108 G:D

@ G‐>E‐>D $ 110 H:D

@ H‐>F‐>D $ 158

E:F

@ E‐>G‐>F $ 110 F:E

@ F‐>G‐>E $ 110 G:E

@ G‐>E $ 100 H:E

@ H‐>F‐>G‐>E $ 160

E:G

@ E‐>G $ 100 F:G

@ F‐>G $ 10 G:F

@ G‐>F $ 10 H:F

@ H‐>F $ 50

E:H

@ E‐>G‐>F‐>H $ 160 F:H

@ F‐>H $ 50 G:H

@ G‐>F‐>H $ 60 H:G

@ H‐>F‐>G $ 60

E:I

@ E ‐> G ‐> F ‐> I $ 165 F:I

@ F ‐> I $ 55 G:I

@ G ‐> F ‐> I $ 65 H:I

@ H ‐> I $ 10

E:J

@ E‐>G $215 F:J

@ F ‐> I ‐> J $ 105 G:J

@ G‐>F $115 H:J

@ H‐>I ‐> J 

E:K

@ E‐>G‐>O‐>N‐>L‐>K $ 303 F:K

@ F‐>I ‐> K $ 202 G:K

@ G‐>O‐>N‐>L‐>K $ 203 H:K

@ H‐> $157

E:L

@ E‐>G‐>O‐>N‐>L $ 253 F:L

@ F‐>G‐>O‐>N‐>L $ 163 G:L

@ G‐>O‐>N‐>L $ 153 H:L

@ H‐>I‐> L$ 120

E:M

@ E‐>G‐>O‐>M $ 214 F:M

@ F‐>G‐>O‐>M $ 124 G:M

@ G‐>O‐>M $ 114 H:M

@ H‐>I‐>J‐>L‐>M $ 17

E:N

@ E‐>G‐>O‐>N $ 214 F:N

@ F‐>G‐>O‐>N $ 124 G:N

@ G‐>O‐>N $ 114 H:N

@ H‐>I‐>J‐>L‐>N $ 15

E:O

@ E‐>G‐>O $ 204 F:O

@ F‐>G‐>O $ 114 G:O

@ G‐>O $ 104 H:O

@ H‐>F‐>G‐>O $ 164

E:P

@ E‐>G‐>P $ 210 F:P

@ F‐>G‐>P $ 120 G:P

@ G‐>P $ 110 H:P

@ H‐>F‐>G‐>P $ 170

240 3.63 9.53

# Backbone Paths Avg. Backbone Path Length Avg. Source-Receiver Pat

240 3.63 9.53

# Backbone Paths Avg. Backbone Path Length Avg. Source-Receiver Pat

Sample Backbone Paths for Routers A Through H
Parameter x2 scales all propagation d

Link # Link Endpoints Delay (ms) x2

= 2 x2

= 0.5

1 A‐B 21 42 10.52 B‐C 25 50 12.5

3 B‐D 8 16 44 B‐L 75 150 37.5

5 C‐H 12 24 6

6 D‐E 10 20 57 D‐F 33 66 16.58 E‐G 33 66 16.59 F‐G 7 14 3.5

10 F‐H 12 24 611 F‐I 22 44 1112 G‐O 23 46 11.513 G‐P 19 38 9.5

14 I‐H 14 28 715 I‐J 8 16 416 I‐K 22 44 1117 J‐L 20 40 10

18 K‐L 7 14 3.5

19 L‐M 12 24 620 L‐N 6 12 3

21 L‐O 14 28 7

22 M‐O 6 12 3

23 N‐O 8 16 4

24 O‐P 14 28 7
Engineering Relationship Among Router S

(MesoNet simplification – only routers have speeds

Speed Relationships

Router Class SpeedBackbone r1

x BBspeedup

PoP r1/r2Access r1/r2/r3Fast Access r1/r2/r3

x Bfast

Directly Connected Access r1/r2/r3

x Bdirect

Parameter Valuer1 x1r2 4r3 10BBspeedup 2Bdirect 10Bfast 2

Sample Parameters Values

Router Class Speed x1

= 800 x1

= 1600

Backbone r1

x BBspeedup 1600 3200

PoP r1/r2 400 800Access r1/r2/r3 40 80Fast Access r1/r2/r3

x Bfast 80 160

Directly Connected Access r1/r2/r3

x Bdirect 400 800

Parameter x1 scales all router speeds

Given a fixed r2, r3, BBspeedup, Bdirect and Bfast,
Domain View of Network Speed

(MesoNet simplification – packets have no size)

^default maximum transfer unit size for the Internet is 1500 bytes

Assuming speeds dimensioned in packets per millisecoand packets sized at 1500

^ 8-bit bytes:

Router Class x1

= 800 Speed (Gbps) x1

= 1600 Speed (G

Backbone 1600 19.2 3200 38.4PoP 400 4.8 800 9.6Access 40 0.48 80 0.96Fast Access 80 0.96 160 1.92Directly Connected Access 400 4.8 800 9.6
Buffer Size Determination

RTT

×

C

(RTT

×

C) n

Choice of 4 Buffer Provisioning Algorithms

recommended practicesuggested by researchers 

((RTT

×

C) + ((RTT

×

C) n))

2 interpolation

Buffers =

directly set

Buffers =Buffers =Buffers = <integer value>

RTT = average round-trip propagation delay of the topology

C = capacity of a given router type – determined by network speed settingn = expected number of flows transiting the router

Qfactor = buffer size scaling factorBufferSize = Buffers

x Qfactor

For selected algorithm, x3 = Qfactor scales buffer size 

For specified Qfactor, x3 = algorithm determines buffer s
Sources & Receivers
Approximate Number of Sources & Rece

baseSources = target

^ # sources under each access router



U = scaling factor for target # sources

# sources = O(baseSources

x



U

x # access routers) 

# receivers = O(baseSources

x



U

x # access routers 

x 4) 

3 O(51,000) O(204,000)1 O(17,000) O(68,000)



U # sources # receivers

baseSources = 100 # access routers = 170

^# sources and receivers becomes known exactly only when taking int

the distribution of sources and receivers under various access route

Setting x6 (= 

U) scales the number of sources and receivers in 
Access Router Classes & Source Distrib

1 – probNs - probNsf

probNsf

probNs

Probability Source Under

D-class Directly connected access routers

F-class Access routers with fast speed

N-class Access routers with typical speed

Access Router Classes

Setting x7 = (probNs, probNsf) scales the distribution of s

and adjusts the number of sources in the topology

Setting x8 = (probNr, probNrf) performs a similar function on 

Target # sources under 3 access routers: 1 in each class) = 3 x baseSources x 



U

# sources under each 

N-class router = 3 x baseSources x 



U

x probNs

# sources under each 

F-class router = 3 x baseSources x 



U

x probNsf

# sources under each 

D-class router = 3 x baseSources x 



U x (1 - probNs - probNsf

6.2

62.231.6

% sources

2,160

21,60010,980

# sources

8

40

122

# routers

D-class 270F-class 540

N-class 90

router class sources/router

Given 



U = 3, baseSources = 100, probNs = 0.1, probNsf = 0.6
Source/Receiver Distributions Influence Traffi

D-class 6.2 0.8

F-class 62.2 3.9

N-class 31.6 95.3

router class % sources % receivers

Given 



U = 3, baseSources = 100, probNs = 0.1, probNsf = 0.6, probNr = 0.8, probNr

NN flows 30.F

N flows 60.

FF flows 2.4

D

N flows 6.1

D

F flows 0.7

DD flows 0.0Flow class % flo

FF, F

N, D

N and 

D

F flows represent Web-centric traffic and NN flows represen

DD flows represent traffic exchange with large data repositories
Source & Receiver Interface Speeds 

Hfast 80 960

Hbase

8 96

Interface Speeds ppms Mbps

Assuming packet size is 1500 8-bit bytes:

Model permits two interfac

x5 specifies the probability that a source/receiver has an Hfast i

(1- x5) gives the probability that a source/receiver has an Hbase interfac

Example
User Behavior
Source Behavior^

^Note: this simplified diagram omits a flow connection phase that occurs befo

and also the potential for the connection phase to fail – after which source en

x11 specifies average Think Time x12 specifies probability sou

Think Time Expired

Select Receiver &

File Size (Pareto Distribution)

Finished

Select Think Time

(Exponential Distribution)

Select Think Time

(Exponential Distribution)

Too Long

(reactive sources only)

Too Slow (reactive sources only)

Select Think Time

(Exponential Distribution)

Select R

File Size (Pa

Select Think Time

(Exponential 

Distribution)
File Size Selection

x9 specifies 

on

 shape of Pareto distributionon average size (packets)

Size of Web Objects

Mx moviesSx software downloadsFx documents

Larger File Size Multipliers

Mp moviesSp software downloadsFp documents

Larger File Probabilities

Probability of Web Object 

(1 – Fp

– Sp

– Mp) 

Given a fixed 

 Given fixed Fx, Sx, Mx

x10 specifies (Fp, Sp, Mp) 

Characterize Web Objects (

on, 

) 

Characterize Larger Files [(Fx, Fp), (Sx, Sp), (Mx, Mp)] 
Specifying Intense Spatiotemporal T

x13 specifies (Jon, Joff)

Joff fraction of simulated time after which jumbo file transfers end

Jon fraction of simulated time after which jumbo file transfers beginJx size multiplier for jumbo files

Jumbo File Characteristics

During selected time periods, DD flows can transfer jumbo files

Given a fixed Jx

Characterize Jumbo Files (Jx, Jon, Joff)
Specifying Long-Lived Flows

x14 specifies (n, LLon, SourceType, SourceLocation, Receiver

Behavior of each long-lived flow is monitored in detail

ReceiverLocation[n] access router under which each long-lived receiver is loca

SourceLocation[n] access router under which each long-lived source is locat

SourceType[n] congestion control algorithm used by each long-lived flow

LLon[n] fraction of simulated time after which each long-lived flow n number of long-lived flows

Characteristics of Long-Lived Flows

7 BIC6 HTCP5 FAST4 Scalable TCP3 CTCP2 HSTCP1 TCP

0 select from probability distribution (see x15)

ID Source Type
Protocols
Assignment of Congestion Control Alg

Probabilistically assign each source (except for long-lived flows) a congestion 

BIC prBICTCP

HTCP prHTCP

FAST prFAST

Scalable TCP prSCALABLE

CTCP prCTCP

HSTCP prHSTCP

TCP prTCP

Probability

Congestion 

Control 

Algorithm

x15 specifies (prTCP, prHSTCP, prCTCP, prSCALABLE, prFAST, prH
Initial Congestion Window & Slow Start T

Congestion window (cwnd) determines how many packets may be sent by a so

At Initial Slow Start Threshold (sst) cwnd increase switches from exponen

0

20406080

100120140160

0 20 40 60 80 100

Time

CWND

Initial cwnd

Initial sst

exponentialincrease

linearincrease

0

20406080

100120140160

0 20 40 60 80 100

Time

CWND

Initial cwnd

Initial sst

0

20406080

100120140160

0 20 40 60 80 100

Time

CWND

Initial cwnd

Initial sst

0

20406080

100120140160

0 20 40 60 80 100

Time

CWND

Initial cwnd

Initial sst

0

20406080

100120140160

0 20 40 60 80 100

Time

CWND

Initial cwnd

Initial sst

0

20406080

100120140160

0 20 40 60 80 100

Time

CWND

0

20406080

100120140160

0 20 40 60 80 100

Time

CWND

Initial cwnd

Initial sst

exponentialincrease

linearincrease

Initial cwnd determines how many packets may be sent prior to first ack

x16 specifies initial cwnd x17 specifies initial s
Simulation & Measureme

Control
Measurement Interval Size & Simulation 

Measurements taken as time series sampled periodically: every measurem

M 

x MI simulation duration

MI number of measuremeM measurement interval 

Controlling Measurement & Sim

3000 3500 4000 4500 5000 5500 6000 6500 7000 7500

0

100020003000400050006000700080009000

Evolution of Flows Condition 5 - TCP

Time

Active Flows

Sending Flows

Normal Congestion ControlConnecting Flows

Flows in Initial Slow Start

3000 3500 4000 4500 5000 5500 6000 6500 7000 7500

0

100020003000400050006000700080009000

Evolution of Flows Condition 5 - TCP

Time

Active Flows

Sending Flows

Normal Congestion ControlConnecting Flows

Flows in Initial Slow Start

x18 specifies 

M x19 specifies MI
Source Startup Pattern

Sources enter the sending state at simulation startup according to a spec

prREST probability a source starts sending after random time: mean think t

prONthird probability a source starts sending after random time: mean 66% 

x 

prONsecond probability a source starts sending after random time: mean 33% 

x 

prON probability a source starts in the sending state

Initial Think Times for Sources

x20 specifies (prON, prONsecond, 

prREST = (1 – prON

– prONsecond

– prONthird)

0

100020003000400050006000

0 50 100 150 200 250

Time

Sending Flows prREST 0.50 prONthird 0.17 prONsecond 0.08

prON 0.25
Two Sample Experiments

BIC, CTCP, FAST, 

FAST-AT, HSTCP, 

HTCP, Scalable

BIC, CTCP, FAST, 

FAST-AT, HSTCP, 

HTCP, Scalable

Alternate CongestionControl Algorithms

60 min. – 96-98% Web objects; 2-4% document transfers; smaller number of service-pack and movie downloads

30:70 & 70:30

232/2

1x10-4 to 1x10-2

19.2 & 38.4

17,455 & 26,085

Sec. 8 High SST

60 min. – 96-98% Web objects; 2-4% document transfers; smaller number of service-pack and movie downloads

Scenario

30:70 & 70:30

Ratio (%) of Sources using Alternate Congestion-Control to Standard TCP Congestion-Control

232/2 Initial Slow-Start Threshold

2x10-9 to 2x10 Packet Loss Rate -2

Backbone Speed (Gbps) 192 & 384Network Size (sources) 174,600 & 261,792Characteristic Sec. 9 High SST

BIC, CTCP, FAST, 

FAST-AT, HSTCP, 

HTCP, Scalable

BIC, CTCP, FAST, 

FAST-AT, HSTCP, 

HTCP, Scalable

Alternate CongestionControl Algorithms

60 min. – 96-98% Web objects; 2-4% document transfers; smaller number of service-pack and movie downloads

30:70 & 70:30

232/2

1x10-4 to 1x10-2

19.2 & 38.4

17,455 & 26,085

Sec. 8 High SST

60 min. – 96-98% Web objects; 2-4% document transfers; smaller number of service-pack and movie downloads

Scenario

30:70 & 70:30

Ratio (%) of Sources using Alternate Congestion-Control to Standard TCP Congestion-Control

232/2 Initial Slow-Start Threshold

2x10-9 to 2x10 Packet Loss Rate -2

Backbone Speed (Gbps) 192 & 384Network Size (sources) 174,600 & 261,792Characteristic Sec. 9 High SST

Joint work with Jim Filliben, Dong Yeon Cho and Edward Schwart
7 Congestion Control Algorithms 

x 32 Co

Factor-> x1 x2 x3 x4 x5 x6 x7 

Condition -- -- -- -- -- -- -- 

1 8000 1 0.5 5000 100 0.04/0.004/0.0004 0.7 2 16000 1 0.5 5000 100 0.04/0.004/0.0004 0.3 3 8000 2 0.5 5000 100 0.02/0.002/0.0002 0.7 4 16000 2 0.5 5000 100 0.02/0.002/0.0002 0.3 

5 8000 1 1 5000 100 0.02/0.002/0.0002 0.3 6 16000 1 1 5000 100 0.02/0.002/0.0002 0.7 7 8000 2 1 5000 100 0.04/0.004/0.0004 0.3 

8 16000 2 1 5000 100 0.04/0.004/0.0004 0.7 9 8000 1 0.5 7500 100 0.02/0.002/0.0002 0.3 

10 16000 1 0.5 7500 100 0.02/0.002/0.0002 0.7 11 8000 2 0.5 7500 100 0.04/0.004/0.0004 0.3 12 16000 2 0.5 7500 100 0.04/0.004/0.0004 0.7 13 8000 1 1 7500 100 0.04/0.004/0.0004 0.7 14 16000 1 1 7500 100 0.04/0.004/0.0004 0.3 15 8000 2 1 7500 100 0.02/0.002/0.0002 0.7 16 16000 2 1 7500 100 0.02/0.002/0.0002 0.3 17 8000 1 0.5 5000 150 0.02/0.002/0.0002 0.3 18 16000 1 0.5 5000 150 0.02/0.002/0.0002 0.7 19 8000 2 0.5 5000 150 0.04/0.004/0.0004 0.3 20 16000 2 0.5 5000 150 0.04/0.004/0.0004 0.7 21 8000 1 1 5000 150 0.04/0.004/0.0004 0.7 22 16000 1 1 5000 150 0.04/0.004/0.0004 0.3 23 8000 2 1 5000 150 0.02/0.002/0.0002 0.7 24 16000 2 1 5000 150 0.02/0.002/0.0002 0.3 25 8000 1 0.5 7500 150 0.04/0.004/0.0004 0.7 26 16000 1 0.5 7500 150 0.04/0.004/0.0004 0.3 27 8000 2 0.5 7500 150 0.02/0.002/0.0002 0.7 28 16000 2 0.5 7500 150 0.02/0.002/0.0002 0.3 29 8000 1 1 7500 150 0.02/0.002/0.0002 0.3 30 16000 1 1 7500 150 0.02/0.002/0.0002 0.7 31 8000 2 1 7500 150 0.04/0.004/0.0004 0.3 32 16000 2 1 7500 150 0.04/0.004/0.0004 0.7 

29-4 Orthogonal Fractional Factorial De

(7 

x 32 =) 224 Runs for Small, Slow Network and (7 

x 32 =) 224 Runs for Larg

Note: factor designators x1 to x9 not the same as used in this talk
MesoNet Tractable?

Large, Fast Network 

Initial Slow-Start Th

Small, Slow Network with High 

Initial Slow-Start Threshold

2,568,480,122

17,390,781

7,258,056

11,466,429

Flows Completed

764,739,915,978

5,048,119,1662,138,998,7643,414,017,482

Data Packets Sent

Total all Runs 26,055,028,851 7,47

Max. Per Condition 175,947,632 5Min. Per Condition 72,944,797 2

Avg. Per Condition 116,317,093 3

Statistic Flows Completed Data P

Large, Fast Network 

Initial Slow-Start Th

Small, Slow Network with High 

Initial Slow-Start Threshold

2,568,480,122

17,390,781

7,258,056

11,466,429

Flows Completed

764,739,915,978

5,048,119,1662,138,998,7643,414,017,482

Data Packets Sent

Total all Runs 26,055,028,851 7,47

Max. Per Condition 175,947,632 5Min. Per Condition 72,944,797 2

Avg. Per Condition 116,317,093 3

Statistic Flows Completed Data P

43.97 739.04 Max. CPU hours (one run)

196.56 2,392.41 Avg. Memory Usage (Mbytes)

12.58 203.04 Min. CPU hours (one run)

26.15 421.23 Avg. CPU hours (per run)

5,857.18 94,355.28 CPU hours (224 Runs)

Large, Fast Network 

with High Initial SlowStart Threshold

Small, Slow Network 

with High Initial SlowStart Threshold

43.97 739.04 Max. CPU hours (one run)

196.56 2,392.41 Avg. Memory Usage (Mbytes)

12.58 203.04 Min. CPU hours (one run)

26.15 421.23 Avg. CPU hours (per run)

5,857.18 94,355.28 CPU hours (224 Runs)

Large, Fast Network 

with High Initial SlowStart Threshold

Small, Slow Network 

with High Initial SlowStart Threshold

48 processors available to run simulations

competed in 1 week competed in 3 months35 weeks of CPU time 131 months of CPU time
Future Work

• Conduct 20-factor sensitivity analysis of 

MesoNet [data analysis underway] 

• Use MesoNet to compare proposed Inte

congestion control algorithms [report drafte

• Develop a reduced scale simulation mo

for cloud computing IaaS (infrastructurea-service) [studying literature, code and deploymen

• Conduct sensitivity analysis of IaaS mod• Compare proposed IaaS resource 

allocation algorithms [studying literature and co

JOINT WORK BETWEEN CxS and CNS Progra

Joint work with Chris Dabrowski, Jim Filliben and Peter Mell
Backup Slides
Dijkstra’s SPF Backbone Routes Co

from Cost Metrics (2 of 2) 

A

C

D

E

G

F

J

K

M

P

B

L

N

O

A1

A1b

A1c

A1a

A2a A2b A2c

C1

C1b

C1c

C1a

C2

C2b

C2c

C2a

H2bH2c

H2a

H1

H1c H1b

H1a

E1

E1b

E1c

E1a

E2b E2a E2c

H1d

H1eH1f

H2f

H2e

H2d

G1bG1c

G1a

G1e

G1d

G2a G2b G2c

G2e

G1f G2d

G2f

P1 P2

P1a

P1b

P1c

P1d

P2a

P2b P2c P2d

P2e

P2f

P2g

K1

K2

K0a

K1a

K1b

K1c K1d

K2a K2b

K2c

K2d

L2

L1

L1a L1b

L1c

L1d

L2aL2b

L2c

L2d

L0a

L0b

J1

J2

J1a

J1b J1c J1d

J1e

J1f

J2a

J2b

J2c

J2d

J2e J2f

N1a

N1b

N1c

N1d

N1e

N1f N2

N2a

N2b

N2c N2d N2e

N2f

M2b

M2c

M2d M2e M2f

M2

M2g

M1a M2a

M1b M1

M1c

M1dM1e M1f M1g

O0a

O1a

O1b

O1c

O1c

O1

O2a

O2

O2b O2c O2d

O2e

O2f

O2g

I2

I2a

I1

I I0a

I1a

I1b I1c I1d

I2g

I2f

I2e

I2d I2b I2c

F2

F2a

F1

F1a

F1b F1c F1d

F2g F2f

F2d

F2c

F2b

F0a

F2e

D2

D2a

D1a D1

D1b

D1c D1d

D2g

D2f

D2d D2b D2c

D2e

D0a

B0a

B1a B1

B1b

B1c B1d

B2

B2a

B2bB2c B2d B2e

B2f

B2g

H

G1 G2

E2

H2

A2

N1

PATHS FOR NODE I PATHS FOR NODE J PATHS FOR NODE K PATHS FI :

A

@ I ‐> H ‐> C ‐> B ‐> A $ 170 J :

A

@ J ‐> I ‐> H ‐> C ‐> B ‐> A $ 220 K:A

@ K‐>I‐>H‐>C‐>B‐>A $ 317 L:A

@ L

I : B @ I ‐> H ‐> C ‐> B $ 120 J : B @ J ‐> I ‐> H ‐> C ‐> B $ 170 K:B

@ K‐>I‐>H‐>C‐>B $ 267 L:B

@ L

I : C @ I ‐> H ‐> C $ 110 J : C @ J ‐> I ‐> H ‐> C $ 160 K:C

@ K‐>I‐>H‐>C $ 257 L:C

@ L

I : D @ I ‐> F ‐> D $ 163 J : D @ J ‐> I ‐> F ‐> D $ 213 K:D

@ K‐>I‐>F‐>D $ 310 L:D

@ L

I : E @ I ‐> F ‐> G ‐> E $ 165 J : E @ J ‐> I ‐> F ‐> G ‐> E $ 215 K:E

@ K‐>L‐>N‐>O‐>G‐>E $ 303 L:E

@ L‐

I : F @ I ‐> F $ 55 J : F @ J ‐> I ‐> F $ 105 K:F

@ K‐>I ‐> F $ 202 L:F

@ L‐

I : G @ I ‐> F ‐> G $ 65 J : G @ J ‐> I ‐> F ‐> G $ 115 K:G

@ K‐>L‐>N‐>O‐>G $ 203 L:G

@ L

I : H @ I ‐> H $ 10 J : H @ J ‐> I ‐> H $ 60 K:H

@ K‐>I $157 L:H

@ L

I : J @ I ‐> J $ 50 J : I @ J ‐> I $ 50 K:I

@ K ‐> I $ 147 L:I

I : K @ I ‐> K $ 147 J : K @ J ‐> L ‐> K $ 110 K:J

@ K‐>L $110 L:J

I : L @ I ‐> J ‐> L $ 110 J : L @ J ‐> L $ 60 K:L

@ K‐>L $50 L:K

@ L

I :

M

@ I ‐> J ‐> L ‐> M $ 160 J :

M

@ J ‐> L ‐> M $ 110 K:M

@ K‐>L‐>M $ 100 L:M

@ L

I : N @ I ‐> J ‐> L ‐> N $ 149 J : N @ J ‐> L ‐> N $ 99 K:N

@ K‐>L‐>N $ 89 L:N

@ L

I :

O

@ I ‐> J ‐> L ‐> N ‐> O $ 159 J :

O

@ J ‐> L ‐> N ‐> O $ 109 K:O

@ K‐>L‐>N‐>O $ 99 L:O

@ L

I : P @ I ‐> J ‐> L ‐> N ‐> O ‐> P $ 169 J : P @ J ‐> L ‐> N ‐> O ‐> P $ 119 K:P

@ K‐>L‐>N‐>O‐>P $ 109 L:P

@ L

PATHS FOR NODE M PATHS FOR NODE N PATHS FOR NODE O PATHS FOR NODE P

M:A

@

M‐>L‐>B‐>A $ 323 N:A

@ N‐>L‐>B‐>A $ 312 O:A

@

O‐>G‐>E‐>D‐>B‐>A $ 314 P:A

@ P‐>G‐>E‐>D‐

M:B

@

M‐>L‐>B $ 273 N:B

@ N‐>L‐>B $ 262 O:B

@

O‐>G‐>E‐>D‐>B $ 264 P:B

@ P‐>G‐>E‐>D‐

M:C

@

M‐>L‐>J‐>I‐>H‐>C $ 270 N:C

@ N‐>L‐>J‐>I‐>H‐>C $ 259 O:C

@

O‐>G‐>F‐>H‐>C $ 264 P:C

@ P‐>G‐>F‐>H‐

M:D

@

M‐>O‐>G‐>E‐>D $ 224 N:D

@ N‐>O‐>G‐>E‐>D $ 224 O:D

@

O‐>G‐>E‐>D $ 214 P:D

@ P‐>G‐>E‐>D 

M:E

@

M‐>O‐>G‐>E $ 214 N:E

@ N‐>O‐>G‐>E $ 214 O:E

@

O‐>G‐>E $ 204 P:E

@ P‐>G‐>E $ 21

M:F

@

M‐>O‐>G‐>F $ 124 N:F

@ N‐>O‐>G‐>F $ 124 O:F

@

O‐>G‐>F $ 114 P:F

@ P‐>G‐>F $ 12

M:G

@

M‐>O‐>G $ 114 N:G

@ N‐>O‐>G $ 114 O:G

@

O‐>G $ 104 P:G

@ P‐>G $ 110

M:H

@

M‐>L‐>J‐>I‐>H $ 170 N:H

@ N‐>L‐>J‐>I‐>H $ 159 O:H

@

O‐>G‐>F‐>H $ 164 P:H

@ P‐>G‐>F‐>H 

M:I

@

M ‐> L ‐> J ‐> I $ 160 N:I

@ N ‐> L ‐> J ‐> I $ 149 O:I

@

O ‐> N ‐> L ‐> J ‐> I $ 159 P:I

@ P ‐> 

M:J

@

M‐>L‐>J $ 110 N:J

@ N‐>L $99 O:J

@

O‐>N‐>L‐>J $ 109 P:J

@ P‐>O‐>N‐>L‐>

M:K

@

M‐>L‐>K $ 100 N:K

@ N‐>L‐>K $ 89 O:K

@

O‐>N‐>L‐>K $ 99 P:K

@ P‐>O‐>N‐>L‐

M:L

@

M‐>L $ 50 N:L

@ N‐>L $ 39 O:L

@

O‐>N‐>L $ 49 P:L

@ P‐>O‐>N‐>L $ 

M:N

@

M‐>O‐>N $ 20 N:M

@ N‐>O‐>M $ 20 O:M

@

O‐>M $ 10 P:M

@ P‐>O‐>M $ 

M:O

@

M‐>O $ 10 N:O

@ N‐>O $ 10 O:N

@

O‐>N $ 10 P:N

@ P‐>O‐>N $ 2

M:P

@

M‐>O‐>P $ 20 N:P

@ N‐>O‐>P $ 20 O:P

@

O‐>P $ 10 P:O

@ P‐>O $ 10
Other Sample MesoNe

Topologies
Sample Dumbbell Topology from an Empirical S
Sample Simulated Topology (I) from MesoN
Sample Simulated Topology (IIc & IId) from Me

IIc – asymmetric routesIId – SPF based on propagation dela
Sample Simulated Topology (ST) from Meso

A

B

D

C

F

G

H

I

J

E

B0a

F0a

I0a

I1a

Sources B0a1-B0a5

Sources B0a6-B0a10

Sources F0a1-F0a5 Sources F0a6-F0a10

H0a

Sources H0a1-H0a5

Sources H0

Receivers I0a1-I0a5 Receivers I0a6-I0a10

Receivers I1a1-I1a5

Receivers I1a6-I1a10

Bottleneck Link

SPF based on propagation delays