Validating Simulations of Largescale Computer Networks

Kevin Mills

October 27, 2015
Problem Statement

• We pioneered methods to simulate and 

analyze macroscopic behavior in large 

networks, but...

• A key hurdle remains:

– to demonstrate convincingly that our simulations 

exhibit behaviors consistent with real networks.

• This hurdle exists generally within the 

scientific field of big network simulation...

10/27/2015 ITL Science Day 2
Background (I)

• DoE COMBINE workshop:

– identifies need for verification 

and validation (V&V) 

techniques to increase 

confidence in simulations of 

big networks

– identifies need for research to 

characterize uncertainty in big 

network simulations

– suggests recent advances in 

configurable, empirical 

network test beds might create 

an opportunity to improve the 

situation

10/27/2015 ITL Science Day 3
Background (II)

• Operational networks cannot be controlled so 

that simulation results can be duplicated there

• Parameters assigned in simulations might not 

reflect values from real networks

• Configurable network test beds could provide:

1. reproducible platforms on which to verify 

simulation results

2. foundations for making measurements that can 

form the basis for simulator parameterizations

10/27/2015 ITL Science Day 4
Background (III)

• We had already developed the MesoNet

simulator and had run experiments to study 

global patterns of congestion and user 

experience in large TCP/IP networks

• We had already installed a rudimentary inhouse network emulation facility, based on 

Emulab technology developed at the 

University of Utah

10/27/2015 ITL Science Day 5
Project Aims

1. Investigate and develop rigorous method to 

verify accuracy and characterize uncertainty 

of large-scale network simulations

2. Advance the state-of-the-art in V&V for largescale network simulations

3. Demonstrate the findings from an earlier 

simulation study hold in an empirical setting 

10/27/2015 ITL Science Day 6
Technical Approach

10/27/2015 ITL Science Day 7

1(b) Scale down MesoNet Simulations

1(c) Recruit a New Employee

1(a) Scale up & Refresh In-house Emulab

2 Replicate MesoNet Sensitivity Analysis

 on Emulab
10/27/2015 ITL Science Day 8

1(a) Progress on Emulab

Count Server Memory Disk NICs

12 860 2 GB 300 GB 6@1 Gbps

38 R610 12 GB 140 GB 6@1 Gbps

Count Server Memory Disk NICs

56 R610 12 GB 1 TB 6@1 Gbps

24 R620 32 GB 1 TB 8@1 Gbps

20 R630 32 GB 1 TB 8@1 Gbps

Previous 50-node In-house Emulab

Upgraded 100-node In-house Emulab

http://emulab2.antd.nist.gov/

• 2x number of available nodes, 

with increased memory size and 

speed, 3x disk capacity and 2.3x 

more network connections 

• 10x increase in backbone speed 

from 1 to 10 Gbps

• 2x number of switch ports

• Larger, faster control network

• Remote power cycling added

• Upgraded to most current 

version of Emulab control 

software
1(b) Progress on MesoNet

10/27/2015 ITL Science Day 9

4-Tier Topology from US ISP

Tier Node Type Count

1 Backbone 16

2 PoP 32

3 Access 170

4 Source 51,588

4 Receiver 206,352

Total All 258,158

FIT ON EMULAB 40x SLOWER SPEED

38.4 Gbps Backbone

960 Mbps Backbone
Planned Next Steps

• 1(c) Complete recruiting of new hire (interviews 

conducted, candidate identified and has applied)

• Merge scaled-down MesoNet onto scaled-up Emulab

to determine envelope of feasibility

– How many physical Emulab nodes required for 258,158 

virtual nodes?

– Can Emulab support 40x scaled down MesoNet speeds? 

• 2 Replicate MesoNet sensitivity analysis for 40x scaled 

down MesoNet speeds on Emulab and evaluate results

– Same qualitative results?

– Reasonable basis for V&V method?

– Possible to characterize uncertainty of measured outputs?

10/27/2015 ITL Science Day 10
Questions?

10/27/2015 ITL Science Day 11