# Presentation: 406705

## Virtual Machines in Distributed Environments

- José A. B. Fortes
- Advanced Computing and Information System Laboratory
- University of Florida, Gainesville

## ACIS Lab history + statistics

- Founded 8/10/01
- 20+ people
- 5 “associated” faculty
- ECE, CISE, MSE
- Also 10+ “associated” faculty from Purdue, CMU, U. Colorado, Northwestern U., U. Mass., NCSU ...
- 7 M dollars in funding, 5 M dollars in equipment
- Approximately 1.5 M of subcontracts
- NSF, NASA, ARO, IBM, Intel, SRC, Cisco, Cyberguard ...
- Computer infrastructure/lab
- 300+ CPUs, .5 Tflops, 9 TBytes, Gigabit connections
- Access to CPUs at Purdue, Northwestern, Stevens I.

## Research trusts and sample projects

- Distributed computing (NSF, ARO, IBM, Intel)
- In-VIGO middleware for grid-computing
- UF Collaborators: R. Figueiredo, H. Lam, O. Boykin, A. George ECE; S. Sinnott, MSE; P. Sheng, CE., ...
- Purdue U., Northwestern U., Stevens Tech., ...
- Distributed information processing (NSF)
- Transnational digital government
- UF Collaborators: S. Su, CISE
- CMU, U. Colorado, U. Mass, North Carolina S.U., ...
- Nanocomputing (NSF, NASA, SRC)
- Biologically-inspired delay-based computing
- UF Collaborators: J. Principe, J. Harris, ECE
- Purdue U., Texas A&M, ...

## ACIS infrastructure at a glance

- Aggregate resources
- .3  to .5 Teraflops
- 7 to 10 TeraBytes (disk)
- .5 TeraBytes (memory)
- 1Gb, 10 Gb and .1 Gb

## Outline

- What’s in a talk title
- Environment as a container for app execution
- Distributed “a la” power grid
- Virtualization for creation and coexistence of different environments in physical resources
- A Grid-building recipe
- Words are easy, let’s build it: In-VIGO
- Architecture, Deployments and Futures
- Virtual Machines, Data, Networks and Applications
- Turning Applications into Grid Services
- Conclusions

## On-demand computing

- User: _this_ app _now_ and _here _and _as needed_.
- Provider: _any _app _any_ time _somewhere_ on similar resources
- Embodiments:
- Data-centers
- Grid-computing
- “coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations”
- 			*in the “The anatomy of the Grid,” Foster et. Al*
- local control, decentralized management
- open  general-purpose standards
- non-trivial QoS
- 			*per I. Foster’s “What is the Grid? A 3-point Checklist”*

## Resource sharing

- Traditional computing/data center solutions:
- Multitask/multiuser operating systems, user accounts, file systems ...
- Always available but static configurations
- Sharing possible if apps run on similar execution environments
- Centralized administration
- Tight control on security, availability, users, updates, etc
- Distributed Grid/datacenter requirements
- Multiple administrative domains
- Different policies and practices at each domain
- Many environments possible
- Dynamic availability
- Must run all kinds of applications
- Application user will neither trust unknown users sharing the same resource nor redevelop application to run in different environments
- Resource owner will neither trust arbitrary users nor change environment for others’ applications

## “Classic” Virtual Machine

- Copy of a real machine
- “Any program run under the VM has an effect identical with that demonstrated if the program had been run in the original machine directly” 1
- Isolated from other virtual machines
- “...transforms the single machine interface into the illusion of many” 2
- Efficient
- “A statistically dominant subset of the virtual processor’s instructions is executed directly by the real processor” 2
- Also known as a “system VM”
*1** “Formal Requirements for Virtualizable Third-Generation Architectures”, G. Popek and R. Goldberg, Communications of the ACM, 17(7), July 1974*
*2** “Survey of Virtual Machine Research”, R. Goldberg, IEEE Computer, June 1974*

## Process vs. System  VMs

- In Smith and Nair’s “The architecture of Virtual machines”, Computer, May 2005

## Classic Virtual Machines

- Virtualization of instruction sets (ISAs)
- Language-independent, binary-compatible (*not* JVM)
- 70’s (IBM 360/370..) – 00’s (VMware, Microsoft Virtual Server/PC, z/VM, Xen, Power Hypervisor, Intel Vanderpool, AMD Pacifica ...)
- ISA+ OS + libraries + software = execution environment

## 1 user, 1 app, several environments 

- 1 user, 1 app, several environments

- Compute Server

- Compute Server

- Compute Server

- Compute Server

- Compute Server

- Compute Server

**Grid**

- Slide provided by M. Zhao

## Many users, 1 app, many environments

- Compute Server

- Compute Server

- Compute Server

- Compute Server

- Compute Server

- Compute Server

- +CH3D

- VM

- +ArcView

- VM

**Grid**

**Middleware**

- Slide provided by M. Zhao

## Virtualization technology for grids 

- Resource virtualization technology
- Enables a resource to _simultaneously_ appear as _multiple_ resources with possibly _different functionalities_
- Polymorphism, manifolding and multiplexing
- Virtual networks, data, applications, interfaces, peripherals, instruments ...
- Emergent technologies

## Virtual networks

- Public
-  network A

- Private network C

- Private network B

- Public
-  network D

- Internet

**VR**

- Virtual network

- Virtual network

- Virtual network

- Virtual network

**R**

**N**

**F**

**Router**

**NAT**

**Firewall**

**Virtual**
**Router**

**H**

**Host**

**VH****1**** to VH****2**

**VH****3**** to VH****4**

**H**

**N**

**H****4**

**H****2**

**H**

**H**

**VH****2**

**VR****A**

**VH****4**

**VH**

**VR****C**

**R**

**F**

**H****3**

**H****1**

**H**

**VR****B**

**VH**

**N**

**VH****1**

**VR****D**

**H**

**H**

**VH****3**

**Virtual Space**

**Physical Space**

- logical links:
- multiple physical links, routing via native Internet routing
- tunneling, virtual routers, switches, ...
- partial to total isolation

- Slide provided by M. Tsugawa

## Data/file virtualization

- NFS
- Client

- NFSD

- Server

- Mountd

-  Client

- NFS Server ‘S’

- Export /home to
- all uids on compute
- server C

- Compute Server ‘C’

- mount
- S:/home

## Web services framework

- allows programs on a network to find each other, communicate and interoperate by using standard protocols and languages

## Basic service description: interface definition

- abstract or reusable service definition that can be instantiated and referenced by multiple service implementation definitions

- different implementations using the same application can be defined to reference different service definitions – a form of virtualization

## Application virtualization

**Regular**
**Service**

**Restricted**
**Service**

**Composed**
**Service**

**Augmented**
**Service**

- Virtual Application Monitor

- App3

- App1

- App2

## A Grid-building recipe

- Virtualize to fit needed environments
- Use services to generate “virtuals”
- Aggregate and manage “virtuals”
- Repeat  as needed

-  Net result:
-  users interact with virtual entities provided
- by services
-  middleware interacts with physical resources
-  In-VIGO is a working proof-of-concept!

## The In-VIGO approach

-  local control, decentralized management
-  open  general-purpose standards
-  non-trivial QoS

## In-VIGO: a user’s view

- Enables computational engineering and science _In-V_irtual _I_nformation _G_rid _O_rganizations
- Motivations:
- Hide complexity of dealing with cross-domain issues
- From application developers
- From end users
- Provide secure execution environments
- Goals:
- Application-centric: support unmodified applications
- Sequential, parallel
- Batch, interactive
- Open-source, commercial
- User-centric: support Grid-unaware users

## http://invigo.acis.ufl.edu

## The In-VIGO portal

## Virtual workspace

## The In-VIGO portal

## Setting up ...

## Interface and workflow

## File manager (1)

## File manager (2)

## The In-VIGO portal

## Native interactive interface

## nanoHUB (current middleware infrastructure)

- Campus Grids
- Purdue, GLOW

**Grid**

- Capability Computing

- Science Gateway

- Workspaces

- Research apps

- Virtual backends
- Virtual Cluster with VIOLIN

- VM

- Capacity Computing

- nanoHUB VO

**Middleware**

- Slide provided by Sebastien Goasguen

## In-VIGO 1.0 architecture diagram

- Deployed at UF/ACIS since Fall 2003
- nanoHUB: Summer 2005
- On-going deployments: SURA/SCOOP, UF/HPC

## Virtual Machine System

*Enables on-demand instantiation of whole-O/S VMs for virtual workspaces*
- Has access to physical resources (host)
- Create, configure, query, destroy VMs
- In-VIGO users have access to virtual resource (guest)

## VM services

- Provide means to efficiently create/configure/destroy VMs,
- generic across VM technologies
- Directed Acyclic Graph (DAG) model for defining application-centric VMs
- Cost-bidding model for choosing compute servers for VM instantiation

- Slide provided by Arijit Ganguly

- (SC 2004)

## Architectural Components of VM Service

**VM Creation Request from Client **
**(e.g. In-VIGO) **

- (1)
- VM Request

- (6)
- VM  Classad

- (2)
- Request Estimate

- (3)
- VM Creation
- Cost

- (4)
- Create VM

- (5)
- VM Classad

- mcnabb

- vws010

**VMPlant**
**Daemon**

- brady

- vws001

**VMPlant**
**Daemon**

- favre

- vws005

**VMPlant**
**Daemon**

- mcnair

- vws002

- manning

- vws003

**VMShop (VMArchitect**
**VMCreator, VMCollector, VMReporter)**

- Host OS
- (VMPlant)

- Host OS (VMPlant)

- Host OS
- (VMPlant)

**VMPlant**
**Daemon**

- Slide provided by Arijit Ganguly

## VMPlant API

- Clone VM
- Instantiate a new container
- Fast copying of a base VM image
- Virtual disk
- Suspended memory (if available)
- Configure VM
- Execute scripts/jobs inside container to tailor to a particular instance
- Communication crossing container boundaries to provide inputs/retrieve outputs
- Destroy VM
- Terminate container, delete non-persistent state

## Data access virtualization

- Grid virtual file systems (GVFS)
- On-demand setup, configuration and tear-down of distributed file systems
- Unmodified applications access file-based data in the same manner they would in a local environment
- Use and extend Network File Systems (NFS)
*Multiple*,* independent* file system sessions *share* one or more accounts in file servers
- File system data is transferred on-demand, on a per-block basis

## Grid Virtual File System (GVFS)

**kernel**

**NFS**
**server**

**proxy**

**VM state server S**

**WAN**

**Compute server C**

**VM****state**

**VMM**

- Logical user accounts [*HCW’01*] and virtual file system [*HPDC’01*]
- Shadow account + file account, managed by middleware
- NFS call forwarding via middle tier user-level proxy
- User identities mapped by proxy
- Provides access to user data, VM images

***Map identities;******Forward RPC calls***

## Challenge: VM State Transfer

- Compute Server

- Compute Server

- Compute Server

- Compute Server

**Grid**

**Middleware**

**VM State Servers**

- Dynamic, efficient transfer of large VM state is important

- Many users, apps and environments

- Slide provided by M. Zhao

## User-level Extensions

**kernel**

**NFS**
**server**

**proxy**

**VM state server S**

**WAN**

**Compute server C**

**VMM**

- Client-side proxy disk caching

**buffer**

**block-****based****cache**

- Application-specific meta-data handling

- Encrypted file system channels and cross-domain authentication

*[Zhao, Zhang, Figueiredo, HPDC’04]*

**file-****based****cache**

**disk   **

**mem**

**VM state**

## Putting it all together: GUI Application example

**Front**
**end ‘F’**

**Physical server**
**pool P**

**User ‘X’**

**Data**
**Server D1**

**User ‘Y’**

**Data**
**Server D2**

**Information**
**service**

## Virtual network services

- VMShop allocates a remote VM
- Great; now how to access it?
- Need to isolate traffic from host site
- For most flexibility, need full TCP/IP connectivity
- ViNe, IPOP being developed at UF ACIS
- Related work: Virtuoso/VNET (NWU), Violin (Purdue)

## In-VIGO Virtual Networks - ViNe

- IP overlay on top of the Internet
- Operation similar to site-to-site VPN
- Designed to address issues that VPN does not solve:
- High administrative overhead for many sites
- VPN firewalls need a static public IP address

## In-VIGO Virtual Networks - ViNe

- Public
-  network A

- Private network C

- Private network B

- Public
-  network D

- Internet

**VR**

**R**

**N**

**F**

- Router

- NAT

- Firewall

- Virtual
- Router

**H**

**N**

**H****4**

**H****2**

**H**

**H**

**VR****A**

**VR****C**

**R**

**F**

**H****3**

**H****1**

**H**

**VR****B**

**N**

**VR****D**

**H**

**H**

- Virtual Space

- Physical Space

- Slide provided by M. Tsugawa

## ViNe – Communication in virtual space

- Public
-  network A

- Private network C

- Private network B

- Public
-  network D

- Internet

**VR**

- Virtual network

- Virtual network

- Virtual network

- Virtual network

**R**

**N**

**F**

- Router

- NAT

- Firewall

- Virtual
- Router

**H**

- Host

**H**

**N**

**H****4**

**H****2**

**A**

**H**

**VH****2**

**VR****A**

**VH****4**

**VH**

**VR****C**

**R**

**F**

**H****3**

**H****1**

**H**

**VR****B**

**VH**

**N**

**VH****1**

**VR****D**

**B**

**H**

**VH****3**

- Virtual Space

- Physical Space

- Packet with header VH1→VH2 is directed to VRB

- VRB looks-up its routing table with Subnet(Virtual Space) –IP(Physical Space) entries. It indicates that the packet should be forwarded to “A”.

- ViNe packet is encapsulated with an additional header for transmission in physical space: B→A:(VH1→VH2)

- ViNe header is stripped off for final delivery.

- Original, unmodified packet VH1→VH2 is delivered.

- Slide provided by M. Tsugawa

## ViNe – Local communication

-  Local communication is kept local in both Physical and Virtual space.
-  ViNe does not interfere with physical communication.
-  Virtual space can be used only when needed.

- Slide provided by M. Tsugawa

## ViNe Firewall/NAT traversal

- VRs connected to the public network proxy (queue) packets to VRs with limited connectivity. The latter open connection to the queue VR to retrieve packets.
- VRs with limited connectivity are not used when composing routing tables. Routing tables are made to direct packets to queue VRs.
- The approach supports multi-level NAT.
- The approach also works under DHCP since the changing IP is not considered for routing.

## ViNe organization

- Routing tables are created/destroyed as needed (e.g., join/leave of sites, creation of a new ViNe, etc).
- VRs exchange routing information with each other
- Communication of sensitive information (e.g., routing tables, VRs host certificates) is encrypted.
- Administrator of a participating site is involved only during the setup/configuration phase. No intervention is needed when machines join/leave network.

- Slide provided by M. Tsugawa

## ViNe Overhead

- When firewall/NAT traversal is not required
- depends on performance of VRs and available physical network
- Overhead = 0 – 5% of available bandwidth
- up to 150 Mbps for VR on 3 GHz Xeon
- When firewall/NAT traversal is required
- also depends on the allocation of VRs to proxy/queue traffic
- 10 – 50% in initial experiments. Optimizations under investigation.

- Slide provided by M. Tsugawa

## ViNe Security

- Site-related
- security policies are not changed by enabling ViNe
- minimal change may be needed to allow ViNe traffic in private IP space
- ViNe traffic consists of IP packets that are visible in LANs (tunneling is only used across domains)
- Network policies can be applied to ViNe traffic
- Firewalls can inspect ViNe traffic
- Intrusion detection systems and monitoring works unmodified
- ViNe-related
- ViNe routers do not route packets to/from the Internet
- All communication between VRs are authenticated
- Sensitive VR messages are encrypted
- VRs are not accessible in ViNe space
- ViNe connects hosts without links in physical IP infrastructure
- But it does so only where we want to have it

- Slide provided by M. Tsugawa

## ViNe: On-going work

- Management of Virtual Networks
- Automated and secure management (definition, deployment, tear-down, merge, split and join/leave of hosts) of virtual networks is under development in the context of ViNe project
- The idea is to dynamically and securely and reconfigure ViNe routers in response to client (privileged users, local site administrators, grid administrators, grid middleware) requests
- In collaboration with ANL

- Slide provided by M. Tsugawa

## ViNe: Auditability

- ViNe does not modify packets generated by participating hosts
- Regular network traffic inspection can be performed in each participating site
- In addition, ViNe Routers can log all routed traffic (performance implications are under investigation)
- Side-process can combine traffic logs for global network traffic analysis

- Slide provided by M. Tsugawa

## IPOP virtual network

- Motivations:
- Enable self-configuring virtual networks – focus on making it simple for individual nodes to join and leave
- Decentralized traversal of NATs and firewalls
- Approach: IP-over-P2P
- Overhead of adding a new node is constant and independent of size of the network
- Peer to peer routing
- Self-organizing routing tables
- Ring topology with shortcuts
*N* nodes, *k* edges per node; O(1/*k* log2(*N*)) routing hops
- Adaptive, 1-hop shortcuts based on traffic inspection
- Mobility: same IP even if VM migrates across domains
- A. Ganguly, A. Agrawal, P. O. Boykin, R. Figueiredo – IPDPS 2006, HPDC 2006

- Slide provided by R. Figueiredo

## Applications

- Distributed computing VM “appliances”:
- Define once, instantiate many
- Homogeneous software configuration and private network address spaces
- Facilitates a model where resources are pooled by the various users of a community (e.g. nanoHUB)
- Homogeneous configuration facilitates deployment of security infrastructures (e.g. X.509-based IPsec host authentication)

- Slide provided by R. Figueiredo

## Usage examples

- “Grid appliance”
- Condor node for job submission/execution
- Automatically obtains a virtual address from virtualized DHCP server and joins a pool
- Can submit and flock jobs within virtual network
- Download VMware player and VM image: http://www.acis.ufl.edu/~ipop/grid_appliance
- On-going: domain-specific customizations
- nanoHUB: WebDAV client, Rappture GUI toolkit
- SCOOP (coastal ocean modeling): clients to access data catalog and archive
- Archer (computer architecture): support for large, read-only checkpoints and input files

- Slide provided by R. Figueiredo

## Application virtualization

## Grid-enabling unmodified applications

**Virtual Application **
**Enabling**

- VA Framework

- Other Frameworks

- Enabling Service

- Enabler

- IS
- Service

**Virtual Application **
**Service Utilization**

- VA 2
- Service

- VA 3
- Service

- VA 1
- Service

- User

- VFS Service

- VM
- Service

- Enabling

- Utilization

**Portal**
**Tier**

**Virtual Application**
**Tier**

**Virtual Grid**
**Tier**

**Virtual Application **
**Customization and Generation**

- Administrator

- Portal Interface

- Customization and Generation

- Portal Interface

- Portal Interface

- ViNe
- Service

- Enabler provides
- Command-line syntax
- Application-related labels
- Parameter(s), type-set values, entire applications
- Resource and execution environment metadata
- Architecture, OS libraries, environment variables
- Grid-services created, deployed and possibly customized using
- Generic Application Service (GAP)
- Virtual Application Service (VAS)
- Grid-user interacts with the virtual application through a Web-portal to execute applications on virtualized resources

- Customization Service

- GAP/VAS Generator
- Service

## Summary and conclusions

- Virtualization technology decouples physical resource constraints from user and application requirements
- Big win, novel rethinking
- Virtual resources are to grid computing what processes are to operating systems
- Developers can concentrate on applications, not end resources
- Web-services provide interoperability  and a framework for composition and aggregation of applications
- Includes delivering virtuals and virtualizing applications
- Wide adoption creates large reusable toolboxes, e.g. for automatic interface generation
- Users need only know of service interfaces
*In-VIGO middleware effectively integrates virtualization and Web-services technologies to easily enable and deliver applications as Grid-services*

## Current In-VIGO team

- Sumalatha Adabala
- Vineet Chadha
- Renato Figueiredo
- José A. B. Fortes
- Arijit Ganguly
- Herman Lam

- Andrea Matsunaga
- Sanjee Sanjeepan
- Yuchu Tong
- Mauricio Tsugawa
- Jing Xu
- Jian Zhang
- Ming Zhao
- Liping Zhu

**http://www.acis.ufl.edu/**

## Acknowledgments

**Collaborators**
- In-VIGO team at UFL
- http://www.acis.ufl.edu/invigo
- Rob Carpenter and Mazin Yousif at Intel
- Peter Dinda and Virtuoso team at NWU
- http://virtuoso.cs.northwestern.edu
- NCN/NanoHub team at Purdue University
- http://www.nanohub.org
- Kate Keahey, ANL
**Funding**
- NSF
- Middleware Initiative and Research Resources Program
- DDDAS Program
- Army Research Office
- IBM Shared University Research
- Intel
- VMWare
- Northrop-Grumman

## In-VIGO futures

- Service-oriented middleware
- Migration of features in development versions of In-VIGO
- Virtual file system service
- Virtual Networking for aggregating resources across firewalls
- Virtual application services (VAS/GAP)
- Automatic installation of applications on In-VIGO
- Autonomic capabilities (ICAC’05)
- Real-time grid computing