**IQ-ECho: Interactive Quality of Service Across Heterogeneous
Hardware**

## Karsten Schwan, Georgia Tech 

## Greg Eisenhauer, Georgia Tech 

## Matthew Wolf, Georgia Tech

*Summary*

***This project addresses the efficient transfer of large data across
wide-area networks, focusing on important scientific applications such
as remote visualization and real-time collaboration. The IQ-ECho
middleware mechanisms can involve both the application and the network
level in adapting to changes in network and platform conditions, with a
goal of enabling high-fidelity, end-to-end scientific collaboration,
targeting distributed computing environments that range from
Internet-based to high end wide area network infrastructures.***

**1. Introduction**

To attain high performance in the real-time exchange of data across
collaborating machines and end users, this project is developing and
evaluating middleware methods and techniques for coordinating
application-level with network transport-level adaptations of data
communication. This basic approach complements previous work on
TCP-friendly communication and on adaptive transport protocols,
attempting to meet application needs without violating fairness in
network resource usage.

**2. IQ-ECho**

The approach is embodied in the IQ-ECho middleware, which implements the
distribution of scientific data to remote collaborators. Using IQ-ECho,
application-level adaptations like selective data down-sampling are
triggered by transport-level information provided by the instrumented
IQ-RUDP protocol underlying IQ-ECho\'s communications. The application-
to network-layer exchange of information necessary for such coordinated
adaptations is implemented with quality attributes, which provide a
lightweight way for an application to provide quality of service
information and to describe its adaptation to the transport layer, and
for IQ-RUDP to share network status information with an

In addition to triggering application-level adaptations and reacting to
certain changes in network state, IQ-RUDP can also re-adapt its own
communication behavior after an application adaptation has been
performed, in part to remain fair to other network flows. Such
transport-level reactions can be performed at higher rates and with
smaller overheads than possible at application level.

**3. SmartPointer and IQ-ECho**

IQ-ECho is a key piece of technology behind **SmartPointer**, which
serves as a driving application and has communication demands similar to
wide-area collaboratories and tera-scale supercomputing ventures.

The SmartPointer application requires a communications infrastructure
that can be flexible, adaptive, and yet support high performance.
Traditional HPC-style communications systems like MPI offer the required
high performance but rely on the assumption that communicating parties
have a priori agreements on membership lists and on the basic contents
of the messages being exchanged. For the applications we target,
however, both data types and subscription lists of communicating parties
must be flexible. This need for flexibility has led some designers to
adopt techniques like Java\'s RMI or meta-data representations such as
XML+SOAP. These methods have high costs that interfere with total
performance, because data marshalling becomes a key issue.

The IQ-ECho middleware addresses these concerns in several different
ways. In particular, it provides the following:

-   Information flows are represented as **event streams**, using the
    publish/subscribe model.

-   There is no centralization; channels are represented by distributed
    data structures.

-   Subscribers preserve transparency of local versus remote receivers,
    with implementations for underlying peer-to-peer communications over
    TCP, UDP, Wireless, multicast, and others.

-   Dynamic extension of existing formats and discovery/operation on
    format contents is enabled through run-time dynamic code generation
    of subscriber-specified filters.

**3. Milestones Achieved**

The IQ-RUDP implementation has been integrated with several different
network measurement packages, including those from Nagi Rao and
Constantinos Dovrolis, both funded under the SciDAC project. This
enables end users to choose their preferred bandwidth-measuring
approach.

Further, a key point of integration this year was to enable
resource-aware filtering inside of the SmartPointer application based on
these network measurements. The visualization infrastructure has been
modified so that it can automatically adapt to real-time changes in
available bandwidth by modifying quality, color depth, or frame rate.
The results of this work are being submitted as a paper to HPDC this
year.

**4. Future Work**

Our future work with the evaluation of IQ-ECho services for remote
collaboration will (1) expand upon our service-level adaptations that
utilize different network-level techniques for assessing current network
bandwidth, (2) use overlay networks to combine the lightweight data
filtering and downsampling methods used in this paper with
heavier-weight methods for data transformation and summarization
executed by additional machines interposed into the path between data
providers and consumers, and (3) integrate with 3^rd^ party
collaboration infrastructures such as the Access Grid, also a SciDAC
project. Such work will focus on high end links using the large data
volumes produced by applications like the DOE Supernova Initiative. The
intent is to use actual TSI data across the 10GB link and other links
connecting GT with ORNL and other DOE sites.

![](media/image1.jpeg){width="3.245833333333333in"
height="2.6465277777777776in"}

> *Figure 1: Current implementation of the* *SmartPointer
> infrastructure, employed in a molecular dynamics visualization.*

**For further information on this subject contact:**

Dr. Thomas D. Ndousse, Program Manager

High-Performance Networks Program

Mathematical, Information, and Computational

Sciences Division

Office of Advanced Scientific Computing Research

[**tndousse@er.doe.gov**](mailto:tndousse@er.doe.gov) Phone:
(301)-903-9960