Selected Papers

GEMnet2: New Network Testbed for
Global R&D
Hisao Uose†, Masayasu Yamaguchi, and Junichi Kishigami
Abstract
NTT has been successfully operating its experimental network GEMnet (Global Enhanced Multifunctional Network) for research and development of global information-sharing services since 1998. To
adapt to new requirements in the business environment and technical advances, we have formulated a
new testbed concept, GEMnet2, and begun construction. Its three key aims are to let us test technologies
for every aspect of communications, provide very wide bandwidth that can accommodate very fast applications without any restraints, and promote collaboration with other research networks. We are using
wavelength division multiplexing to build 10-Gbit/s circuits between NTT’s R&D centers in Musashino,
Yokosuka, and Atsugi.

1. Introduction
GEMnet (Global Enhanced Multifunctional Network) is a network testbed operated by NTT Service
Integration Laboratories that was built for researchand development of global information-sharing services [1]-[3]. It started operating in 1998 to help verify new technologies in a real network environment
by providing network and application researcherswith a global network testbed. Since then many
research projects have used GEMnet to conduct
experiments on their own or jointly with other organizations both within and outside Japan, producing
many valuable results [4].
Recent use of GEMnet has focused on experiments
aiming to investigate new global information sharing
services. At the same time, in light of the changing
business environments around NTT and rapid technical development especially in the very-high-speed
applications and photonic transmission technology,
we felt it was necessary to review the purposes andconfiguration of GEMnet. In addition, there areurgent demands for a network testbed on which new
Internet Protocol (IP) technologies can be tested.
When we consider networks for other research pur† NTT Service Integration Laboratories
Musashino-shi, 180-8585 Japan
E-mail: uose.hisao@lab.ntt.co.jp

Vol. 1 No. 5 Aug. 2003

poses, the recent development of research and education (R&E) networks is astonishing. Most R&E networks are funded by governments and non-profit
organizations, which are typically consortiums ofresearch institutes and universities. In Japan, theNational Institute of Informatics (NII) operates Super
SINET (Science Information Network), which is anationwide R&E network funded by the Ministry of
Education, Culture, Science and Sports of Japan. Its
backbone bandwidth is 10 Gbit/s and it has separate
Gigabit Ethernet networks for five dedicated scientific applications [5].
In the United States, there are many research networks funded by the federal government as well asones operated by the private sector. Internet2, a consortium of American universities founded forresearch on next-generation Internet technologies,
operates its own research network called Abilene
with the help of vendors and communications carriers[6]. Many countries have equivalent R&E networks
and they are rapidly being upgraded in performance.
Experience using those R&E networks led to new
requirements for high-performance network testbeds
being studied. The U.S. National Science Foundation(NSF) has announced its “Cyberinfrastructure” concept to promote the use of high-performance networks for scientific applications [7]. It stresses the
key role played by communications networks in the
new methodology of conducting advanced scientific31Selected Papers

experiments.
After evaluating the new requirements for the testbed, we came to the conclusion that we should renovate GEMnet in many aspects from the physical configuration to the ways of using it. The idea is to build
a new high-performance network testbed that can beused to verify a wide variety of technologies fromphysical- to application-layer ones. Our approach tobuilding it is to get the most out of existing resources,by unifying GEMnet with the ultrahigh-speed network testbed (GALAXY) also being operated by
NTT Service Integration Laboratories [8]. To provide
very-high-speed communications links, we decided
to utilize photonic transmission technologies more.
Many of them were developed at NTT Laboratories
and some were already deployed in the GALAXY
network. This will enable the testing of very-highspeed applications (several Gbit/s). Reconfigurationof these two experimental networks began last year.
This paper describes the concept and deployment

plan of GEMnet2. First, we briefly review the history
of GEMnet describing the network configuration anda few examples of experiments conducted on it. Then,we explain the concept of the new testbed and why it
has become necessary. We describe our approach toconstructing the new testbed focusing on the use ofphotonic technologies and collaboration with R&E
networks. Finally, we mention future plans for GEMnet2.
2. History of GEMnet
2.1 Network configuration
Figure 1 shows the configuration as of March 2002.
Since the beginning, the bandwidths between various
connection points have been increased/decreased inaccordance with actual usage and user requirements.
Within Japan, three NTT R&D centers (Musashino,
Yokosuka, and Keihan-na) are interconnected
through dedicated ATM (asynchronous transfer

CA*net II
STAR TAP
ChicagoMREN
20 Mbit/s(ATM, IP/ATM)

Abilene

San JoseMusashino

Otemachi

San Jose

it/s
3 Mb

Yokosuka
Keihan-na

85 Mbit/s

R&D centers
Access points
ATM circuits

Fig. 1. Configuration of GEMnet as of March 2002.

32

NTT Technical Review

Selected Papersmode) links. In the U.S. it has been connected to themetropolitan research and education network(MREN), which is an advanced high-performancebroadband testbed operated by a consortium of
research organizations.
ATM technology was chosen to provide guaranteedbandwidths for a number of application users at thesame time. The bandwidth between Musashino and
Yokosuka is 85 Mbit/s while that between Musashino
and Keihan-na is 3 Mbit/s. Access points to GEMnet
are located in the three R&D centers and also in NTT
Otemachi building. Connection equipment for ATM
links and IP routers is provided at each access point,
allowing researchers to choose either ATM or
IP/ATM for their experiments.
Internationally, GEMnet has been connected overATM circuits to NTT facilities in San Jose in the U.S.
and NTT MSC in Cyberjaya, Malaysia via the NTT
Otemachi building. The circuit between Otemachi
building and the MSC has been discontinued due tothe termination of the research project involving the
MSC. The network did not have an IP connection
until June 2001, when it was connected to Abilene,
which is the backbone network of the Internet2 project.
NTT Laboratories is a member of UCAID (University Corporation for Advanced Internet Development), a cooperative body running the Internet2 project, which promotes research on the next-generation
Internet, with members from more than 200 universities and a number of private companies in the UnitedStates and other countries. Internet2 is operating a 10Gbit/s backbone network called Abilene, which isused by its members for academic research purposes.
The interconnection between Abilene and GEMnet
opened up the opportunity for joint experiments withnot only universities in the United States but also
other research institutes connected to Abilene.
2.2 Usage policy
GEMnet was built specifically for use as a testbed
in experiments required for NTT’s R&D. Its operating policy precludes commercial or business uses
other than R&D. Other participating organizations
sign a joint research agreement with NTT Laboratories that stipulates the terms and conditions of using
GEMnet and its research results. Research partners
include the University of Tokyo, Kyoto University,
Waseda University, University of Illinois, and University of California - Los Angeles (UCLA).

2.3 Applications
Many experiments have been conducted using
GEMnet so far. This section overviews a few examples. Details of these experiments and other experiments are given in ref. [4].
(1) Remote lectures
From 1999 to 2003, NTT Laboratories conducted
remote lecture experiments with Kyoto University
and UCLA over GEMnet. Up to fiscal 2000, theexperiments aimed to identify technical problems
in the remote lecture system. Solutions to thesehave since been implemented. Both professors andstudents gained valuable experience in remote lecture techniques.
(2) Virtual space sharing network (N*Vector)
NTT Laboratories, the University of Tokyo, andthe University of Illinois are conducting joint
research on using a network to allow the sharing ofa virtual space in different locations. GEMnet and
MREN are used to interconnect NTT Yokosuka
R&D center and the two universities. In 2001 thethree participants built a network system for experiments in operating differentiated services (DiffServ) and are using it to measure quality of service(QoS).
(3) IPv6 operation experiments
NTTv6net, an experimental network using IPv6,
stretches from Asia to North America and Europe.
It is connected to many ISPs and IPv6 experimental
networks at IPv6 native Internet exchanges (IXs),
such as NSPIXP6 in Tokyo, PAIXv6, SIX, and
6TAP in the United States, and AMS-IX in Amsterdam. As of September 30, 2001, 42 organizations were interconnected through its border gateway protocol. GEMnet is used as the backbone ofthis world-leading IPv6 network, linking Japan andthe United States.
(4) Data over photonic network (DoPN)
We developed a prototype access system for avariable-rate connection service offering a peakrate of over 1 Gbit/s. To evaluate the scheme’s characteristics, GEMnet was used to test its on-demand
operation. A bandwidth controller client inMusashino R&D center and an optical line termination (OLT) agent in Yokosuka R&D center wereinterconnected via GEMnet, and a remote ondemand bandwidth control experiment was successfully carried out.
3. Concept for the new testbed
The requirements for the network testbed haveVol. 1 No. 5 Aug. 2003

33

Selected Papers

changed as communications technologies haveadvanced. This prompted us to review the position of
GEMnet and its network configuration. The following are the issues that we took into account when weformulated the basic plan for the new testbed.
3.1 Testbed for all layers of technologies
The new testbed should provide an experimental
environment for all aspects of broadband communications. In the physical layer, in which photonic technology is pushing up against the physical limitationsof transmission speed, minute differences in the characteristics of transmission media affect communication performance and quality. This requires us to test
new photonic technologies in the field using actual
fibers with a mixture of different specifications andages.
In the network layer, there are many complicatedissues that need to be solved, including traffic engineering, quality control, routing, and security.
Because the Internet consists of heterogeneous networks operated by different organizations, measures
to solve the problems have many constraints. In addition, conspicuous variations in traffic characteristics
and rapid changes in technology require us to verify
those measures on an actual network with interconnections to other networks.
Broadband applications using IP technology are
emerging very rapidly, so research on applications
operating on a very-high-speed network infrastructure is required. At the same time, the network shouldemulate the actual conditions of commerciallydeployed networks because the characteristics of an
IP network are closely interlinked with those of otherlayers and applications.
Inter-working among researchers in different specialties is another target of this testbed. Althoughmany experiments covering many technical issues
have already been conducted, we feel it is necessary
to provide an environment that will boost closer collaboration among trials.
3.2 Focus on IP and photonic technologies
Internet Protocol (IP) technology has begun to playa dominant role in communications networks providing conventional real-time services including voice
and video services along with conventional Internet
services. This trend requires us to provide a testbed
for investigating the specific problems related to IP
technologies. This can only be done using a network
running many other applications with connections toother networks.
34

Wavelength division multiplexing (WDM) is thebest example of new photonic technologies being
rapidly deployed by many carriers, but effective useof dynamic wavelength switching is still being studied. NTT Laboratories is leading this field and the
new testbed will enable effective verification ofdeveloped technologies because it will be built of
actual fibers and will run actual applications.
3.3 Requirements for higher bandwidth
The latest applications require bandwidths from a
few hundred Mbit/s to several Gbit/s, so the new testbed should be able to provide this level of bandwidth.
Typical examples are scientific applications in fields
such as astronomy, physics, and genetic engineering.
Super High Definition (SHD) video transmission isanother application that requires very high bandwidth[9]. Distributed computing is another interesting
application because it requires broadband connections among distantly located processors, and the performance of the network will greatly affect the total
processing power. With the use of photonic technology, achieving this target is not difficult.
3.4 International collaboration
The original aim of GEMnet was to promote collaboration with research groups abroad as well asones within Japan. The significance of international
cooperation is growing year by year and partnerships
with other R&E networks play an important role inthis respect.
One example of international collaboration alreadystarted is the joint research project on remote observation using a radio telescope with the National
Astronomical Observatory of Japan (NAOJ) and the
University of Chile [10]. In this project, astronomers
at NAOJ’s main campus in Tokyo can access theirmonitoring equipment located in the Atacama desert
in northern Chile via GEMnet. To make this possible
we are collaborating with a number of R&E networks, which provide transit services for our experiment, as shown in Fig. 2. In GEMnet2 the connectivity to other R&E networks will be enhanced to makemore experiments possible.
4. Approach for GEMnet2
Figure 3 shows the planned configuration of GEMnet2 and the research topics that will be conducted onit.

NTT Technical Review

Selected Papers

NAOJ test
instruments

San Pedro deAtacama
NAOJ
base
Japan
NAOJ
MitakacampusTrans-Pacific

USTrans-Americas

Chile

GEMnetAbilene

AMPATHREUNA

Musashino
R&D Center

U. Chile
Partner R&E networks

Fig. 2. Remote monitoring of NAOJ test instruments.
New
photonic
technology

Usuda DeepSpace Center

Traffic
engineering/routing

Musashino
R&D center
Nobeyama Radio
Observatory

Verificationof AWGSTAR

Real-time
VLBI

CRL
Kashima

NII
SINET

CRL
OtemachiVerificationof CWDM

NAOJ
International link
Super highdefinition
video

AtsugiR&D center
Verificationof DWDMand OXCDWDM network
CWDM network
STM-16AWG-STAR network

Yokosuka
R&D center

MIT

Distributed
processing

AbileneUCLA

AMPATH
REUNA
U. Chile

Fig. 3. GEMnet2 and its research topics.

Vol. 1 No. 5 Aug. 2003

35

Selected Papers

4.1 Unification with GALAXY network
GALAXY [8] is another NTT high-speed experimental network, currently being used for real-time
VLBI (very long baseline interferometry) experiments. This network is entirely operated by NTT
Laboratories, from optical fibers to applications,
without the use of any commercial services. All thesynchronous digital hierarchy (SDH) transmission
equipment, WDM devices, repeaters, ATM switches,
IP routers, and layer-2 switches are owned and operated by us. This gives us great flexibility when conducting experiments with extreme requirements. In
addition, we can test lower-layer technologies such as
photonic devices using this framework. Through theoperation of GALAXY, we have accumulated a lot ofknow-how that should be applicable to the new testbed.
4.2 Use of photonic technologies
We plan to utilize new technologies developed in
NTT Laboratories that are applicable to the new testbed, especially the latest photonic devices and transmission equipment utilizing them. In 2001 we
deployed a full-mesh photonic network system(whose product name is AWG-STAR) together with
DWDM (dense WDM) multiplexers in the GALAXY
network. AWG-STAR is a WDM filter with silicabased PLC (planar lightwave circuit) technology
developed by NTT Photonics Laboratories and it can
multiplex/demultiplexing several WDM signals
simultaneously. It connects two sites in Musashino
R&D center with NAOJ and the Communications
Research Laboratory (CRL) using eight wavelengths.
Figure 4 shows the DWDM equipment installed inFig. 4. DWDM multiplexer/demultiplexer used with AWGSTAR.

36

Musashino.
We are planning to extend this network to Atsugi
R&D center using coarse wavelength division multiplexing (CWDM) this year to make high-speed application trials possible between Atsugi and other sites.
Figure 5 shows the testing of optical fiber amplifiers
planned to be used for the circuit.
We also plan to deploy photonic transport systems
developed by NTT Network Innovation Laboratories.
One is a DWMD transmission system which supports
optical cross-connection (OXC). Once this system
has been deployed, we will be able to investigate new
network control schemes using wavelength switching.
4.3 Collaboration with other R&E networks(1) Collaboration with Super SINETIn December 2002 we connected GEMnet toSuper SINET. The link was established using an
OC-12 (622 Mbit/s) wavelength through the
DWDM network connecting Musashino R&D center and the NII building in downtown Tokyo. Thisconnection made possible broadband experimentswith academic sectors within Japan. The GALAXY
project also uses this connection for real-time
VLBI experiments. We will promote joint researchwith other research institutes utilizing this connection and may upgrade the bandwidth.
(2) Upgrade of the trans-pacific link
In May this year, the trans-Pacific link connecting
GEMnet to R&E networks in the United States was
upgraded to OC-3 (155 Mbit/s) from the previous
bandwidth of 20 Mbit/s. The circuit is connected to
Pacific Wave, which is the main connecting point

Fig. 5. Optical amplifiers for CWDM signals.

NTT Technical Review

Selected Papers

for R&E networks on the west coast of the UnitedStates. Pacific Wave is operated by the Universityof Washington and we have a peering agreement*1
with the Pacific Northwest GigaPoP as well as Abilene. This upgrade greatly expands the possibilityof international experiments in combination ofInternet2’s international transit service.
4.4 Network operation
Although owning and operating the testbed by ourselves is ideal for running a high-performance testbed
from the viewpoint of performance and flexibly, its
operation is inevitably very difficult and complicated.
Because GEMnet is now connected to other R&E
networks, proper network management is essential.
We have reorganized the GEMnet operations team sothat it can handle issues related to new photonic technologies as well as higher layer technologies such as
ATM and IP. The new network operations center(GEMnet NOC) is now operating 24 hours a day,every day of the year.
To manage all network equipment centrally, we aredeveloping a special network management system
based on ATMView, which has been developed at
NTT Laboratories to manage multi-vendor networks
built on an ATM network infrastructure. It utilizes anobject-oriented database to store operations data.
Traffic engineering is another important issue for our
testbed. We have deployed a traffic monitoring system called SmartProbe, which was also developed by
NTT Service Integration Laboratories. There are currently three SmartProbes installed within GEMnet,and traffic flow data is collected regularly every day.
5. Conclusions and future plans
Based on a thorough review of GEMnet activities
we have started to renew the network with new operation concept and technologies. The target is to build
a high-performance network testbed (GEMnet2) thatcan be used to test ultrahigh-speed applications aswell as lower-layer technologies within our laboratories. We utilize photonic transmission technologies toenable the testing of very-high-speed applications*1 Agreement on exchanging traffic between IP networks on a bilateral basis.

Vol. 1 No. 5 Aug. 2003

and the focus is on IP technologies. Although GEMnet2 is a research network owned and operated by
NTT Laboratories, it has connections with publicfunded R&E networks. This is a unique feature of
GEMnet2 and allows us to conduct a broader range ofexperiments. Joint research partners will also benefit
from the use of GEMnet2 and collaboration withNTT Laboratories by closely studying the use ofhigh-speed networks with experts.
Network renovation started last year. So far, theconnection to Super SINET and the upgrading of theinternational circuit have been completed. The
deployment of the DWMD circuit connecting
Musashino and Yokosuka R&D centers is under way
and should be completed in September 2003. The
CWDM circuit connecting Musashino and Atsugi
R&D centers is in the design phase and we aim tocomplete its installation by the end of 2003. When we
have completed these two WDM circuits we will
have 10-Gbit/s links among the three R&D centers,
and we plan to test several very-high-speed applications using the network this year.
References[1]

S. Suzuki, K. Kinoshita, K. Miyaho, and H. Yamashita, “Global R&D
Activities and Development of Global Testbed,” NTT REVIEW, Vol.
11, No. 2, pp. 69-78, 1999.
[2] N. Miyaho, F. Hyuuga, R. Arai, and K. Kinoshita, “Network Configuration of GEMnet),” NTT REVIEW, Vol. 11, No. 2, pp. 79-89, 1999.
[3] K. Okada, H. Yamamoto, and H. Yamashita, “NTT’s Global R&D
Strategy and Testbed after Reorganization,” NTT REVIEW, Vol. 12,
No. 2, pp. 8-13, 2000.
[4] K. Okada, J. Kishigami, M. Yamaguchi, N. Miyaho, and N.
Yamauchi, “Current Status and Future Direction of NTT’s Global
Networks for Experiments, GEMnet,” NTT REVIEW, Vol. 14, No. 2,
pp. 26-33, 2002.
[5] S. Asano, “Super SINET,” http://www.ccirn.org/general/2002material/
SuperSINET-CCIRN.ppt, CCIRN General Meeting, Arlington,
U.S.A., 2002.
[6] “Internet2,” http://www.internet2.edu/resources/Internet2Overview.
ppt[7] A. Blatecky, “Future Directions in Networking,” http://web.haystack.
mit.edu/e-vlbi/Blatecky.pdf, First e-VLBI Workshop, MIT HaystackObservatory, U.S.A., 2002.
[8] H. Uose, “Application of Ultrahigh-speed Network to Advanced Science,” NTT Technical Review, Vol. 1, No. 5, pp. 39-47, 2003.
[9] T. Yamaguchi, “SHD Digital Cinema Distribution over a Global
High-speed Network: Internet2,” NTT Technical Review, Vol. 1, No.
5, pp. 10-15, 2003.
[10] N. Yamauchi, E. S. Vera, H. Yamamoto, and H. Uose, “History and
Future Prospects of Research Collaboration between NTT Laboratories and the University of Chile: AccessNova, NTT REVIEW Vol. 14,
No. 2, pp. 53-58, 2002.

37

Selected Papers

Hisao Uose
Senior Research Engineer, Supervisor, NTT
Service Integration Laboratories.
He received the B.E. and M.E. degrees in electrical engineering from Kumamoto University,
Kumamoto, in 1978 and 1980, respectively. He
joined NTT Laboratories, Tokyo, Japan in 1980.
Currently he is in charge of the ultrahigh-speed
application experiments and high-performance
network testbed at NTT Service Integration Laboratories. He was a guest associate professor andguest professor at the National Center for Science Information Systems in 1994-1998 and1998-2000, respectively.

Masayasu Yamaguchi
Senior Research Engineer, Supervisor, NTT
Network Innovation Laboratories.
He received the B.E., M.E. and Ph. D. degrees
in electrical and electronic engineering from
Chiba University, Chiba in 1980, 1982 and 1994,
respectively.
He joined NTT Electrical Communication
Laboratories in 1982, where he mainly engaged
in research on photonic switching systems andrelated technologies. His current interest is metadata technologies enhancing content distribution
services on photonic networks. He is a memberof the Institute of Electronics, Information andCommunication Engineers (IEICE) of Japan andthe IEEE/COMSOC (Communications Society),
LEOS (Lasers and Electro-Optics Society).

38

Junichi Kishigami
Director, Project Manager, NTT Service Integration Laboratories. Currently he is a director at
NTT Laboratories promoting the concept of“metadata” and RFID, which would be a key
function for the new convergence business.
He received bachelor’s and master’s degrees inphysics and doctor degree in electronic engineering from Hokkaido University, Sapporo, Hokkaido. Upon graduation in 1980, he joined NTT as
researcher, where he researches the solid statephysics of thin film head, a key technology in thedevelopment of high capacity magnetic storage
devices. In 1989, he was responsible for thedesign and development of an electronic filingsystem for Video-on-Demand. He was a vicepresident and general manager at IP HQs in NTT
America from 1994 through 1999. His main mission was to create and promote Internet businessboth for US and Japan. He is a member of IEICE
of Japan and Distinguished Speaker of IEEE,
committee at ARIB (Association of Radio Industries and Businesses), INSTAC (Information
Technology Research and Standardization Center) and TV Anytime Forum.

NTT Technical Review