Special Feature

Operations Systems for Type-X and theMPLS Network
Kenichi Oshikiri, Yasushi Honma, Toshio Oimatsu†,
and Toshihiro Nishizono
Abstract
One of the keys to maintaining a carrier-class multiprotocol label switching (MPLS) network system isan operations system for designing and managing highly reliable large-scale MPLS paths and achieving
a managed network. This article describes the operations system (X-OSS) developed to achieve this goal.

1. X-OSS system architecture

paths (LSPs), and establish them in Type-X. Designand management by X-OSS ensures a flexible MPLS
path network to satisfy high quality and reliability.
This X-OSS consists of the four systems shown inFig. 1.
1) The Type-X element management system (XEMS) manages (i.e., monitors and controls) theType-X routers and supervises network topology
data (the connection between router interfaces). It
also establishes the MPLS paths (LSPs) accord-

The Type-X operations support system (X-OSS)
monitors and controls Type-X routers and manages
network resources to design the multiprotocol label
switching (MPLS) network paths, i.e., label switched† NTT Network Service Systems Laboratories
Musashino-shi, 180-8585 Japan
E-mail: oimatsu.toshio@lab.ntt.co.jp

Design path and manage
bandwidth of QoS-guaranteed LSP
Monitor and manage real traffic
Trafficmanagement system
X-TM

Feed back to design

LSP setup data

7
Operator
Topologydata

QoS server
X-QS
1Monitor andcontrol
router/network2Type-Xedge router

33···

Type-X
core router

Generalrouter

Policy server
X-PS

Set up LSP(logical data)

Type-X element management system
X-EMS

Collect traffic 6
data

Set up and manage
policy for edge router435

Set uppolicySet up LSP (foreach router)Type-Xedge router
GeneralrouterCore network

Fig. 1. Entire X-OSS architecture and MPLS management flow.

Vol. 1 No. 5 Aug. 2003

63

Special Feature

ing to the end-to-end logical LSPs setup requests
from X-QS.
2) The quality of service server (X-QS) systemassigns, sets up, and manages the routes andbandwidth of LSPs in the core network based on
the network topology data managed by X-EMS.
3) The policy server system (X-PS) sets up and manages policy information in the edge-router based
upon LSPs designed by X-QS.
4) The traffic management system (X-TM) collects
and manages actual traffic data from each line
interface and each LSP. This traffic data enables
operators to compare predicted bandwidth withactual traffic volumes and feed the results back inorder to redesign the line and LSP bandwidths.
2. X-EMS
2.1 X-EMS equipment management function
The Type-X-dedicated equipment management
functions are added to the OpenViewTM IP network
management function, allowing the Type-X router tobe controlled easily through a graphical user interface(GUI) (Fig. 2). Features include an alarm display, afault display in each router, and equipment management based on mounted devices.
2.2 X-EMS MPLS management function(1) Topology management and LSP setup
X-EMS provides X-QS with the network topology data used for IP network management. It alsotranslates logical LSP setup requests from X-QS
into actual requests that set up LSPs to each router.

It also collates the LSP identifier (equivalent to theMPLS label value) of each router with the logical
LSP identifier of the entire network designated by
X-QS, so that differences in the way LSPs are setup among various kinds of routers can be masked.
(2) Faulty LSP group management
X-EMS can also manage the status of each LSP
by combining LSP setup data with its monitoring
function. X-OSS sets two or more LSPs that takethe same route in a Type-X router as one LSPgroup, and when a fault occurs in a certain LSP
group, the Type-X router automatically switches tothe secondary LSP group. The LSP group list is displayed on the screen, so an operator can easily identify the fault status (normal or faulty) of the relevant
LSP group, along with the type (primary or secondary) of the route currently in use. After the system has recovered from the fault, the LSPs that havebeen switched to the standby routes can be safely
switched back while the status of each LSP in therelevant group is monitored (Fig. 2).
3. X-PS/QS
3.1 Basic functions
The X-QS system designs LSPs according to each
QoS class (expedited or assured forwarding: EF/AF).
This system manages network resources such as network topology data and link bandwidth under each
QoS class, and the queue buffer size in a router; thisenables the design and setup of the optimum MPLS
path (LSP) based on user-defined destination, band(1) X-EMS equipment management window
This window displays router and network
alarms and the router’s installation drawings.

(2) Faulty LSP group management window
This window displays the state of the LSPgroup (normal or abnormal) and routes(primary and secondary) used.

Fig. 2. Examples of X-EMS windows.

64

NTT Technical Review

Special Feature

width, QoS class (EF/AF), and objective functions(e.g., design requirements such as the minimum number of hops and minimum delay).
On the other hand, user traffic in LSPs must be regulated to ensure congestion-free, stable data transfer.
To meet this requirement, X-PS compares the bandwidth of LSPs with that of individual user traffic todetermine whether user traffic can be transferred via
certain LSPs or not. It also manages and sets up thisuser traffic and LSP-mapping information or policy
information (Fig. 3).
3.2 QoS management
The X-QS is extended to manage the queue buffer
in a router, whereas a general design system manages

only link bandwidth between routers as a network
resource. IP packets are actually discarded or delayedwhen traffic on a specific line card (LINF: Line Interface) in the router is congested. Queue managementin the router, therefore, is essential for tight QoS control. Figure 4 shows the way to manage an internalrouter to meet this tight requirement. X-QS calculates
and assigns the size of the queue buffer for the router
output and the weight of each buffer based on therouter hardware configuration, the status of mounted
devices, and the requirements for each QoS class inorder not to discard any data packets. This method
guarantees tight QoS.
3.3 Highly reliable LSP configurationX-QS can design three types of LSPs, as shown inFig. 6 on page 51 to handle network faults.
(1) Primary LSP: active LSP in normal condition(2) Secondary LSP: a backup to a primary LSP if theprimary LSP fails. This LSP has a different routeat the same ingress/egress of the primary LSP.
(3) Sub-gate LSP: standby LSP ready for aningress/egress router of the primary/secondary
LSP or access network fault. This LSP has a different route and different ingress/egress from theprimary LSP.
Combining these three types of LSP makes it possible to select various types of redundancy.

Fig. 3. Examples of X-PS/QS windows.

Possible
generalscope

Possible PS/QS scope
Type-XLINF 1LINF 4LINF 2

LINF 5

Physical/logical link

Physical/logical link(4) Output queue
calculates queue output
rate to satisfy QoS
requirements and read ratio.

(1) Physical/logical link
manages bandwidthin link.

PKG n(2) Input queue
calculates queue output
rate to satisfy QoS
requirements and read ratio.

LINF NSWB(3) Switch port
manages transfer capacity foreach combination of LINFs.

Fig. 4. QoS management coverage.

Vol. 1 No. 5 Aug. 2003

65

Special FeatureComplete bandwidth
guarantee

Partial-guarantee

Non-guaranteed

Bandwidth exclusive type

Bandwidth sharing typeNon-guaranteed type

Individual bandwidth
allocation

Shares bandwidth
in common section

Allocates zero(0) for bandwidth

LSP-A: 10M

LSP-A: 10M

LSP-A: 10M

15 M (10+5)

10 M (10>5)0M

LSP-B: 5M

LSP-B: 5M

LSP-B: 5M

M: bit/s

Primary LSPSecondary LSP

Type X

Fig. 5. Standby LSP bandwidth management method.

3.4 Efficient design of standby LSP bandwidth
The standby LSPs use the following three bandwidth management methods (Fig. 5):(1) Bandwidth exclusive type (1:1): The bandwidthof the primary LSP is guaranteed in the standby
LSP.
(2) Bandwidth sharing type (N:1): A standby LSP’sbandwidth is shared with other standby LSPs.
(3) Non-guaranteed type: A standby LSP’s bandwidth is not guaranteed (zero allocation).
Although the bandwidth exclusive type needs thelargest bandwidth, it can virtually provide leased circuit services. Conversely, the non-guaranteed type,which is a best-effort-type transfer service, needs
zero bandwidth for the standby LSPs.
Combining these bandwidth management methods
with the standby LSP configuration in (3) of 3.3
achieves a wide range of reliable LSPs according toquality and economic requirements.
3.5 Highly reliable design management
Discrepancies between the data of designed LSPs
in X-PS/QS and that of LSPs set in the actual router
could cause network mismanagement. Therefore, ahighly reliable audit function is implemented to collate this data.
3.6 Efficient design of large-scale LSPs
Designing and setting data for a large number ofLSPs individually using the GUI needs a tremendous
amount of effort. To reduce the effort of designinglarge-scale networks, X-PS/QS has functions forimporting large amounts of LSP data from text files.

66

4. X-TM
4.1 Basic functions
X-TM is the IP/MPLS network traffic data collection system based upon InfoVistaTM. It can collecttraffic data from not only Type-X series routers but
also other commercially available ones. The platform
customization functions allow users to create and add
user-defined traffic items, spreadsheets, graphics layout design, reports, statistical forms, and so forth. Inaddition, a standard report (Fig. 6) and standard form(Fig. 7) are prepared in advance to support the management of a carrier-class large-scale network.
4.2 Line traffic data collection
X-TM has forms for reporting traffic data, identifiers of lines that exceed a traffic threshold, and thetop 10 band usage rates. It can display this data graphically in real time. In addition, the daily, weekly, andmonthly data displays also provide easy identification of short-, medium-, and long-term traffic characteristics. Some forms are explained below.
(1) Top 10 form
This form lists the top 10 band usage rates atthousands of observation points, helping users tounderstand the trend of network traffic and findpotential network congestion effectively.
(2) Excess line threshold form
This form indicates the number of discarded anderrored packets, exceeding the preset threshold ineach line (at each collection point), facilitating a
speedy response to user feedback and fault prevention.

NTT Technical Review

Special Feature

Report shows trend.

Fig. 6. Example of standard X-TM report.

4.3 Traffic data collection in each LSP
Like traffic data collection in each line, X-TM cancollect traffic data of each LSP from Type-X routersand display it graphically (violation of committedrate, violation of peak rate, etc). This enables users torecognize the real traffic status of the designed LSP
bandwidth and helps them to optimize the LSP design(i.e., by bandwidth re-allocation).

Cross-sectional compilation and statistical analysis
to collect results for a form listing the top 10 lines
with highest line utilization rate, for example.

Fig. 7. Example of standard X-TM form.

5. Experience to date and future plans
X-OSS has been used in an operating company
since fiscal 2001. It has already designed and managed thousands of LSPs, and is continuing to design
and manage a growing number of LSPs. Based on its
operational results, we are planning to enhance XOSS and support other commercially available
routers as a multi-vendor carrier-grade MPLS pathdesign system.

Kenichi Oshikiri
Engineer, First Promotion Project, NTT Network Service Systems Laboratories.
He graduated from the Nippon Engineering
College, Tokyo. He joined NTT in 1992. Initially, he worked on maintenance operations oftransmission systems. He is currently engaged indeveloping IP network operation support systems.

Toshio Oimatsu
Senior Research Engineer, Network Management Development Project, NTT Network Service Systems Laboratories.
He received the B.S. and M.S. degrees in electronic engineering from Toyama University,Toyama, in 1983, and 1985, respectively. Sincejoining the Electrical Communication Laboratories, NTT, Tokyo, Japan, he has been involved inresearch and development of STM/ATM network operation support systems. He is currently
developing IP network operation support systems. He is a member of the Institute of Electronics, Information and Communication Engineers (IEICE) of Japan.

Yasushi Honma
Engineer, Network Management Development
Project, NTT Network Service Systems Laboratories.
He graduated from the department of electricalengineering of Tokyo Metropolitan College ofTechnology, Tokyo, in 1994. Since joining theNetwork Service Systems Laboratories, NTT,Tokyo, Japan in 1998, he has been involved inresearch and development of ATM network
operation support systems. He is currently developing IP network operation support systems.

Toshihiro Nishizono
Senior Research Engineer, Supervisor, Network Software Service Project, NTT Network
Service Systems Laboratories.
He received the B.S., M.S., and Ph.D. degrees
in information engineering from Kyushu University, Fukuoka, in 1978, 1980, and 1990, respectively. In 1980, he joined the Electrical Communication Laboratories, Nippon Telegraph andTelephone Public Corporation (now NTT),Tokyo, Japan. He worked on the development ofDDX packet switching systems and ATM
switching systems. From 1990 to 1993, heworked in Advanced Telecommunication
Research Laboratories, Kyoto, Japan, on software specifications and distributed operating
systems. From 1998 to 2001, he was engaged ininternational communication service deploymentand IP service development in NTT Communications Corporation, Tokyo, Japan. He is currently working on IP control and management
service research.

Vol. 1 No. 5 Aug. 2003

67