AI-assisted In-band Network Telemetry Framework
for Fast Network Failure Detection in Optical Core Networks
Badr Mochizuki（The Kyoto College of Graduate Studies for Informatics）
Abstract

This paper introduces the key challenges for building an AI-assisted framework for network
failure detection and localization in optical core networks based on in-band network telemetry data.
The main idea consists in feeding telemetry data as input for the AI module in order to achieve realtime network failure detection and localization.

■ 1．Introduction

and its implementation strategies for AI-assisted
dynamic orchestration of INT metadata collection

The aim of this paper is to discuss how toin order to realize failure detection andleverage In-band Network Telemetry (INT) [1] -localization in optical core networks. Finally,that is network state information inserted into

conclusions are presented in Section 6.

production ﬂow packets as header fields - for both
detecting. in real-time when a failure occurs. In
particular, this paper considers exploiting the low
latency of INT in order to maintain highavailability of the optical core data
communication networks. Three key challenges
can be identified: the development of a frameworkfor dynamic orchestration of INT metadata
collection without significantly degrading theoverall network performance in terms of
bandwidth and quality of service; thespecification of a lightweight method for real-time
detection of silent failures (subtle failures that
become harmful over time) from the collected INT
information; the realization of fast and accurate
network failure localization. The design of the
INT metadata collection framework will be ofparticular interest because it will serve a basis
for both network failure detection andlocalization.
I n t h e r e s t o f t h i s p a p e r, w e i n t r o d u c ebackground information in section 2 andsummarize the main research trends in network

Fig. 1: S pectrum grids for traditional opticalcore networks and EONs.

failure detection and localization in section 3. In
Sections 4 and 5, we discuss the key challenges

NAIS Journal

19

■ 2. Backgroundvariable in EON. In this figure, the grid size for

A. Optical core networks

GHz. Therefore, if a lightpath needs 70 GHz, two

traditional optical core networks is fixed to 50

In optical core networks, light is used togrids of size 50 GHz are used, and the remaining

transmit data over optical fibers at the speed of

30 GHz cannot be used by other lightpaths.

light, hence allowing ultrafast transmission andHence the grid is not efficiently utilized in

making it ideal for core networks that span overtraditional optical core networks. Moreover, twolong distances [2].

guard-bands per grid is always needed even if alightpath utilizes multiple grids. On the other
hand, in EON, the grid size is variable. Here, theoptical spectrum can be allocated per FS unit,
and only two guard-bands are needed perlightpath. As a result, the optical spectrum can beutilized more efficiently in EON.
B. Software Defined EONs
Recently, due to the expansion of datacenters
and the exponential increase in internet traffic,
optical core networks can no longer be controlled

Fig. 2: Software defined EONs.

with the traditional management framework.
Software defined networking (SDN) has been

Network operators are updating their

proposed as a promising solution because network

equipment on a regular basis in order to cope

operators can program their network devices

with the continuous increase in internet

much more freely. In SDN, the data plane and the

bandwidth. The network traffic is becoming

control plane are physically separated, and a

increasingly heavy and bursty as the popularity

centralized control plane can add, delete orof cloud computing, Internet of Things (IoT) andmodify forwarding rules on SDN switches [6].

5G mobile communications increases. However,This network architecture is illustrated in Fig. 2

adding new network equipment is costly and hasfor the case of software defined EON denoted asphysical limitations. As a promising cost-effective

SD-EON.

solution, Elastic Optical Networks (EON) have

In the data plane of SD-EONs, packets are sent

been proposed. EONs are optical circuit switchingby the optical switches on the fiber links following

networks where first a lightpath is established,

the predefined packet forwarding rules. On thethen data is transmitted all optically and finallyother hand, the control plane consists of an SDN

the lightpath is released [3]. Here, lightpaths arecontroller that translates network management

established and released dynamically. Please notepolicies into packet forwarding rules. These rulesthat data is transmitted through optical channels,

can be expressed as flow entries which arecalled spectrum grids, in the optical fibers [4].

installed on ﬂow tables at the SD-EON switches.

EON can support efficiently higher data bit rates

The interaction between the SDN controller andthan the traditional optical core networks, by

SD-EON switches is performed using

dividing a wavelength into multiple frequency

communication protocols such as OpenFlow [7].

slots (FSs). Note that FSs have to be allocatedadjacently (spectrum contiguity constraint) [5].

20

C. In-band Network Telemetry

Figure 1 shows that the spectrum grid size is

In-band network telemetry (INT) is the basis

fixed in traditional optical core networks and isfor a variety of applications such as virtual/

NAIS Journalaugmented reality, health monitoring and self-

research because it is an open-source anddriving cars. INT is also a relatively new network

permissively licensed language.

monitoring framework that operates at the data

P4 uses the concept of match-action pipelines

plane. By using programmable network devices,

as illustrated in Fig. 3. Packet forwarding in SDN

it is possible to collect in real-time INT metadata

switches is performed by table lookups and its

such as network devices internal states (e.g.,corresponding actions. Here, the parser, theswitch ID, queue occupancy) and network

ingress pipeline, the egress pipeline and theperformance metrics (e.g., latency). This can be

deparser can be modified by P4 programming. An

achieved by encapsulating information inside theincoming packet is first handled by the parser

packet as headers fields, which are interpreted bythat will extract the packet header, and the

network devices as instructions to collect INT

packet payload is buffered. The extracted header

metadata. Finally, the collected header fields areis then passed to the ingress pipeline that will

extracted and reported to a monitoring system,

process the packet header according to thewhich in turn analyses the INT metadata andprogrammed actions and lookup keys.

reacts if problems are detected in the network.

Furthermore, the ingress pipeline determines anConsequently, INT can significantly enhanceegress port and a queue into which the packet is

network-wide visibility, and network problems

placed. Next, the packet is passed to the egresssuch as network failure can be detected in apipeline which can be programmed to process thetimely manner [1].

packet in a specific manner. Finally, the packet’s
payload and headers are assembled andforwarded to the egress port by the deparser.

■ 3. Related Research
There is extensive research in the literature
about network failure detection and localization[9-11]. However, most of the proposed methods
are implemented in the control plane and hence
present a centralized solution to be implemented
Fig. 3: P4 forwarding model.

at the network controller. The main disadvantageof this approach is the high latency required forgathering network information at the network

D. Data plane programming using P4 language
In traditional SDNs, it is possible to modify the
behavior of the control plane but not that of thecontroller, localizing the network failure andpushing failure recovery instructions to the
network devices.

data plane. Concretely, routing policies can be

Furthermore, in recent years, there have been

flexibly defined, but the corresponding actions

several research work about using INT as acannot be flexibly executed. In order to resolve

network monitoring framework. While most of

this issue, a programming language for the data

the proposed research work has focused onplane named P4 (Programming Protocol-

proposing methods for using INT to debug

independent Packet Processors) has been

network events such as congestion or traffic

proposed by the inventors of OpenFlow in [8]. P4

bursts, little research work has addressed thecan express how incoming packets are processedchallenges about how to design an autonomousby the SDN switches, without modifying the

network monitoring framework as well as

control plane. It is currently widely used in

network failure detection and localization using

NAIS Journal

21

INT. Since network monitoring using INT

maximum transmission unit (MTU). Third, eachrequires collecting networking information

monitoring application may require different INT

regularly, data overhead is introduced in themetadata to be collected at different rates,

network every time INT metadata istherefore some INT metadata may need to be

encapsulated into packets. Hence, it is importantcollected at different rates (more or lessto consider the tradeoff between network

frequently) than others.

overhead and real-time network monitoring. A

The first and second problems can be solved bypromising solution that is aware of the above-

formulating an optimization problem thatmentioned tradeoff is to apply machine learning

maximizes the number of collected INT metadata.

techniques to dynamically and autonomously

For the third problem, a machine learning model

collect INT metadata. In addition, since INT iscan be designed and trained to identify whichused in the data plane, a distributed approach to

INT metadata is important to be collected based

network failure detection and localization is

on the monitoring application’s needs.

required, and the previously proposed centralized

This INT collection framework will constituteapproaches cannot be applied, hence new

the basis for the next key challenge and will haveapproaches have to be considered.

a decisive impact on performance in terms ofIn the next section, we discuss two key

delay and network visibility.

challenges for AI-assisted dynamic orchestration

22of INT metadata collection in order to realize

Key Challenge 2: How to detect and localize

failure detection and localization in SD-EONs.

network failures in real-time?
Network failures can range from silent failures■ 4. Key Challenges-where packets are dropped without notification-

Key Challenge 1: How to dynamically orchestratecausing network service unavailability. Silent

the collection of INT metadata?

failures can be caused by ﬂaws in the software or

to complete power loss of network devices,

Using INT to monitor the network state is oferrors in network configuration. While silent

particular significance because it is a powerful

failures do not cause immediate network service

tool that allows detecting network anomalies in

unavailability, damage can be inﬂicted over timereal-time. INT metadata are collected ands u c h as s e ve r e p e r fo r m an c e d e gr a da tion ,delivered to a network monitoring application.

thrashing memory, flaky I/O, and eventually

Each monitoring application is specialized in

network service unavailability. In addition, silent

identifying a specific network anomaly (e.g.

failure detection and localization can be time

malicious network attacks, congestion or network

consuming because no explicit notification is

failures) and reacts to it.

provided when the failure occurs.

In this section, the main purpose is to

In this section, two sub problems areorchestrate dynamically the process of INT

considered, namely lightweight failure detection

metadata collection in order to maximize network

and accurate network failure localization. In thestatus visibility. This is particularly challenging

first sub-problem, the main challenge consists inbecause of three main problems. First, if INT

devising a method that can detect early indicators

metadata are incorporated in all packets, aof silent failures based on the collected INT

significant bandwidth overhead will be generated,

metadata. This can be achieved by embedding

which in turn will affect the network performanceand collecting an exhaustive amount of INT

by introducing delays in communications. Second,

metadata in order to detect potential indicators of

the amount of INT metadata that can be

failures such as high queue occupancy or highembedded into packets is limited due to thepacket processing time. However, the bandwidth

NAIS Journal

overhead caused by those INT metadata may

number of collected INT metadata can bedegrade the network performance andachieved by solving an optimization problem

consequently increase the failure detection time

where the number of spatial and temporal

which in turn will delay the failure localization

dependencies are maximized. The objective

time. Therefore, the main difficulty consists infunction can be expressed as:

detecting silent failures with minimum INT
metadata in order to realize real-time detection.
In the second sub problem, the goal is to localize

Maximize,+precisely the silent failure. While it is possible to

The above optimization problem should

identify in which path a failure occurred, it isconsider several constraints including thedifficult to determine the exact link or network

network ﬂow capacity cannot be exceeded; an INT

device that is malfunctioning. Rerouting throughmetadata should be collected at most by one

alternative link-disjoint paths might be a

network ﬂow at a given network device.

temporary fix, however this could trigger

In the second problem, it is assumed that each

unwanted congestion or failures in other parts ofmonitoring application may give different

the network. Therefore, it is important to identify

importance to an INT metadata depending on theand locate precisely the failure immediately aftermonitoring application needs. In addition, thisit occurs.

importance may change dynamically dependingon the network state and time. As a solution to■ 5. Implementation strategies

this problem, a learning mechanism can be used
to guide the INT collection process based on the

Theme 1: Orchestration of INT metadata

importance of an INT metadata that could becollection using machine learning

quantified in weights. Furthermore, Machine

In this section, guidelines for solving thelearning models such as deep learning methods

problems presented in key challenge 1 arecould be used in the learning mechanism in order

discussed. The proposed framework can be

to infer those weights. The inputs for the learning

realized by solving two problems: Maximizing themechanism would include not only the collected

number of collected INT metadata and the

INT metadata (e.g. switch ID, ingress/egress port

identification of which INT metadata is importantID, packet processing time or queue length) but

to be collected.

also specific information related the network ﬂow

In the first problem, let’s consider aor the needs of the monitoring applications that

communication network N with D network

will analyze the collected INT metadata. Next, in

devices, a set of M monitoring applications, and aorder to keep an up-to-date network visibility, theset of I INT metadata. These INT metadata haveweights representing the importance of each INT

dependencies that are spatial – that is the INT

metadata must be updated periodically over time.

metadata has to be collected from a specificMoreover, some weights may be updated more

network device - or temporal if the INT metadatafrequently than others depending on how often a

has to be collected within a time limit. A spatial

monitoring application requires an INT

dependency s at device d for INT metadata i andmetadata.

monitoring application m is denoted as s

. In

Two main advantages are provided with thisaddition, a temporal dependency t for INT

framework: bandwidth overhead is reduced bymetadata i and monitoring application m is

maximizing the number of collected INT

denoted as t . Therefore, this problem can be

metadata at a time interval; and an accurate

formulated as a Mixed Integer Linear

network-wide visibility can be achieved by

Programming model. The maximization of theidentifying which INT metadata is important.

i
d,m

i
m

NAIS Journal

23

Finally, the collected INT metadata by theabove-mentioned framework will serve as a basis
for lightweight detection and accurate localizationB）Accurate Failure localization
After a failure is detected, a timed-out path ispassed to the failure localization module which isof silent failures.

then executed in order to determine the precise

Theme 2: Lightweight detection and accurateThis module can localize failures through the use

localization of silent failuresof backward probing in three steps. First, an INT

In this section, the approach for solving the
problems indicated in Key Challenge 2 ispresented. Here, the development of a monitoring
application at each network device is proposedand consists of two modules: lightweight failure
detection module and accurate failure localization

location of the failure along the timed-out path.

probe packet is sent through the timed-out path.
Second, at each intermediate device, the status of
the egress port is checked. If the status is UP, theprobe packet will be forwarded to the next
intermediary device without embedding any
i n f o r m a t i o n i n t h e p r o b e p a c k e t h e a d e r.

module.

Otherwise, if the egress port status is DOWN, theA）Lightweight failure detection

probe packet’s header. Third, the probe packet isThis module will analyze the INT metadata
collected by the INT framework designed intheme 1. Specifically, INT metadata such as
switch ID, ingress and egress port ID are
considered. This module will create locally a“Path table”to store all active paths to allreachable destinations, where each table entry
corresponds to a path to a single destination.
Therefore, there may be multiple table entries fora single destination if there are multiple feasible
paths to that destination. In addition, each tableentry may hold switch ID, ingress and egress port
ID for each intermediate network device of thespecified path. Furthermore, each table entry will
be stored temporarily by setting an aging time inorder to limit the memory usage at each network
device. Here, the table entries are updated before
the aging time has elapsed based on the INT
metadata provided by the INT collection
framework. If the lightweight detection module
does not receive the required INT metadata fromswitch ID and egress port ID are embedded in thesent through the same ingress port back to thesource device. Please note that no additional
information is embedded in the header when theprobe packet is sent backwards to the source.
Finally, the failure localization module will pass
the switch ID and the port ID where the failure
occurred to a failure recovery module.

■ 6. Conclusion and Future work
The strategies mentioned in section 5 will be
confirmed through practical experiments using
the network simulator Mininet [12], and INT
data collection procedures will be implemented in
P4. Furthermore, the P4 program will be written
for the V1Model architecture implemented on
P4.org's bmv2 software switch, using theexperimental setup provided in the P4 tutorial[13]. As a future work, a network failure recovery
module will be implemented in order tocomplement failure detection and localization.

the INT collection framework before the aging
time has elapsed, then all the timed-out path
entries are deleted and a failure is detected atthose paths. Please note that the aging time will
be changed dynamically based on the frequency
of INT metadata collection that is regulated by
the machine learning mechanism used in theframework presented in theme 1.

24

NAIS Journal[References][1] L. Tan, W. Su, W. Zhang, J. Lv, Z. Zhang, J. Miao, X.
Liu, and N. Li, “In-band network telemetry: asurvey”, Computer Networks, vol. 186, pp. 1-24, 2021.
[2] B. C. Chatterjee, N. Sarma, P. P. Sahu, and E. Oki,
“Limitations of Conventional WDM Optical Networks
and Elastic Optical Networks for Possible Solutions”,
Routing and Wavelength Assignment for WDM-based

Optical Networks, Springer, Cham, vol. 410, pp. 101115, 2017.
[3] O. Gerstel, M. Jinno, A. Lord, and S. J. B. Yoo, “Elastic
Optical Networking: A New Dawn for the Optical
Layer?”, IEEE Communications Magazine, vol. 50, no. 2,
pp. S12–S20, 2012.
[4] B. C. Chatterjee, N. Sarma, and E. Oki, “Routing andSpectrum Allocation in Elastic Optical Networks: A
Tutorial”, IEEE Communications Surveys & Tutorials,
vol. 17, no. 3, pp. 1776-1800, 2015.
[5] M. Recalcan, F. Musumeci, M. Tornatore, S. Bregni,
and Achille Pattavina, “Benefits of Elastic Spectrum
Allocation in Optical Networks with Dynamic Traffic”,
IEEE Latin America Transactions, vol. 13, no. 11, pp.
3642-3648, 2015.
[6] B. Nunes, M. Mendonca, X.-N. Nguyen, K. Obraczka,
and T. Turletti, “A survey of software-defined
networking: Past, present, future of programmablenetworks”, IEEE Communications Surveys & Tutorials,
vol. 16, no. 3, pp. 1617–1634, 2014.
[7] O p e n N e t w o r k i n g F o u n d a t i o n , h t t p s : / / w w w.
opennetworking.org[8] P. Bosshart et al., “P4: Programming ProtocolIndependent Packet Processors”, ACM SIGCOMM
Computer Communication Review, vol. 44, no. 3, pp. 88–
95, 2014.
[9] PC Fonseca and ES Mota, “A survey on fault
management in software-defined networks”, IEEE
Communications Surveys & Tutorials, vol. 19, no 4, pp.
2284-2321, 2017.
[10] F. Musumeci, C. Rottondi, G Corani, S. Shahkarami,F, Cugini, and M. Tornatore, “A Tutorial on Machine
Learning for Failure Management in Optical
Networks”, Journal of Lightwave Technology, vol. 37, no.
16, pp. 4125-4139, 2019.
[11] J. Ali, G. Lee, B. Roh, D.K. Ryu, and G. Park,“Software-Defined Networking Approaches for Link
Failure Recovery: A Survey”, Sustainability, vol. 12, no.
10, 2020.
[12] Mininet: http://mininet.org/[13] P4 Tutorial: https://github.com/p4lang/tutorials

◆著者紹介

望月

バドル

Badr Mochizuki

京都情報大学院大学 講師
福井大学大学院工学研究科修了 工学博士
奈良先端科学技術大学院大学情報科学研究科修了 工学修士
元 CNRS 研究所（フランス） 研究員
Al Akhawayn 大学（モロッコ） 工学士

NAIS Journal

25