Patent Description:
Wireless communication networks such as <NUM>, <NUM>, and future <NUM> networks, are dependent on network synchronization and often provide time-sensitive services that depend on the network being synchronized, i.e. time and frequency alignment between radio access network, RAN, nodes in the network.

Specific requirements for RAN timing and synchronization depend on parts of the radio spectrum used and deployed radio technology.

Examples of requirements according to the used spectrum comprise a relatively loose requirements for over the air frequency synchronization in case of Frequency Division Duplex, FDD, wherein FDD-based <NUM> and LTE networks can survive lengthy (><NUM> hour) loss of synchronization; on the other hand, to ensure no interference between uplink and downlink Time Division Duplex, TDD, a much tighter time and phase synchronization is required.

Examples of requirements dependent on the radio technology deployed comprise a relative time sync ≤<NUM> among neighboring radio nodes in case of deployment of coordinated RAN features such as Coordinated Multipoint, CoMP and absolute time sync ~<NUM> across the whole network in case of beamforming with NR-TDD. In general, a mix of absolute (across the whole network) and relative (among neighboring radio nodes) criteria requires a time sync of ~<NUM> or less to support a network's optimal spectrum utilization and advanced RAN features.

There is also a need for increased reliability in the timing source. While today's FDD-based LTE network can continue to operate for hours after sync loss with no degradation, in the future, loss of timing may have an immediate impact on RAN performance. A solution of deploying at least one Global Positioning System, GPS, receiver at each site is not competitive in terms of cost. The growing trend is to use a transport network to synchronize the RAN. This, in turn, implies the need to deploy routers and fronthaul units that fully support timing and sync transport requirements via PTP protocols such as IEEE1588, while delivering the highest level of performance. A further problem is that RAN nodes may be implemented using different technologies and have different level of compliance to a standard, either because being manufactured by different companies or because designed by different branches of the same company. In such a scenario, timing errors may occur for clocks of the RAN nodes in the network towards a common reference time base, i.e. a global clock.

A synchronization misalignment between different RAN nodes is difficult to spot because the synchronization paths from a source Global Navigation Satellite System, GNSS, to RAN nodes can be perfectly functional with no active alarms along the synchronization chain, but the synchronization among RAN nodes can be disrupted due to the accumulation of delay asymmetries in some links. This kind of issues can be spotted at a radio interface with specific instruments or by means of new features implemented in a RAN node for statistically comparing phases of network synchronization at the Radio Interface. However, the absence of alarms along the synchronization chain may require intervention of domain experts for network troubleshooting to locate the synchronization failure. In case of alarms, the analysis of alarm and performance logs on the RAN nodes along the synchronization chain may allow identification of the most probable causes of the fault; this activity is generally carried out manually by domain experts and it is generally time consuming.

Document <CIT> discloses a time synchronization anomaly detection process executing one or more machine learning-based analyser to detect time synchronization anomalies along a path in a deterministic network.

An object of the invention is to enable detection and localisation of a synchronization problem and/or root cause analysis of synchronization issues among a plurality of nodes operating in a communications network.

To achieve said object, according to a first aspect of the present invention there is provided a method for monitoring a synchronization status among a plurality of nodes operating in a communications network. The method of this first aspect comprises obtaining information associated with a plurality of nodes, the nodes having synchronized clocks, wherein an information for a node is indicative of one or more parameter values. The method also comprises creating a first dataset based on the obtained information associated with the plurality of nodes, the first dataset comprising time-aligned samples of the one or more parameters for a time interval; creating a second dataset in form of a plurality of time-slices comprising the samples from the first dataset and statistics determined based on the samples from the first dataset calculated for the time interval. The method further comprises running a first Machine Learning, ML, model for anomaly detection on the first dataset, the first ML model generating first labels identifying anomalous nodes; and identifying anomalous nodes and time-slices when the anomalous nodes exhibit anomalous behavior using the first labels and the second dataset to determine relationships between the anomalous nodes and other nodes of the network.

According to a second aspect of the present invention there is provided a device for monitoring a synchronization status among a plurality of nodes operating in a communications network. The device comprising a processor and a memory, the memory having stored thereon instructions executable by the processor. The instructions, when executed by the processor, cause the device to obtain information associated with a plurality of nodes, the nodes having synchronized clocks, wherein an information for a node is indicative of one or more parameter values. The device is operative to create a first dataset based on the obtained information associated with the plurality of nodes, the first dataset comprising time-aligned samples of the one or more parameters for a time interval; create a second dataset in form of a plurality of time-slices comprising the samples from the first dataset and statistics determined based on the samples from the first dataset calculated for the time interval. The device is operative to run a first Machine Learning, ML, model for anomaly detection on the first dataset, the first ML model generating first labels identifying anomalous nodes; and identify anomalous nodes and time-slices when the anomalous nodes exhibit anomalous behavior using the first labels and the second dataset to determine relationships between the anomalous nodes and other nodes of the network.

According to a third aspect of the present invention there is provided a computer program comprising instructions which, when run in a processing unit on a device, cause the device to obtain information associated with a plurality of nodes, the nodes having synchronized clocks, wherein an information for a node is indicative of one or more parameter values; create a first dataset based on the obtained information associated with the plurality of nodes, the first dataset comprising time-aligned samples of the one or more parameters for a time interval; create a second dataset in form of a plurality of time-slices comprising the samples from the first dataset and statistics determined based on the samples from the first dataset calculated for the time interval; run a first Machine Learning, ML, model for anomaly detection on the first dataset, the first ML model generating first labels identifying anomalous nodes; identify anomalous nodes and time-slices when the anomalous nodes exhibit anomalous behavior using the first labels and the second dataset to determine relationships between the anomalous nodes and other nodes of the network.

According to a fourth aspect of the present invention there is provided a computer program product comprising a computer readable storage medium on which a computer program, as mentioned above, is stored.

In an embodiment the method may comprise running an explainer model on the first ML model and the first dataset, wherein an output of the explainer model is a plurality of feature importance vectors for the time interval; obtaining a plurality of second labels associated with the feature importance vectors for the time interval, wherein a second label indicates a type of anomaly; running a second ML model taking the plurality of the feature importance vectors as an input; and identifying the type of anomaly of the anomalous nodes.

In an embodiment the method may comprise running a third ML model on the first dataset and a plurality of associated first labels; running an explainer model on the third ML model and the first dataset, wherein an output of the explainer model is a plurality of feature importance vectors for the time interval; obtaining a plurality of second labels with the feature importance vectors for the time interval, wherein a second label indicates a type of anomaly; running a second ML model taking the plurality of the feature importance vectors as an input; and identifying the type of anomaly of the anomalous nodes.

In an embodiment the method may comprise obtaining a score as an output of the first ML model for the time interval, wherein the score indicates a degree of the anomaly behavior.

In an embodiment the information associated with a plurality of nodes is obtained asynchronously.

In an embodiment the method may comprise creating the first dataset by associating one or more of the obtained parameter values with a node for a time-slice.

In an embodiment the first dataset comprises statistics obtained from a node.

In an embodiment the method may comprise creating the second dataset by evaluating one or more performance metrics associated with a node for a time-slice, wherein the evaluating is done based on the obtained parameter values.

In an embodiment the second dataset comprises information defining time and/or frequency synchronization topology of the network.

In an embodiment the nodes are synchronized by Precision Time Protocol, PTP, or Network Time Protocol, NTP.

In embodiments of the device for monitoring a synchronization status among a plurality of nodes, the device is operative to carry out the embodiments of the method described above.

For better understanding of the present disclosure, and to show more readily how the invention may be put into effect, reference will now be made, by way of example, to the following drawings, in which:.

Embodiments will be illustrated herein with reference to the accompanying drawings. These embodiments are provided by way of example so that this disclosure will be thorough and complete, and will fully convey the scope of the inventive concept to those skilled in the art.

Fault identification, fault localization, and root cause analysis are aspects to address to guarantee reliability of a distributed synchronization function. However, current synchronization troubleshooting tools may detect the presence of a synchronization issue, but lack in providing information on the location and cause of the fault in a complex network of heterogeneous nodes.

An anomalous behavior of a node in a communication network may be due to a local error or because one or more nodes providing synchronization are having problems. By using a Machine Learning, ML, model and information on nodes and on evolution of a network synchronization topology of the nodes, a device may detect, localize, and identify anomalous nodes and a root of the anomalous behavior. The disclosed invention reduces domain expert intervention and time to solution, and provides scalability in case of increasing number and heterogeneity of nodes in the network.

Figure la schematically shows an example of a communications system in which the present invention may be practiced. The illustrative communications system comprises a device <NUM> for monitoring a synchronization status among a plurality of nodes <NUM>-<NUM>, the plurality of nodes <NUM>-<NUM>, and a network management system, NMS <NUM>. While for simplicity and brevity of the disclosure in the drawings the NMS <NUM> is represented as a single network element, it would be clear to a person skilled in the art that a network management system may be implemented using several network elements.

The plurality of nodes <NUM>-<NUM> may be hardware devices connected to form networks and implemented in any type of technologies, these nodes may perform various functions in the network, such as baseband, router, microwave system, and packet fronthaul. The plurality of nodes <NUM>-<NUM> may or may not exhibit an anomalous behaviour. This anomalous behaviour may be due to, for example, interrupted connections, out of service events, poor quality of synchronization that a node can provide. The nodes <NUM>-<NUM> may be synchronized by a networking protocol for clock synchronization, such as Precision Time Protocol, PTP, for data plane, and Network Time Protocol, NTP, for control plane.

The device <NUM> and the NMS <NUM> may be hosted on the same physical device, such as a router, gateway, and any device with computing, storage, and network connectivity to the communication network when active, as shown in <FIG>. In another embodiment, the devices <NUM> and the NMS <NUM> may be stand-alone physical devices. Furthermore, functionalities performed by device <NUM> and the NMS <NUM> may be performed in a plurality of physically separated nodes arranged in a cloud environment or by a centralized entity.

In a preferred embodiment, the NMS <NUM> executes management applications that monitor and control the plurality of nodes, <NUM>-<NUM>, communicates with the plurality of nodes <NUM>-<NUM>, and collects information associated with the plurality of nodes <NUM>-<NUM>, through a network monitoring protocol, such as Simple Network Management Protocol, SNMP, IP Flow Information Export, IPFIX, and Sample Flow, sFlow.

The device <NUM> may receive information collected by the NMS <NUM> as shown in <FIG> through a protocol such as Secure Shell/ Transport Layer Security, SSH/TLS, File Transfer Protocol, FTP/SFTP, Secure Copy, SC; or the device <NUM> may receive information from the plurality of nodes <NUM>-<NUM> as shown in <FIG>, through a protocol such as SSH/TLS, SNMP, NETCON.

<FIG> shows a method for monitoring a synchronization status among a plurality of nodes <NUM>-<NUM> operating in a communications network. The method may be carried out by a device <NUM>.

Referring to the method in <FIG>, in step <NUM>, the device <NUM> obtains <NUM> information associated with a plurality of nodes <NUM>-<NUM>. According to an embodiment, the information associated with the plurality of nodes <NUM>-<NUM> may be obtained asynchronously, i.e. the process of reading (capturing, obtaining) the information is not synchronized amongst the nodes. The device <NUM> may obtain the information from an NMS <NUM> that previously collected the information from the nodes <NUM>-<NUM> or directly from the nodes <NUM>-<NUM> as illustrated in <FIG>, respectively.

The information obtained for a node (one of the nodes <NUM>-<NUM>) is indicative of one or more parameter values characterizing operation of the node at a point in time. A parameter value may vary continuously over time or be discrete. Examples of parameter values in case of PTP protocol are indicated in ITU-T G. <NUM>, such as defaultDS. clockIdentity that uniquely identifies a clock; currentDS. stepsRemoved that indicates a distance between a slave clock and a grandmaster clock in the network; parentDS. parentPortIdentity that specifies a value of a portIdentity of a port on a master node that issues a Sync messages used in synchronizing the local clock; parentDS. grandmasterIdentity that indicates the clock identity for the grandmaster clock; portDS. portIdentity. clockIdentity that specifies the values of clockIdentity and portIdentity for each node port; portDS. portState that indicates the PTP port state for each node port (Master, Passive, Uncalibrated, Slave). The parameter values have been sampled at a time interval, such as <NUM> minutes.

In step <NUM>, the device <NUM> creates <NUM> a first dataset based on the obtained information associated with the plurality of nodes <NUM>-<NUM>. The first dataset comprises a plurality of tables associated with the plurality of nodes <NUM>-<NUM>. A table is associated with a node. An entry of the table associated with the node comprises time-aligned samples of the one or more parameters associated with the node, wherein the one or more parameters sampled at a point in time have been mapped to a corresponding time interval. The table associated with the node comprises one or more entries referring to one or more time intervals. The plurality of tables may comprise different parameter values. According to an embodiment, the first dataset may further comprise statistics associated with the node <NUM>-<NUM>. The statistics may be obtained directly from the node <NUM>-<NUM> and may be calculated based on parameter values collected at the node during a time interval and characterizing operation of the node. Obtaining statistics from the node <NUM>-<NUM> instead of parameters for determining the statistics may reduce the number of transmissions between the node <NUM>-<NUM> and the device <NUM>, and processing time to determine the statistics. However, in an alternative embodiment at least some parameters for determining the statistics may be received from the node.

<FIG> shows an example of a table of a first dataset with <NUM> entries corresponding to different time-slices of the day <NUM>-<NUM>-<NUM> in which parameters and statistics have been collected for a node with IP address <NUM>. The parameters and statistics comprise BestMasterChange that indicates whether a Best Master Clock Algorithm, BMCA, selected another Slave port or not in a time-slice, localSlavePort that indicates which port the node received synchronization information from at the time of sampling, averageMasterSlaveDelay, averageMeanPathDelay, averageOffsetFromMaster, currentUtcOffset, currentUtcOffsetValid, frequency Traceable, grandmasterClockQuality. clockAccuracy, offsetFromMaster. Further parameters and statistics may also be obtained from a node and their examples can be found in IEEE - P1588/D1. <NUM> Annex M. <FIG> also shows that similar data can be obtained for other nodes in the network - IP addresses of the other nodes are specified at the bottom of the table (e.g. <NUM>. <NUM>, <NUM>. <NUM>, etc.), but the data obtained is not shown.

In step <NUM>, the device <NUM> creates <NUM> a second dataset in a form of a plurality of time-slices comprising samples from the first dataset and statistics determined based on the samples from the first dataset calculated for the time interval. The second dataset may also comprise statistics of the first dataset. The samples and the statistics represent features for the second dataset.

According to an embodiment, the second dataset is created by aligning in time the plurality of tables of the first dataset. The time alignment may be done in respect of the same frequency of the statistical data available at the NMS <NUM> and it may be required if the nodes <NUM>-<NUM> are implemented using different technologies. The time alignment of the plurality of tables associated with the plurality of nodes is ensured by the use, for example, of NTP as synchronization protocol for the control plane of the plurality of nodes <NUM>-<NUM>, specifically for the synchronization of performance and alarm collection.

The second dataset provides a network time and/or frequency synchronization topology for each time-slice, i.e. a topological snapshot of the network from a synchronization point of view. The network synchronization topology is a directed graph describing a hierarchical master-slave architecture for clock distribution. The network synchronization topology may dynamically change in each time-slice, depending on anomalous events, such as interrupted connections, out of service events, poor quality of the synchronization that a node can provide.

<FIG> shows an example of a second dataset with entries for ten nodes referring to <NUM> time-slices (called slice in <FIG>) and comprising parameter values characterizing operation of the node and statistics (called attributes in <FIG>) such as BestMasterChange, LocalSlavePort, averageMeanPathDelay, clockQualitty. clockClass, clockState, grandmasterClockQuality. clockClass, grandmasterIdentity, maxOffsetFromMaster, meanPathDelay, offsetFromMaster, offsetFromMasterStdDev, parentPortIdentity. clockIdentity.

In step <NUM>, the device <NUM> runs <NUM> a first Machine Learning, ML, model for anomaly detection on the first dataset. The first ML model may be an unsupervised ML model, such as Isolation Forest. The first ML model takes as an input the first dataset and generates as an output first labels identifying anomalous nodes. A first label indicates if a node <NUM>-<NUM> for a time interval exhibits an anomalous behavior or not.

In step <NUM>, the device <NUM> identifies <NUM> nodes <NUM>, <NUM> and <NUM> as anomalous as well as time-slices during which the nodes <NUM>, <NUM> and <NUM> exhibit anomalous behavior by using the output of the first ML model. Moreover, the device <NUM> uses the second dataset to determine relationships between the anomalous nodes <NUM>, <NUM> and <NUM> and other nodes <NUM>-<NUM> of the network synchronization topology. The knowledge of the nodes <NUM>, <NUM> and <NUM> that exhibit anomalous behavior provided by the first ML model and the knowledge of the dynamic variation of the network synchronization topology provided by the second dataset provide an operator or user with information about which node originated the problem in the network synchronization topology. In particular, the second dataset provides information about how the anomalous node is correlated with the other nodes in the network synchronization topology, wherein some of the other nodes may exhibit an anomalous behavior. In case of PTP protocol and as defined in in ITU-T G-<NUM>, for example, a network synchronization topology represents a hierarchical master-slave architecture, wherein nodes may be masters or slaves. A master is a node distributing time to slaves and may also be a Grandmaster, GM, that gets its time from a primary reference source, such as a GPS satellite signal. A slave is a node that is remote from the master and is synchronizing to it. For example, if a slave node exhibits an anomalous behavior, the root of the anomaly may be the master or grandmaster node. And this may be understood thanks to the knowledge of the network synchronization topology at different points in time.

An example of the use of the second dataset is herein provided by referring to <FIG>. The two time-slices refer to a <NUM> minutes interval of the day <NUM>-<NUM>-<NUM>. Supposing that an output of the first ML model indicated node <NUM>. <NUM> as anomalous, the second dataset in <FIG> may be used to understand if node <NUM>. <NUM> is actually the root of the anomaly or if its anomalous behavior is caused by another node up in the synchronization topology hierarchy. In the table in <FIG>, the values of parameters LocalSlavePort, GrandmasterIdentity, parentPortIdentity, and stepsRemoved in the <NUM>:<NUM> time-slice have not changed compared with the previous time-slice (<NUM>:<NUM>), but the value of the parameter BestMasterChange changed from "No" to "Yes". This means that the Best Master Clock Algorithm running on node <NUM>. <NUM> has changed the synchronization source during the time-slice at least once. This value is an indication of anomalous behavior and confirms the output of the first ML model, however the root of the anomalous behavior may be a node up in the synchronization topology hierarchy, since the Best Master Clock Algorithm of node <NUM>. <NUM> decided to change the synchronization source. Therefore, the next step would be to check the behavior of the nodes up in the hierarchy. Note that the values of parameters LocalSlavePort, GrandmasterIdentity, parentPortIdentity, and stepsRemoved have not changed because the topology change took place when the data relating to the node <NUM>. <NUM> had already been sampled, therefore without the parameters BestMasterChange the behavior of the node <NUM>. <NUM> would have seemed regular and not anomalous.

According to optional embodiments, the device <NUM> provides also information on a type of anomaly that a node exhibits. Examples of types of anomalies are false anomaly that indicates the presence of an anomaly even though there is no anomaly; slow delay variations that indicate a delay causing a clock drift and a consequent phase misalignment among downstream nodes and fast delay variations that indicate a delay that may cause a change of synchronization source; Global Navigation Satellite System, GNSS, failure that indicates a damage to an antenna cabling or antenna, a local jamming, or a bad download of data from satellites; PTP failure, that indicates a network outage or an increase in network congestion. Two alternative embodiments for determining the type of anomaly will be described herein.

According to first embodiments and in the optional step <NUM>, after running <NUM> the first ML model for anomaly detection on the first dataset, the device <NUM> runs <NUM> an explainer model on the first ML model and on the first dataset. An explainer model (also referred to as interpret ML model) is a tool for interpreting predictions of a ML model and associating an importance value with each predictor. In other words, the explainer model helps determining which features are the most important for the model and how they contribute to the predictions. Examples of explainer models comprise SHAP, LIME, InterpretML, and ELI5. An output of the explainer model may be a plurality of feature importance vectors for the time interval, wherein a feature importance vector is a vector of scalar numbers in the range [<NUM>,<NUM>] associated with a feature. In other words, the explainer model generates as output a feature importance vector for each anomalous node <NUM>, <NUM> and <NUM> at a time-slice.

In the next optional step <NUM>, the device <NUM> obtains a plurality of second labels associated with the feature importance vectors. A second label is associated with a feature importance vector by an expert. The second label indicates a type of anomaly. The expert explicitly labels the feature importance vectors associated with nodes for time interval based on his/her domain expertise gained from studying the anomalies generated in a controlled test environment or in a known network environment. A second ML model takes as input the labeled feature importance vectors in a training phase of the model.

In the next optional step <NUM>, the device <NUM> runs <NUM> a second ML model taking a plurality of feature importance vectors as an input. The output of the second ML model is an indication of a type of anomaly exhibited by a node for a time interval corresponding to a feature importance vector used as an input of the second ML model. The second ML model is a supervised ML model, such as Random Forest. Alternatively, a different supervised ML model may be used.

According to alternative embodiments, after running <NUM> the first ML model for anomaly detection on the first dataset, the device <NUM> runs <NUM> a third ML model on the first dataset and the plurality of associated first labels obtained from the first ML model, in step <NUM>. The third ML model may be a supervised ML model, such as Random Forest.

In the next optional step <NUM>, the device <NUM> runs <NUM> an explainer model on the third ML model and the first dataset. An output of the explainer model is a plurality of feature importance vectors for the time interval. In other words, each node for a time interval is associated with a feature importance vector.

In the next optional step <NUM>, the device <NUM> obtains a plurality of second labels associated with the feature importance vectors. The second labels indicate a type of anomaly and a second label is associated with a feature importance vector by an expert. The expert explicitly labelled the feature importance vectors for a time interval according to the anomalies generated in a controlled test environment or in a known network environment based on domain expertise. A second ML model takes as input the labeled feature importance vectors in a training phase of the model.

In the next optional step <NUM>, the device <NUM> runs <NUM> a second ML model taking the plurality of the feature importance vectors as an input. The output of the second ML model is an indication of a type of anomaly exhibited by a node for a time interval corresponding to a feature importance vector used as an input of the second ML model. The second ML model is a supervised ML model, such as Random Forest.

In the next optional step <NUM>, the indication of the anomaly type produced in step <NUM> is associated with the corresponding anomalous node <NUM>, <NUM> and <NUM>. If an indication of an anomaly type of a node is for example 'false anomaly , it may help detect an erroneous identification of the node as anomalous in step <NUM>.

According to an embodiment in an optional step <NUM>, the first ML model generates as an output also a score for a node in a time interval. The score indicates a degree of the anomaly. The score may be used to graphically represent anomalous nodes <NUM>, <NUM> and <NUM> in a topology graph: an anomaly score added to each node of the topology graph or a color intensity proportional to the absolute value of the anomaly score may allow the localization of a node that exhibit an anomalous behavior and the degree of such anomalous behavior. In an alternative embodiment the anomaly score may be used directly as a numerical value as illustrated in the bottom graphs presented in <FIG> and <FIG>, where negative values indicate anomalous behavior and the lower the score the more severe the anomaly. A positive score indicates a non-anomalous behavior.

In some embodiments the First, Second, and Third ML models may be trained before being used. The three ML models may be trained offline or at scheduled time. If desired the method could be further improved by use of a validation process as known in the art.

<FIG> shows a flow chart illustrating example of steps and functions for performing a method according to embodiments. Two functional paths may be identified: a domain independent path providing an anomaly classification and a domain dependent path providing a topology localization of the anomaly.

In the domain independent path, the first dataset, generated by performing steps <NUM> and <NUM>, is used as an input to a first ML model. The first ML model is performed according to step <NUM> and generates as an output first labels identifying anomalous nodes <NUM>, <NUM> and <NUM>. An output of the first ML model is then used as an input of the third ML model and the explainer model, according to steps <NUM> and <NUM>. The output of the explainer model and second labels obtained performing step <NUM> are used as input of a second ML model according to step <NUM>. The second ML model provides anomaly classification according to step <NUM>.

In the domain dependent path, the first dataset is used to create a second dataset according to step <NUM>. The anomalous nodes <NUM>, <NUM> and <NUM> identified by the first ML model are mapped on a network synchronization topology obtained from the second dataset and filtered according to topology dependencies to locate a root cause of an anomaly behavior according to step <NUM>.

Localization of an anomaly in the topology, i.e. outcome of the domain dependent path, and anomaly classification, i.e. output of the domain independent path, may be combined to deliver relevant Root Cause Analysis, RCA, information to an operator or user.

An example scenario in which the present invention may be practiced is in relation to <NUM> RAN nodes synchronized using PTP, where radio units exhibit time misalignments detected at the air-interface, but with unknown root cause. In the example scenario, a device <NUM> may train a first, a second, and a third ML models and apply the trained models according to steps <NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM> and then perform an RCA according to steps <NUM> and <NUM>.

The method disclosed in <FIG> has been validated in a test network including radio access nodes (radio units, RU, baseband units, BB) and transport nodes (routers, RR, and microwave front-haul units and backhaul units, MW). Two grandmaster nodes (GM1 and GM2) are root timing references and all other clocks synchronize directly to one them by means of PTP. The physical network topology, wherein each node is identified with an acronym indicating the technology and a number, is shown in <FIG>. A first dataset has been split between training set and test set. The test network has been stressed with a number of test cases affecting network synchronization down to the radio interface. Two examples of anomalous behavior are shown in <FIG> and <FIG>. The top half of <FIG> and <FIG> shows four plots. The first plot shows phase alignment error in milliseconds for node BB211, and the other three plots show score value for nodes BB211, BB210, and BB209. In particular in the other three plots, a dashed line represents a scalar value of the score, wherein the scalar values is within an interval [-<NUM>, <NUM>] and the full line represents the binary value of the score, i.e. <NUM> or <NUM>. If the scalar value of the score is higher than <NUM>, the binary value of the score is <NUM> and indicates a regular behavior, if the scalar value of the score is lower than <NUM>, the binary value of the score is <NUM> and indicates an anomalous behavior. Clock delay and score values have been calculated from <NUM>:<NUM> to <NUM>:<NUM> on May <NUM> in <FIG> and from <NUM>:<NUM> to <NUM>:<NUM> on May <NUM> in <FIG>. The bottom half of <FIG> and <FIG> shows a network synchronization topology obtained at <NUM>:<NUM> am on May <NUM> and at <NUM>:<NUM> am on May <NUM>, respectively. Note that physical and synchronization topology may be different. Each node of the network synchronization topology comprises a number indicating anomaly score obtained according to step <NUM> and shown in the corresponding plot. Note that a score value higher than <NUM> indicates a regular behavior and a score value lower than <NUM> indicates an anomalous behavior. In <FIG>, node BB209 at <NUM>:<NUM> am on May <NUM> shows an anomaly classified by the second ML model according to step <NUM> as a fast delay variation. Looking at the corresponding synchronization topology graph, BB210 shows the highest anomaly score followed by BB209 and BB211. The link between BB210 and BB209 is dashed because even though the physical link is still present, the two nodes are not logically connected since BB210 disqualified BB209 as synchronization source because of the anomalous behavior.

In <FIG>, node BB209 at <NUM>:<NUM> am on May <NUM> shows an anomaly classified by the second ML model according to step <NUM> as slow delay variation. In the corresponding synchronization topology graph, BB209 shows the highest anomaly behavior followed by BB211, BB210, and RR210.

<FIG> is a block diagram illustrating one embodiment of a device <NUM>, comprising a processor <NUM>, a computer program product <NUM> in the form of a computer readable storage medium <NUM> in the form of a memory <NUM> and communication circuitry <NUM>.

The memory, <NUM>, contains instructions executable by the processor, <NUM>, such that the device <NUM>, is operative to obtain, <NUM>, information associated with a plurality of nodes <NUM>-<NUM>. The device <NUM> may be operative to receive the information from an NMS <NUM> or similar system, or directly from the plurality of nodes <NUM>-<NUM>. The device <NUM> is operative to create <NUM> a first dataset based on the obtained associated information and to create <NUM> a second dataset in form of a plurality of time-slices. The device <NUM> is further operative to run <NUM> a first Machine Learning, ML, model for anomaly detection on the first dataset. Then the device <NUM> is also operative to identify <NUM> anomalous nodes <NUM>, <NUM> and <NUM> and time-slices when the anomalous nodes <NUM>, <NUM> and <NUM> exhibit anomalous behavior. In the operation of identifying the device <NUM> uses the first labels and the second dataset to determine relationships between the anomalous nodes <NUM>, <NUM>, <NUM> and other nodes <NUM>-<NUM> of the network.

The device, <NUM>, is further operative to run <NUM> an explainer model on the first ML model and the first dataset. The device <NUM> may obtain <NUM> a plurality of second labels associated with the feature importance vectors for the time interval, wherein a second label indicates a type of anomaly. The plurality of second labels may be manually associated with the feature importance vectors for the time interval by an expert. Then, the device <NUM> may run <NUM> a second ML model taking the plurality of the feature importance vectors as an input and use the output of the second ML model to identify <NUM> the type of anomaly of the anomalous nodes <NUM>, <NUM>, <NUM>.

Alternatively, the device <NUM> is further operative to run <NUM> a third ML model on the first dataset and a plurality of associated first labels; run <NUM> an explainer model on the third ML model and the first dataset, wherein an output of the explainer model is a plurality of feature importance vectors for the time interval. The device <NUM> may obtain <NUM> a plurality of second labels associated with the feature importance vectors for the time interval, wherein a second label indicates a type of anomaly. The plurality of second labels may be manually associated with the feature importance vectors for the time interval by an expert. Then, the device <NUM> may run <NUM> a second ML model taking the plurality of the feature importance vectors as an input and use the output of the second ML model to identify <NUM> the type of anomaly of the anomalous nodes <NUM>, <NUM>, <NUM>.

The device, <NUM>, is further operative to perform the operations of the method described in the embodiments disclosed earlier.

The device, <NUM>, may include a processing circuitry (one or more than one processor), <NUM>, coupled to communication circuitry, <NUM>, and to the memory <NUM>. The device, <NUM>, may comprise more than one communication circuitry. For example, one communication circuitry may be for connecting to the plurality of nodes <NUM>-<NUM>, and another interface may be for connecting to the NMS <NUM>. For simplicity and brevity only one communication circuitry, <NUM>, has been illustrated in <FIG>. By way of example, the communication circuitry, <NUM>, the processor(s) <NUM>, and the memory <NUM> may be connected in series as illustrated in <FIG>. Alternatively, these components <NUM>, 601and <NUM> may be coupled to an internal bus system of the device, <NUM>.

The memory <NUM> may include a Read-Only-Memory, ROM, e.g., a flash ROM, a Random Access Memory, RAM, e.g., a Dynamic RAM, DRAM, or Static RAM, SRAM, amass storage, e.g., a hard disk or solid state disk, or the like.

The computer program product <NUM> comprises a computer program <NUM>, which comprises computer program code loadable into the processor <NUM>, wherein the computer program <NUM> comprises code adapted to cause the device <NUM> to perform the steps of the method described herein, when the computer program code is executed by the processor <NUM>. In other words, the computer program <NUM> may be a software hosted by the device <NUM>.

It is to be understood that the structures as illustrated in <FIG> are merely schematic and that the device, <NUM>, may actually include further components which, for the sake of clarity, have not been illustrated, e.g., further interfaces or processors. Also, it is to be understood that the memory, <NUM>, may include further program code for implementing other and/or known functionalities.

It is also to be understood that the device, <NUM>, may be provided as a virtual apparatus. In one embodiment, the device, <NUM>, may be provided in distributed resources, such as in cloud resources. When provided as virtual apparatus, it will be appreciated that the memory, <NUM>, processing circuitry, <NUM>, and communication circuitry, <NUM>, may be provided as functional elements. The functional elements may be distributed in a logical network and not necessarily be directly physically connected. It is also to be understood that the device, <NUM>, may be provided as a single-node device, or as a multi-node system.

<FIG> schematically illustrates, in terms of a number of functional units, the components of a device <NUM> according to an embodiment. The device <NUM> comprises a first obtaining unit <NUM> configured to obtain <NUM> information associated with a plurality of nodes <NUM>-<NUM>; a first creating unit <NUM> configured to create <NUM> a first dataset based on the obtained information associated with the plurality of nodes <NUM>-<NUM>; a second creating unit <NUM> configured to create <NUM> a second dataset in form of a plurality of time-slices; a first running unit <NUM> configured to run <NUM> a first Machine Learning, ML, model for anomaly detection on the first dataset; a first identifying unit <NUM> configured to identify <NUM> anomalous nodes <NUM>, <NUM> and <NUM> and time-slices when the anomalous nodes <NUM>, <NUM> and <NUM> exhibit anomalous behavior using the first labels and the second dataset to determine relationships between the anomalous nodes <NUM>, <NUM>, <NUM> and other nodes <NUM>-<NUM> of the network.

Claim 1:
A method for monitoring a synchronization status among a plurality of nodes (<NUM>-<NUM>) operating in a communications network, the method comprising:
- obtaining (<NUM>) information associated with a plurality of nodes (<NUM>-<NUM>), the nodes (<NUM>-<NUM>) having synchronized clocks, wherein an information for a node (<NUM>-<NUM>) is indicative of one or more parameter values;
- creating (<NUM>) a first dataset based on the obtained information associated with the plurality of nodes (<NUM>-<NUM>), the first dataset comprising time-aligned samples of the one or more parameters for a time interval;
- creating (<NUM>) a second dataset in form of a plurality of time-slices comprising the samples from the first dataset and statistics determined based on the samples from the first dataset calculated for the time interval;
- running (<NUM>) a first Machine Learning, ML, model for anomaly detection on the first dataset, the first ML model generating first labels identifying anomalous nodes (<NUM>, <NUM>, <NUM>);
- identifying (<NUM>) anomalous nodes (<NUM>, <NUM>, <NUM>) and time-slices when the anomalous nodes (<NUM>, <NUM>, <NUM>) exhibit anomalous behavior using the first labels and the second dataset to determine relationships between the anomalous nodes (<NUM>, <NUM>, <NUM>) and other nodes (<NUM>-<NUM>) of the network.