Patent Description:
A Customer Premises Equipment (CPE) in a telecommunications network is a device typically located in a customer's home or business that connects the customer to an operator's network (such as a Digital Subscriber Line (DSL) network, a Fibre To The Premises (FTTP) network, and/or a cellular network). The CPE may also provide the customer with a local area network (including a wireless local area network) to connect one or more devices to the network operator's network.

A CPE may be remotely managed by the operator. Remote management may be implemented by an Auto-Configuration Server (ACS) using the TR-<NUM> protocol (as standardised by the Broadband Forum). The TR-<NUM> protocol defines a dataset that is periodically collected from the CPE for analysis by the ACS. This dataset identifies all devices connected to the CPE. The ACS analyses this data as part of a management service (e.g. to identify faults).

The dataset collected under the TR-<NUM> protocol identifies each device connected to the hub by its Media Access Control (MAC) address. However, some devices now change their MAC address based on a MAC randomisation process. MAC randomisation is a process in which the device uses a randomly generated MAC address, instead of its actual MAC address, when communicating with other devices in a network. One of the first uses of MAC randomisation was for devices to transmit a randomly generated MAC address as part of a probe request message when scanning for network access points, but use their actual MAC address when connecting to one of those network access points. This process was subsequently developed so that a device would connect to a network access point using a MAC address randomly generated for that network (such that the device uses a different randomly generated MAC address for each network it connects to). Each network would thereafter identify that device by its network-specific randomly generated MAC address. This process has more recently been developed so that a device, once connected to a network, periodically (e.g. once per day) updates this identifier in the network with a new randomly generated MAC address. This may be implemented by reconnecting to the network with a new, randomly generated MAC address.

This periodic MAC randomisation process creates a technical problem in that the MAC address can no longer be used as a persistent identifier for a device over a time-period that is greater than the periodicity of the MAC randomisation. For example, if a first record in a network-specific dataset is collected prior to an instance of a periodic MAC randomisation process and a second record in the same network-specific dataset is collected following that MAC randomisation instance, then it cannot be determined whether those records relate to two separate devices or to the same device that has performed a MAC randomisation. This problem applies to the dataset collected using the TR-<NUM> protocol, but also applies to any other dataset which identifies each device connected to a particular network by its MAC address. When applied to the TR-<NUM> dataset, the operator's management processes (such as fault identification and/or network optimisation) may suffer as a result.

<CIT> discloses that a service maintains a database of media access control (MAC) addresses of devices in a network and their associated telemetry data captured from the network. The service identifies a new MAC address being used by a particular device in the network.

Article "<NPL>et al. , discloses a method for identifying BLE devices with randomized MAC addresses without using device-specific values.

A first embodiment of a telecommunications network <NUM> will now be described with reference to <FIG>. The telecommunications network <NUM> includes an operator's network <NUM> and a customer's network <NUM>. The operator's network <NUM> and customer's network <NUM> are connected by an access connection <NUM>, which in this embodiment is a Digital Subscriber Line (DSL). The customer's network <NUM> includes a Customer Premises Equipment (CPE) <NUM> and a plurality of user devices <NUM>.

The CPE <NUM> includes an access network communications interface <NUM>, a processor <NUM>, memory <NUM>, a wired communications interface <NUM> and a wireless communications interface <NUM>. The access network communications interface <NUM> enables the CPE <NUM> to communicate with the operator's network <NUM> via the access connection <NUM>. The wired and wireless communications interfaces <NUM>, <NUM> respectively enable the CPE <NUM> to provide a wired local area network and a wireless local area network.

In this embodiment the plurality of user devices <NUM> each include a wireless communications interface <NUM> (for communicating with the wireless local area network of the CPE <NUM>).

The CPE <NUM> is identifiable by a MAC address for a Network Interface Controller (NIC) associated with its access network communications interface <NUM>. Each user device of the plurality of user devices <NUM> is identifiable by a MAC address of a NIC associated with its respective communications interface <NUM>.

The operator's network <NUM> includes an access network communications node <NUM> and a Network Management System (NMS) <NUM>. The access network communications node <NUM> enables the NMS <NUM> (and any other node of the operator's network, or any node in an external network connected to the operator's network) to communicate with the CPE <NUM> via the access connection <NUM>. The NMS <NUM> includes a communications interface <NUM>, a processor <NUM> and a memory <NUM>.

A first embodiment of a method of the present invention will now be described with reference to <FIG>. <FIG> illustrates steps performed by the CPE <NUM>'s processing and memory modules <NUM> and <NUM>, and <FIG> illustrates steps performed by the NMS <NUM>'s processing and memory modules <NUM> and <NUM>.

In a first step (S101) of the first process, the CPE's processor <NUM> implements a diagnostics function to collect data on each device of the plurality of devices <NUM>. This data is collected periodically and this periodicity may be remotely configured by the network operator (this periodicity may be in a range from a hundred milliseconds to every ten seconds). Each record in the data includes the MAC address of the user device, a timestamp representing the time the data was collected from the user device, a hostname of the user device, and parameters of the user device. These parameters include Received Signal Strength Indicator (RSSI), downlink physical layer (PHY) rate, and uplink PHY rate. The parameters of the user device may also include:.

This data is hereinafter referred to as detailed diagnostics data. The detailed diagnostics data is stored in the CPE's memory <NUM>.

In step S103, the CPE's processor <NUM> implements a compression function to convert the detailed diagnostics data into summarised diagnostics data. The detailed diagnostics data is converted to summarised diagnostics data periodically and this periodicity may be remotely configured by the network operator (this periodicity may be in the range from one minute to <NUM> minutes). The conversion may be implemented by applying one or more statistical functions to the detailed diagnostics data, such as a summation, average, maximum, or minimum of each parameter of the user device. The summarised diagnostic data is also stored in the CPE's memory <NUM>.

In step S105, the CPE transmits the summarised diagnostic data to the NMS (that is, via the access connection <NUM>). Turning to <FIG>, in step S201, the NMS <NUM> receives the summarised diagnostic data and stores it in memory <NUM>. Steps S101 to S105 and step S201 are performed periodically such that the NMS <NUM> collects additional records and adds them to the summarised diagnostic data already stored in memory <NUM>. In this embodiment, the summarised diagnostic data stored in memory <NUM> spans a time period that is greater than the MAC randomisation time period implemented by one or more user devices of the plurality of user devices <NUM>.

The NMS <NUM> stores a first and second table based on the summarised diagnostic data. The first table - a key performance indicator table - includes values for one or more key performance indicators for each user device of the plurality of user devices <NUM>. These key performance indicators include, for example, a total active time, total poor RSSI time, and total poor coverage time (which may be calculated from the summarised diagnostic data). The key performance indicator table includes the MAC address and hostname as identifiers of the user device. The key performance indicator table is updated with new records when additional records are added to the summarised diagnostic data. In this embodiment, the key performance indicator table is updated with new records for the past <NUM> days (such that any data older than <NUM> days is removed).

An example key performance indicator table - having a single key performance indicator for total active time - is shown in <FIG>. It is noted, for the sake of clarity, that placeholder MAC addresses (MAC1, MAC2, MAC3) are used, and these are not in the form of real MAC addresses.

The second table - a daily hostname count table is populated with records as described in step S203 below. This first embodiment allows the NMS <NUM> to determine whether multiple records of the key performance indicator table having different MAC addresses relate to the same user device and, if so, merge those records of the key performance indicator table.

In step S203, the NMS processor <NUM> retrieves data for the current day in the summarised diagnostic data table and processes the retrieved data to calculate, for each distinct hostname in the retrieved data, a first count - named SampleCount - being a count of all records in the retrieved data having that hostname, and a second count - named UniqueTimeStampCount - being a count of all distinct timestamps of the subset of records in the retrieved data having that hostname. The daily hostname count table is updated with an entry for each distinct hostname with its SampleCount and UniqueTimeStampCount values. Each record contains a further field - CalculationDate - identifying the day the SampleCount and UniqueTimeStampCount values were calculated.

The daily hostname count table includes records for each hostname for each day of a time period, which in this embodiment is <NUM> days (that is, the above process was implemented in each of the past <NUM> days so as to record the SampleCount and UniqueTimeStampCount values for each distinct hostname in each of the past <NUM> days).

An example daily hostname count table is shown in <FIG>. In day <NUM> of this example, the SampleCount value is <NUM> and the UniqueTimeStampCount is <NUM>. This may be calculated where there is MAC randomisation event on day <NUM> such that there are two records using the same timestamp, but different MAC addresses, for the same hostname.

In step S205, the NMS processor <NUM> retrieves data for the past <NUM> days in the daily hostname count table. The NMS processor <NUM> processes the retrieved data to identify each distinct hostname and calculate, for each distinct hostname, a uniqueness ratio. The uniqueness ratio is calculated as the sum of SampleCount over the <NUM> days for that hostname divided by the sum of UniqueTimeStampCount over the <NUM> days for that hostname. NMS processor <NUM> creates a third table - a hostname uniqueness table - which includes a record for each distinct hostname and further indicates, for each distinct hostname, a binary IsUnique field to indicate whether the uniqueness ratio for that hostname is below a uniqueness threshold (e.g. <NUM> or <NUM>). Each record contains a further field - CalculationDate - identifying the day the record is created.

An example hostname uniqueness table is shown in <FIG>.

In step S207, NMS processor <NUM> joins the key performance indicator table and the hostname uniqueness table on the hostname field to create a new table - the key performance indicator merged table. The key performance indicator merged table includes all fields from the key performance indicator table and, for that hostname, the IsUnique and CalculationDate fields from the hostname uniqueness table.

In step S209, NMS processor <NUM> processes all records in the key performance indicator merged table to create two new fields - unique identifier and unique hostname. The unique identifier field is determined based on the following logic:.

The unique hostname is determined based on the following logic:.

An example of the key performance indicator merged table following step S209 is shown in <FIG>.

In step S211, NMS processor <NUM> groups records in the key performance indicator merged table having the same (non-NULL) unique identifier and same (non-NULL) unique hostname such that the key performance indicator values are combined (e.g. summed).

An example of the key performance indicator merged table following step S211 is shown in <FIG>.

In step S213, NMS processor <NUM> identifies one or more configurations for the network by analysing data in the key performance indicator merged table. These reconfigurations may be, for example, updating Quality of Service (QoS) parameters for the most active set of devices (that is, those having the greatest "total active time" value), updating steering policies for the most active set of devices, and/or reconfiguring a transmission property of the access point (or set of access points in a mesh network) based on a coverage KPI indicator.

The process may then loop back to step S201 for a subsequent performance on the next day. On the next day, the NMS <NUM> receives additional data to be added to the summarised diagnostic data. The key performance indicator table and daily hostname count table are then updated based on the most recent <NUM> days of summarised diagnostic data. The remaining steps of the above first embodiment may then be performed on the updated records.

The above first embodiment enables the NMS <NUM> to determine when multiple records having different MAC addresses relate to the same user device. This is achieved by determining that the hostname associated with those multiple records is sufficiently unique and using the most recent MAC address associated with that hostname as a unique identifier. All records in the key performance indicator table that use that unique hostname may then be analysed as relating to a single user device.

In the above embodiment, the dataset included records for a single CPE. However, the skilled person will understand that the NMS <NUM> may collect data from multiple CPEs, and the same method may be applied to this multiple-CPE dataset (e.g. by isolating data for a particular CPE based on a CPE identifier, such as a serial number, and applying the same method to the dataset for that particular CPE).

The skilled person will understand that the first embodiment may be applied when a user device is identified by another form of persistent device label, such as a server name or fully qualified domain name. Furthermore, the first embodiment may be applied in any scenario where a dataset for a CPE's network identifies a user device by its MAC address and device label and that MAC address may change.

A second embodiment of a method of the present invention will now be described with reference to <FIG>, <FIG>, <FIG> and <FIG>. This second embodiment utilises the same telecommunications network as described in <FIG>. In this second embodiment, the CPE <NUM> performs the same process as illustrated in <FIG> such that it sends summarised diagnostic data to the NMS <NUM>. In this embodiment, NMS <NUM> performs the following machine learning process to determine that multiple records having different MAC addresses relate to the same user device.

In a first step (S301), as illustrated in <FIG>, the NMS <NUM> receives the summarised diagnostic data and stores it in memory <NUM>. In step S303, NMS processor <NUM> retrieves the MAC address, RSSI, downlink PHY rate, and uplink PHY rate of each record for a first time period (this may be limited to those records of the key performance indicator table - in the first embodiment above - where the unique identifier and unique hostname fields are NULL). This first time period is selected such that it is less than a MAC randomisation time period of any one of the user devices. For example, if the MAC randomisation time period is such that one or more user devices updates its MAC address once every <NUM> hours, then the time period of the retrieved data is less than <NUM> hours (e.g. <NUM> hours, <NUM> hours, or <NUM> hours).

In step S305, NMS processor <NUM> divides the retrieved data into a training dataset and a validation dataset. In step S307, NMS processor <NUM> determines the minimum and maximum values of each parameter for all records in the training dataset, and scales the values of each parameter of each record independently between <NUM> and <NUM> based on their original value and the minimum and maximum values.

In step S309, NMS processor <NUM> processes the training dataset to determine a count of values of each parameter in consecutive non-overlapping intervals of <NUM> (that is, <NUM> to <NUM>, <NUM> to <NUM>,. , <NUM> to <NUM>) for each MAC address for a plurality of time intervals (that is, a first hour, a second hour, etc.) in the time period.

The NMS processor <NUM> therefore determines, for a first MAC address for a first hour in the time period, a count of values of each parameter between <NUM> and <NUM>, a count of values of each parameter between <NUM> and <NUM>, and so on; determines, for the first MAC address for the second hour in the time period, a count of values of each parameter between <NUM> and <NUM>, a count of values of each parameter between <NUM> and <NUM>, and so on. A <NUM>-dimensional vector is created for each MAC address in the training dataset for each hour in the time period, which contains the counts of downlink PHY rate in each interval from <NUM> to <NUM> to <NUM> to <NUM>, the counts of RSSI in each interval from <NUM> to <NUM> to <NUM> to <NUM>, and the counts of uplink PHY rate in each interval from <NUM> to <NUM> to <NUM> to <NUM>. An example of this vector being constructed from these counts is illustrated in <FIG>. In step S311, these vectors for the training dataset are stored and each associated with an identifier, the identifier being unique to the MAC address (and therefore the user device) from which the vector was derived in step S309.

In step S313, a neural network is trained to map between the vectors of the training dataset and the associated identifier. In this second embodiment, the "neuralnet" library of the R programming language was used in which the input layer consists of the vectors, the output layer consists of the identifiers associated with the vectors, and the hidden layer has a number of dimensions that is greater than a count of distinct identifiers and less than a count of elements in the vector (that is, <NUM> in this example in which each vector contains <NUM> intervals for each of the three parameters). In more detail, the neuralnet parameters are selected as follows:.

It is noted that the above process is performed on summarised diagnostic data for a particular CPE, such that the neural network is trained for that particular CPE. If the NMS <NUM> receives data from multiple CPEs, then the above process is performed on the subset of data relating to a particular CPE (that is, by retrieving data in step S301 relating to that particular CPE by using a unique identifier for that CPE, such as its serial number) such that the remaining steps train a neural network for that particular CPE. The neural network may also be retrained periodically, such as every hour, based on the most recent data in the summarised diagnostic data.

In step S315, the NMS processor <NUM> processes the validation dataset in the same way as described above in steps S307 to S311 (i.e. NMS processor <NUM> scales the values of each parameter of each record in the validation dataset, creates vectors for each MAC address, and stores each vector in memory with an associated identifier, the identifier being unique to the MAC address). In step S317, the trained neural network is tested using the vectors of the validation dataset as input. If the performance of the trained neural network satisfies a threshold (based on a comparison of the output of the trained neural network with the identifiers associated with each vector in the validation dataset), then the trained neural network is validated and may be used in the subsequent steps of this second embodiment. If the performance of the trained neural network does not satisfy the threshold, then the neural network may be retrained (for example, by using different training data and/or different neuralnet parameters).

Turning to <FIG>, in step S319, NMS processor <NUM> retrieves data from the summarised diagnostic data covering the RSSI, downlink PHY rate and uplink PHY rate of each record for a second time period. This second time period is now greater than the MAC randomisation time period of one or more of the user devices, such that multiple records relating to the same device may have different MAC addresses such that it cannot be determined, based on those records alone, whether those multiple records relate to the same user device or not. To solve this problem, in step S321, NMS processor <NUM> processes the data retrieved in step S319 in the same way as described above in steps S307 to S309 (i.e. NMS processor <NUM> scales the values of each parameter of each record in the retrieved data, and creates vectors for each MAC address). In step S323, NMS processor <NUM> applies the validated neural network to the data (retrieved in step S319 and processed in step S321), which outputs an identifier for each input vector. This identifier therefore identifies all records in the data that relate to the same device, even if they use different MAC addresses. In step S325, the data may be analysed with this identifier (and this analysis may involve other parameters that did not form part of the vector) to identify one or more reconfigurations of the telecommunications network. This analysis may be, for example:.

The skilled person will understand that it is non-essential that the neuralnet library of the R programming language is used, and that any suitable machine learning process may be used instead (including a support vector machine, random forest, or other form of neural network, such as a convolutional neural network or a recurrent neural network). The data preparation stages of steps S303 to S311 may still be applied when using these other machine learning processes.

In the above embodiments, the CPE <NUM> and access network communications node <NUM> are connected by a DSL. However, this is non-essential and other forms of access connection, such as fixed wireless access or FTTP, may also be used.

In the first embodiment, the hostname uniqueness threshold is in a range of <NUM> to <NUM>.

However, this is non-essential. This threshold is determined by the MAC randomisation period. The shorter the randomisation period is (relative to the time window used for determining the hostname uniqueness value, which was <NUM> days in the first embodiment), the higher the threshold will be. In other words, the hostname uniqueness threshold has an inverse relationship to the MAC randomisation period.

Claim 1:
A method of operating a data processing apparatus (<NUM>) to analyse a wireless local area network, the wireless local area network including a plurality of devices (<NUM>) wherein at least one device of the plurality of devices (<NUM>) is configured to change its Media Access Control, MAC, address, the method comprising the steps of:
obtaining data including a plurality of records, wherein each record of the plurality of records relates to a device of the plurality of devices (<NUM>) and includes:
a device label,
a MAC address, and
a set of performance metrics;
for each device label in the plurality of records, determining a uniqueness value for that device label;
processing each record of the plurality of records so as to assign a unique identifier to the record by determining that the MAC address of the record is a local MAC address and that the uniqueness value of the device label satisfies a uniqueness threshold; and
analysing the set of performance metrics for a plurality of records having the same unique identifier,
characterised in that the uniqueness value is determined based on a ratio of a count of records in the plurality of records that use that device label to a count of distinct timestamps in the plurality of records that use that device label.