Patent Description:
Document <CIT> describes a method of publishing data elements related to industrial devices operating on an industrial site at regular intervals, such as every ten minutes. In this method, the data elements generated by the industrial devices are combined into files, for example JSON-files, and the files are continuously stored in a source database. The actual publishing is done by regularly extracting a set of files from the source database, compressing it, and sending it to an application server. More precisely, each regular publishing process involves the following steps:.

Accordingly, in this known method, the collection of data elements from industrial devices and their storage on the one hand, and the publication of data elements related to the industrial devices on the other hand, are two separate and decoupled processes. While the collection and storage of data elements is a continuous process, the publication of data elements is done periodically according to a well-defined schedule. Consequently, this known method requires substantial storage capacity to store the data elements collected from the industrial devices at least until they have been published in line with the publishing schedule. Also, with this known method, precise remote monitoring of an industrial site is limited, due to the non-negligible timespan (<NUM> minutes) between the publication of two consecutive file sets.

In view of the above, it is an object of the present disclosure to provide an improved method of transmitting industrial telemetry data elements from an industrial site to a remote server. This improved method should be suited to the streaming of industrial telemetry data elements, i.e., should allow the transmission of the data elements from their origin (the industrial devices) to the remote server in a quasi-continuous flow.

According to the present disclosure, this object is achieved with a method of streaming industrial telemetry data elements from an industrial site to a remote server, the method comprising the following steps: receiving a stream of industrial telemetry data elements from a multitude of industrial devices operating on the industrial site, storing each received industrial telemetry data element in a data elements queue, periodically extracting a number N of data elements from the data elements queue and aggregating the extracted N data elements into one data frame, such that the size of the resulting data frame does not exceed a maximum frame size, storing each resulting data frame in a data frames buffer, and periodically extracting a data frame from the data frames buffer and sending the extracted data frame to the remote server, wherein a data element is removed from the data elements queue and stored in a collecting database as an excess data element if the time the data element has spent in the data elements queue exceeds a maximum queuing time, wherein a data frame is stored in the collecting database as an excess data frame instead of being stored in the data frames buffer if the data frames buffer is full, wherein the maximum queuing time is optimised in real time as a function of the latest number of data elements removed from the data elements queue and the frequency at which the data frames buffer is full, and wherein the number N of data elements extracted in step c is optimised in real time in view of the size of the latest resulting data frame and of the number of data elements aggregated into this data frame, while respecting said maximum frame size.

Indeed, in this method, the data elements from the industrial devices transit with little or no delay towards the remote server. In case of congestion, data elements and/or data frames are dropped into a collecting database to restore a steady data flow. The data elements queue and the data frames buffer provide the flexibility that is needed for a reliable streaming of the data elements. Thanks to the queue and to the buffer, the data flow can be managed and adapted as a function of the amount of traffic. The memory footprint of the present method is small, so that it can be executed by a resource-constrained device, such as an IoT gateway. According to the present method, by constantly optimising the maximum queuing time and the number N of extracted data elements, few data elements are dropped, and the amount of data pushed to the remote server is increased.

The following features can be optionally implemented, separately or in combination with each other:.

The present disclosure also provides a computing device, in particular an IoT gateway, configured for carrying out a method as defined above.

A further aspect of the present disclosure is a computer software comprising instructions to implement a method as defined above when the software is executed by a processor.

A further aspect of the present disclosure is a computer-readable non-transient storage medium on which said computer software is stored.

The features, details and advantages of the present disclosure will become more readily apparent from the following detailed description and the accompanying figure, which illustrates how an IoT gateway according to one embodiment of the present disclosure processes a stream of measurements coming from an industrial site, and how the IoT gateway pushes the processed measurements to a remote server in the cloud.

The only drawing shows the different processing stages in one exemplary embodiment of the method according to the present disclosure.

The method exemplified by the drawing is a method of streaming industrial telemetry data elements <NUM> from an industrial site <NUM> to a remote server in the Cloud <NUM>. Typically, this method is executed by a computing device, such as an IoT gateway, which sits between the industrial site <NUM> and the Cloud <NUM>.

The method shown in the figure comprises the following steps:.

In a first step S1, the IoT gateway receives a stream of industrial telemetry data elements <NUM> (here, sensor measurements) from a multitude of industrial devices operating on the industrial site <NUM>.

In a second step S2, each received industrial telemetry data element is stored in a data elements queue <NUM>.

In a step S3, an extractor <NUM> periodically extracts a number N of data elements from the data elements queue <NUM>.

In a subsequent step S4, the extracted data elements (identified by the reference number <NUM>) are aggregated into one data frame <NUM>, preferably by a serializer <NUM>. The aggregation is done such that the size of the resulting data frame <NUM> does not exceed a maximum frame size.

In a next step S5, each resulting data frame <NUM> is stored in a data frames buffer <NUM>.

In a final step S6, a data frame is periodically extracted from the data frames buffer <NUM> and sent to the remote server in the Cloud <NUM>.

To manage the flow of the data elements <NUM> from the industrial site <NUM> to the Cloud <NUM>, and in particular to avoid congestion, the method shown in the figure provides the two following measures:.

In the first measure, a data element is removed from the data elements queue <NUM> and stored in a collecting database <NUM> as an excess data element if the time the data element has spent in the data elements queue <NUM> exceeds a maximum queuing time. This is illustrated by the arrow A in the figure.

In the second measure, a data frame <NUM> is stored in the collecting database <NUM> as an excess data frame instead of being stored in the data frames buffer <NUM> if the data frames buffer <NUM> is full. This is illustrated by the arrow B in the figure.

The shown method includes a first control loop L1 to optimise the maximum queuing time associated with the data elements queue <NUM>. According to this control loop L1, the maximum queuing time is optimised in real time as a function of the latest number of data elements removed from the data elements queue <NUM> and the frequency at which the data frames buffer <NUM> is full.

In the shown example, the maximum queuing time is optimised using a machine learning algorithm, namely a first Multi-Armed Bandit algorithm <NUM>. The two inputs of the first Multi-Armed Bandit algorithm <NUM> are identified by the arrows C and D in the figure. As already mentioned, one input is the latest number of data elements dropped from the data elements queue <NUM> (arrow C), and the other input is the overflow frequency of the data frames buffer <NUM> (arrow D). The output of the first Multi-Armed Bandit algorithm <NUM> is an updated estimation of the maximum queuing time, cf. Arrow E in the figure.

The illustrated method also includes a second control loop L2 to optimise the number N of data elements that are periodically extracted by the extractor <NUM>. The number N of extracted data elements is optimised in real time in view of the size of the latest resulting data frame <NUM> and of the number of data elements <NUM> aggregated into this data frame, while respecting the maximum size that a data frame <NUM> may have (e.g., <NUM> kB). In the shown example, the optimisation loop L2 is implemented using a machine learning algorithm, namely a second Multi-Armed Bandit algorithm <NUM>. The input of the second Multi-Armed Bandit algorithm <NUM> is identified by the arrow F, and the output by the arrow G.

Preferably, the data elements queue <NUM> is implemented via an active queue management algorithm, such as a CoDel algorithm, in accordance with the RFC <NUM> standard.

The data frames buffer <NUM> is preferably implemented via a Leaky Bucket algorithm. The purpose of such a Leaky Bucket algorithm <NUM> is to deliver data frames for publishing at a constant rate. The Leaky Bucket algorithm <NUM> ensures a constant delay between each data frame transmission to the remote server in the Cloud <NUM>. This constant delay is a parameter of the Leaky Bucket algorithm <NUM>, which may be set by a user depending on the use case. The constant delay may for example be <NUM> milliseconds.

Another parameter of the Leaky Bucket algorithm <NUM> is the buffer size, i.e., the size of the "leaky bucket". Incoming data frames <NUM> are continuously added to the buffer (the "leaky bucket") as long as the overall size of the data frames stored in the buffer does not exceed the buffer size. In an overflow situation, i.e., when the "leaky bucket" is full, the first control loop L1 adapts the maximum queuing time to reduce the fill level of the "leaky bucket" and stop the overflow. For example, the first control loop L1 may first raise the maximum queuing time and then reduce it.

The serializer <NUM> is preferably a Sensor Measurement List, SenML, serializer according to the RFC <NUM> standard.

The collecting database <NUM> is a temporary database, which stores the excess data elements and excess data frames for later delivery to the remote server in the Cloud <NUM>. As indicated by arrow H, excess data elements and excess data frames are periodically retrieved from the collecting database <NUM> and added to the data frames buffer <NUM> to be sent to the remote server. This delivery retrying amounts to a third control loop L3. More precisely, an excess data element, whose delivery is to be retried, is sent by a router <NUM> to the serializer <NUM> to be included into a data frame <NUM> before being added to the data frame buffer <NUM> as part of the data frame <NUM>. An excess data frame, whose delivery is to be retried, is directly added by the router <NUM> to the data frames buffer <NUM>, bypassing the serializer <NUM> (cf.

In the illustrated embodiment, the method also includes a circuit breaker algorithm <NUM>. This algorithm stops attempts to send data frames to the remote server if the connection thereto is lost. In this case, data frames leaving the data frames buffer <NUM> are diverted into the collecting database <NUM> until the end of a predetermined timeout, cf. After the end of the timeout, if the connection with the remote server has been re-established, sending of data frames leaving the data frames buffer to the remote server resumes. As shown in the figure, cf. arrow K, the information that the connection to the remote server is lost may be provided to the circuit breaker algorithm <NUM> by an ETP publisher algorithm <NUM>. The ETP publisher algorithm <NUM> is in charge of the actual publication of the data frames to the Cloud <NUM>. The ETP publisher algorithm <NUM> is only exemplary and may be replaced by any other appropriate cloud communication protocol.

The purpose of the circuit breaker algorithm <NUM> is to give the ETP publisher algorithm <NUM> sufficient time to re-establish a connection with the Cloud <NUM>. Indeed, the time needed by the ETP publisher algorithm <NUM> to reconnect to the Cloud <NUM> is usually longer than the constant delay between two consecutive data frames leaving the data frames buffer <NUM>.

In the illustrated method, excess data elements and excess data frames, whose storage time in the collecting database <NUM> exceeds a predetermined threshold, are transferred from the collecting database <NUM> to a permanent archive <NUM>, cf. More precisely, each excess data element and each excess data frame is stored in the collecting database <NUM> with a Time To Live, TTL. If the TTL of the data element/data frame has expired, a periodic cleaning algorithm <NUM> extracts the same from the collecting database <NUM> and transfers it to the permanent archive <NUM>.

The streaming method of the present disclosure has in particular the following advantages:.

Thanks to the two auto-adaptive machine learning control loops L1 and L2, the method only requires little configuration from a user. In particular, a user does not need to set the number N for the periodic measurements extraction and serialisation. Also, the user does not need to set the maximum queuing time. These two parameters are notoriously difficult to estimate. Accordingly, thanks to the two control loops L1 and L2, the method of the present disclosure is auto-adaptive to the number of incoming data elements, optimises the size of the frames to be sent to the remote server, and minimizes the amount of data lost during the streaming.

Also, the method of the present disclosure is particularly suited for running on resource-constrained devices because it only requires a limited amount of memory.

Claim 1:
A method of streaming industrial telemetry data elements (<NUM>) from an industrial site (<NUM>) to a remote server, the method comprising the following steps:
a. receiving (S1) a stream of industrial telemetry data elements (<NUM>) from a multitude of industrial devices operating on the industrial site (<NUM>);
b. storing (S2) each received industrial telemetry data element in a data elements queue (<NUM>);
c. periodically extracting (S3) a number N of data elements from the data elements queue (<NUM>) and aggregating (S4) the extracted N data elements into one data frame (<NUM>), such that the size of the resulting data frame (<NUM>) does not exceed a maximum frame size;
d. storing (S5) each resulting data frame in a data frames buffer (<NUM>); and
e. periodically extracting (S6) a data frame from the data frames buffer (<NUM>) and sending (S6) the extracted data frame to the remote server,
wherein a data element is removed from the data elements queue (<NUM>) and stored in a collecting database (<NUM>) as an excess data element if the time the data element has spent in the data elements queue (<NUM>) exceeds a maximum queuing time,
wherein a data frame is stored in the collecting database (<NUM>) as an excess data frame instead of being stored in the data frames buffer (<NUM>) if the data frames buffer is full,
wherein the maximum queuing time is optimised in real time as a function of:
- the latest number of data elements removed from the data elements queue (<NUM>); and
- the frequency at which the data frames buffer (<NUM>) is full, and
wherein the number N of data elements extracted in step c is optimised in real time in view of the size of the latest resulting data frame (<NUM>) and of the number of data elements (<NUM>) aggregated into this data frame, while respecting said maximum frame size.