ENCODER-DECODER ARCHITECTURE FOR GENERATING NATURAL LANGUAGE DATA BASED ON TELEMETRY DATA

Systems and methods provide techniques for generating natural language data based on telemetry data. In one embodiments, a method includes at least operations configured to receive telemetry data, wherein the telemetry data includes temporal telemetry vectors; process the temporal telemetry vectors using an encoder model in order to generate a feature vector for the telemetry data; and process the feature vector using a decoder model in order to generate the natural language data.

BACKGROUND

Various methods, apparatuses, and systems are configured to provide techniques for generating natural language data based on telemetry data. Through applied effort, ingenuity, and innovation, the inventors have developed solutions for generating natural language data based on telemetry data.

BRIEF SUMMARY

In general, embodiments disclosed herein provide methods, apparatuses, systems, computing devices, and/or the like that are configured to enable generating natural language data based on telemetry data. For example, certain embodiments disclosed herein provide methods, apparatuses, systems, computing devices, and/or the like that are configured to generate natural language data based on telemetry data using encoder-decoder model.

In accordance with one aspect, a method is provided. In one embodiment, the method comprises receiving the telemetry data, wherein the telemetry data comprises a plurality of temporal telemetry vectors; processing the plurality of temporal telemetry vectors using an encoder model in order to generate a feature vector for the telemetry data, wherein the encoder model is configured to process each temporal telemetry vector of the plurality of temporal telemetry vectors at a corresponding encoding timestep of a plurality of encoding timesteps associated with the encoder model; and processing the feature vector using a decoder model in order to generate the natural language data, wherein the natural language data comprises a plurality of natural language tokens, and wherein the decoder model generates each natural language token of the plurality of natural language tokens at a corresponding decoding timestep of a plurality of decoding timesteps associated with the decoder model.

In accordance with another aspect, a computer program product is provided. The computer program product may comprise at least one computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising executable portions configured to receive the telemetry data, wherein the telemetry data comprises a plurality of temporal telemetry vectors; process the plurality of temporal telemetry vectors using an encoder model in order to generate a feature vector for the telemetry data, wherein the encoder model is configured to process each temporal telemetry vector of the plurality of temporal telemetry vectors at a corresponding encoding timestep of a plurality of encoding timesteps associated with the encoder model; and process the feature vector using a decoder model in order to generate the natural language data, wherein the natural language data comprises a plurality of natural language tokens, and wherein the decoder model generates each natural language token of the plurality of natural language tokens at a corresponding decoding timestep of a plurality of decoding timesteps associated with the decoder model.

In accordance with yet another aspect, an apparatus comprising at least one processor and at least one memory including computer program code is provided. In one embodiment, the at least one memory and the computer program code may be configured to, with the processor, cause the apparatus to receive the telemetry data, wherein the telemetry data comprises a plurality of temporal telemetry vectors; process the plurality of temporal telemetry vectors using an encoder model in order to generate a feature vector for the telemetry data, wherein the encoder model is configured to process each temporal telemetry vector of the plurality of temporal telemetry vectors at a corresponding encoding timestep of a plurality of encoding timesteps associated with the encoder model; and process the feature vector using a decoder model in order to generate the natural language data, wherein the natural language data comprises a plurality of natural language tokens, and wherein the decoder model generates each natural language token of the plurality of natural language tokens at a corresponding decoding timestep of a plurality of decoding timesteps associated with the decoder model.

DETAILED DESCRIPTION OF VARIOUS EMBODIMENTS

Overview

Various embodiments disclosed herein utilize encoder-decoder machine learning architectures to generate natural language data based on telemetry data in an efficient and reliable manner. By utilizing encoder-decoder machine learning architectures to generate natural language data based on telemetry data, various embodiments disclosed herein are able to perform maintenance need predictions as well as sensor deficiency predictions based on low quality maintenance logs. In some examples, the natural language data may be used to improve functionality, improve the user experience, and/or prevent catastrophic or otherwise costs conditions. In doing so, various embodiments disclosed herein significantly enhance monitoring of many real-world systems, such as heating, ventilation, and air-conditioning systems, based on sensory telemetry data.

Example System Architecture

FIG. 1illustrates an example system architecture100within which embodiments disclosed herein may operate. The architecture100includes a predictive inference system105configured to interact with one or more client computing devices102A-C, such as client computing device102A, client computing device B102B, and client computing device C102C. The predictive inference system105may be configured to generate natural language data for telemetry data received from a telemetry server computing device107, and provide the generated natural language data to the client computing devices102A-C. The telemetry data received by the predictive inference system105may be associated with one or more monitored systems, such as one or more heating, ventilation, and air-conditioning systems. The predictive inference system106may further be configured to train a machine learning model for generating natural language data based on maintenance log data stored in a storage subsystem108of the predictive inference system, where the maintenance log data may be stored on a maintenance log server computing device110.

In some embodiments, telemetry data refers to data obtained by recording readings of one or more sensor devices configured to monitor one or more monitored systems (e.g., a heating, ventilation, and air-conditioning system). Examples of sensor devices whose readings are used to generate telemetry data include bag filter sensors, on-coil temperature sensors, supply air temperature sensors, environment humidity sensors, fan angular motion sensors, etc. In some embodiments, telemetry data includes a set of time and value pairs for every sensor and/or every variable.

The predictive inference system105may communicate with the client computing devices102A-C using a network104. The network104may include any wired or wireless communication network including, for example, a wired or wireless local area network (LAN), personal area network (PAN), metropolitan area network (MAN), wide area network (WAN), or the like, as well as any hardware, software and/or firmware required to implement it (such as, e.g., network routers, etc.). For example, the network104may include a cellular telephone, an 802.11, 802.16, 802.20, and/or WiMax network. Further, the network104may include a public network, such as the Internet, a private network, such as an intranet, or combinations thereof, and may utilize a variety of networking protocols now available or later developed including, but not limited to Transmission Control Protocol/Internet Protocol (TCP/IP) based networking protocols. For instance, the networking protocol may be customized to suit the needs of the group-based communication system. In some embodiments, the protocol is a custom protocol of JavaScript Object Notation (JSON) objects sent via a Websocket channel. In some embodiments, the protocol is JSON over RPC, JSON over REST/HTTP, and the like.

The predictive inference system105may include a predictive inference computing device106and a storage subsystem108. The predictive inference computing device106may be configured to generate natural language data based on telemetry data. The storage subsystem108may be configured to store telemetry data as well as data associated with one or more predictive models utilized by the predictive inference computing device106. The storage subsystem108may include one or more storage units, such as multiple distributed storage units that are connected through a computer network. Each storage unit in the storage subsystem108may store at least one of one or more data assets and/or one or more data about the computed properties of one or more data assets. Moreover, each storage unit in the storage subsystem108may include one or more non-volatile storage or memory media including but not limited to hard disks, ROM, PROM, EPROM, EEPROM, flash memory, MMCs, SD memory cards, Memory Sticks, CBRAM, PRAM, FeRAM, NVRAM, MRAM, RRAM, SONOS, FJG RAM, Millipede memory, racetrack memory, and/or the like.

The predictive inference system105may receive the telemetry data from at least one of the client computing devices102A-C as well as a telemetry server computing device107. For example, a telemetry server computing device107may be a server device configured to monitor readings of one or more sensor devices associated with one or more monitored systems. An example of a telemetry server computing device107is a server device associated with a heating, ventilation, and air-conditioning system. The telemetry data may be stored in the storage subsystem108of the predictive inference system105. Examples of sensor devices whose readings recorded and transmitted by the telemetry server computing device107include bag filter sensors, on-coil temperature sensors, supply air temperature sensors, environment humidity sensors, fan angular motion sensors, etc.

Exemplary Predictive Inference Computing Device

The predictive inference computing device106may be embodied by one or more computing systems, such as apparatus200shown inFIG. 2. The apparatus200may include processor202, memory204, input/output circuitry206, and communications circuitry208. The apparatus200may be configured to execute the operations described herein. Although these components202-208are described with respect to functional limitations, it should be understood that the particular implementations necessarily include the use of particular hardware. It should also be understood that certain of these components202-208may include similar or common hardware. For example, two sets of circuitries may both leverage use of the same processor, network interface, storage medium, or the like to perform their associated functions, such that duplicate hardware is not required for each set of circuitries.

In some preferred and non-limiting embodiments, the processor202may be configured to execute instructions stored in the memory204or otherwise accessible to the processor202. In some preferred and non-limiting embodiments, the processor202may be configured to execute hard-coded functionalities. As such, whether configured by hardware or software methods, or by a combination thereof, the processor202may represent an entity (e.g., physically embodied in circuitry) capable of performing operations according to an embodiment disclosed herein while configured accordingly. Alternatively, as another example, when the processor202is embodied as an executor of software instructions, the instructions may specifically configure the processor202to perform the algorithms and/or operations described herein when the instructions are executed.

The communications circuitry208may be any means such as a device or circuitry embodied in either hardware or a combination of hardware and software that is configured to receive and/or transmit data from/to a network and/or any other device, circuitry, or module in communication with the apparatus200. In this regard, the communications circuitry208may include, for example, a network interface for enabling communications with a wired or wireless communication network. For example, the communications circuitry208may include one or more network interface cards, antennae, buses, switches, routers, modems, and supporting hardware and/or software, or any other device suitable for enabling communications via a network. Additionally or alternatively, the communications circuitry208may include the circuitry for interacting with the antenna/antennae to cause transmission of signals via the antenna/antennae or to handle receipt of signals received via the antenna/antennae.

Exemplary Client Computing Device

Referring now toFIG. 3, the client computing device102A-C may be embodied by one or more computing systems, such as apparatus300shown inFIG. 3. The apparatus300may include processor302, memory304, input/output circuitry306, and communications circuitry308. Although these components302-308are described with respect to functional limitations, it should be understood that the particular implementations necessarily include the use of particular hardware. It should also be understood that certain of these components302-310may include similar or common hardware. For example, two sets of circuitries may both leverage use of the same processor, network interface, storage medium, or the like to perform their associated functions, such that duplicate hardware is not required for each set of circuitries.

In some preferred and non-limiting embodiments, the processor302may be configured to execute instructions stored in the memory304or otherwise accessible to the processor302. In some preferred and non-limiting embodiments, the processor302may be configured to execute hard-coded functionalities. As such, whether configured by hardware or software methods, or by a combination thereof, the processor302may represent an entity (e.g., physically embodied in circuitry) capable of performing operations according to an embodiment disclosed herein while configured accordingly. Alternatively, as another example, when the processor302is embodied as an executor of software instructions (e.g., computer program instructions), the instructions may specifically configure the processor302to perform the algorithms and/or operations described herein when the instructions are executed.

The communications circuitry308may be any means such as a device or circuitry embodied in either hardware or a combination of hardware and software that is configured to receive and/or transmit data from/to a network and/or any other device, circuitry, or module in communication with the apparatus300. In this regard, the communications circuitry308may include, for example, a network interface for enabling communications with a wired or wireless communication network. For example, the communications circuitry308may include one or more network interface cards, antennae, buses, switches, routers, modems, and supporting hardware and/or software, or any other device suitable for enabling communications via a network. Additionally or alternatively, the communications circuitry308may include the circuitry for interacting with the antenna/antennae to cause transmission of signals via the antenna/antennae or to handle receipt of signals received via the antenna/antennae.

Exemplary Telemetry Server Computing Device

Referring now toFIG. 4, the telemetry server computing device107may be embodied by one or more computing systems, such as apparatus400shown inFIG. 4. The apparatus400may include processor402, memory404, input/output circuitry406, and communications circuitry408. Although these components402-408are described with respect to functional limitations, it should be understood that the particular implementations necessarily include the use of particular hardware. It should also be understood that certain of these components402-408may include similar or common hardware. For example, two sets of circuitries may both leverage use of the same processor, network interface, storage medium, or the like to perform their associated functions, such that duplicate hardware is not required for each set of circuitries.

In some embodiments, the processor402(and/or co-processor or any other processing circuitry assisting or otherwise associated with the processor) may be in communication with the memory404via a bus for passing information among components of the apparatus. The memory404is non-transitory and may include, for example, one or more volatile and/or non-volatile memories. In other words, for example, the memory404may be an electronic storage device (e.g., a computer-readable storage medium). The memory404may include one or more databases. Furthermore, the memory404may be configured to store information, data, content, applications, instructions, or the like for enabling the apparatus400to carry out various functions in accordance with example embodiments disclosed herein.

In some preferred and non-limiting embodiments, the processor402may be configured to execute instructions stored in the memory404or otherwise accessible to the processor402. In some preferred and non-limiting embodiments, the processor402may be configured to execute hard-coded functionalities. As such, whether configured by hardware or software methods, or by a combination thereof, the processor402may represent an entity (e.g., physically embodied in circuitry) capable of performing operations according to an embodiment disclosed herein while configured accordingly. Alternatively, as another example, when the processor402is embodied as an executor of software instructions (e.g., computer program instructions), the instructions may specifically configure the processor402to perform the algorithms and/or operations described herein when the instructions are executed.

In some embodiments, the apparatus400may include input/output circuitry406that may, in turn, be in communication with processor402to provide output to the user and, in some embodiments, to receive an indication of a user input. The input/output circuitry406may comprise a user interface and may include a display, and may comprise a web user interface, a mobile application, a query-initiating computing device, a kiosk, or the like. In some embodiments, the input/output circuitry406may also include a keyboard (e.g., also referred to herein as keypad), a mouse, a joystick, a touch screen, touch areas, soft keys, a microphone, a speaker, or other input/output mechanisms. The processor and/or user interface circuitry comprising the processor may be configured to control one or more functions of one or more user interface elements through computer program instructions (e.g., software and/or firmware) stored on a memory accessible to the processor (e.g., memory404, and/or the like).

The communications circuitry408may be any means such as a device or circuitry embodied in either hardware or a combination of hardware and software that is configured to receive and/or transmit data from/to a network and/or any other device, circuitry, or module in communication with the apparatus400. In this regard, the communications circuitry408may include, for example, a network interface for enabling communications with a wired or wireless communication network. For example, the communications circuitry408may include one or more network interface cards, antennae, buses, switches, routers, modems, and supporting hardware and/or software, or any other device suitable for enabling communications via a network. Additionally or alternatively, the communications circuitry408may include the circuitry for interacting with the antenna/antennae to cause transmission of signals via the antenna/antennae or to handle receipt of signals received via the antenna/antennae.

Example Data Flows and Operations

Various embodiments disclosed herein utilize encoder-decoder machine learning architectures to generate natural language data based on telemetry data in an efficient and reliable manner. By utilizing encoder-decoder machine learning architectures to generate natural language data based on telemetry data, various embodiments disclosed herein are able to perform maintenance need predictions as well as sensor deficiency detections based on. In some examples, the natural language data may be used to improve functionality, improve the user experience, and/or prevent undesired cost consequences. In doing so, various embodiments disclosed herein significantly enhance monitoring of many real-world systems, such as heating, ventilation, and air-conditioning systems, based on sensory telemetry data.

FIG. 5is a flowchart diagram of an example process500for performing predictive inference of natural language data based on telemetry data. Via the various operations ofFIG. 5, the predictive inference computing device106can perform, in some examples, efficient and reliable predictive inference of natural language data based on telemetry data. In some embodiments, performing efficient and reliable predictive inference of natural language data based on telemetry data enables the predictive inference computing device106to generate predicted log descriptions for equipment maintenance logs in one or more monitored systems, e.g., in a heating, ventilation, and air-conditioning system.

The process500begins at operation501when the predictive inference computing device106receives telemetry data comprising one or more temporal telemetry vectors. A temporal telemetry vector may be a collection of one or more values that each indicate a respective operational property of one or more monitored systems and/or one or more monitored instruments at a particular temporal unit (e.g., at a particular point in time and/or during a particular period of time). In some embodiments, the telemetry data is associated with a heating, ventilation, and air conditioning system. In some embodiments, the telemetry data is determined based on timeseries data depicting sensory outputs of one or more sensor devices associated with one or more monitored systems.

An operational example of timeseries data600that can be generated based on telemetry data is depicted inFIG. 6. As depicted inFIG. 6, the telemetry600includes a first timeseries graph601that depicts the values of a bag filter differential pressure sensor over time; a second timeseries graph602that depicts the values of an on-coil temperature sensor over time; and a third time-series graph603that depicts the values of a supply air temperature sensor over time.

Returning toFIG. 5, at operation502, the predictive inference computing device106processes the plurality of temporal telemetry vectors using an encoder recurrent neural network in order to generate a feature vector for the telemetry data, where the encoder recurrent neural network is configured to process each temporal telemetry vector of the plurality of temporal telemetry vectors at a corresponding encoding timestep of a plurality of encoding timesteps associated with the encoder recurrent neural network. In some embodiments, the encoder recurrent neural network is configured to, at each time step, process a temporal telemetry vector in accordance with one or more parameters of the encoder recurrent neural network in order to generate a hidden state for the time step and provide the generated hidden state to a subsequent time step of operation of the encoder recurrent neural network. In some embodiments, the feature vector for the telemetry data is generated based on the hidden state of an ultimate timestep of operation of the encoder recurrent neural network.

In some embodiments, the encoder recurrent neural network is a recurrent neural network configured to generate a feature vector for a given input telemetry data. In some embodiments, to generate the feature vector, the encoder recurrent neural network processes, at each time step, each temporal telemetry vector of the given input telemetry data in accordance with one or more parameters to generate a hidden state. In some of the noted embodiments, the feature vector is generated based on the hidden state of an ultimate timestep of the encoder recurrent neural network.

In some embodiments, the encoder recurrent neural network is trained in combination with a decoder recurrent neural network discussed in relation to operation503using a training method that utilizes gradient descent with backpropagation through time. In some embodiments, to train the encoder-decoder model comprising the encoder recurrent neural network and the decoder recurrent neural network discussed in relation to operation503, the predictive inference computing device106utilizes the maintenance logs configured to describe subsets of telemetry data. For example, the target output data may include human-generated maintenance log data generated after performing one or more maintenance operations, where determining whether each maintenance log is deemed to provide target output data for a telemetry data subset is determined based on temporal proximity between a time of a maintenance operation associated with the maintenance log and a time of the temporal units associated with the telemetry data subset.

In some embodiments, processing the plurality of temporal telemetry vectors using the encoder recurrent neural network in order to generate the feature vector for the telemetry data comprises, at each current encoding timestep of the plurality of encoding timesteps: (i) identifying a prior hidden state of an immediately prior encoding timestep of the plurality of encoding timesteps; (ii) processing the prior hidden state using one or more encoding parameters of the encoding recurrent neural network to generate a current hidden state; (iii) determining whether the current encoding timestep is a final encoding timestep of the plurality of encoding timesteps; (iv) in response to determining that the current encoding timestep is the final encoding timestep, determining the feature vector based on the current hidden state; and (v) in response to determining that the current encoding timestep is not the final encoding timestep, providing the current hidden state to an immediately subsequent encoding timestep of the plurality of encoding timesteps.

In some embodiments, the encoder recurrent neural network may be configured to receive the temporal telemetry vectors in accordance with a temporal order of temporal units associated with the temporal telemetry vectors, such that each temporal telemetry vector is processed in a time step that is after a timestep at which a temporal telemetry vector associated with a precedent temporal unit is processed but before a time step at which a temporal telemetry vector associated with a subsequent temporal unit is processed. In some embodiments, the encoder recurrent neural network is a conventional recurrent neural network or a long short-term memory (LSTM) recurrent neural network. In some embodiments, the parameters of the encoder recurrent neural network include one or more gate-specific parameters, such as gate-specific parameters associated with defined gates of an LSTM architecture. While various embodiments of the present invention have described an LSTM-based encoder model and an LSTM-based decoder model, a person of ordinary skill in the relevant technology will recognize that other machine learning models (such as other non-recurrent-neural-network-based machine learning models) can be used to construct each of the encoder model and the decoder model.

At operation503, the predictive inference computing device106processes the feature vector using a decoder recurrent neural network in order to generate the natural language data, where the natural language data comprises a plurality of natural language tokens, and where the decoder recurrent neural network generates each natural language token of the plurality of natural language tokens at a corresponding decoding timestep of a plurality of decoding timesteps associated with the decoder recurrent neural network. In some embodiments, the plurality of natural language tokens comprise an automated description text for the telemetry data. In some embodiments, the automated description text may be a maintenance log or a faulty state description.

In some embodiments, the decoder recurrent neural network is a recurrent neural network configured to process the feature vector to generate natural language tokens (e.g., natural language words) of a predicted natural language output. In some embodiments, the decoder recurrent neural network may generate, at each time step, a natural language token of the predicted natural language output. For example, the decoder recurrent neural network may generate, at a first time step, the natural language token “insufficient”; at a second time step, the natural language token “humidity”; at a third time step, the natural language token “excessive”; and at a fourth time step, the natural language token “heat”.

In some embodiments, at an initial timestep, the decoder recurrent neural network is configured to process the feature vector generated by the encoder recurrent neural network in order to generate an initial natural language token and an initial hidden state. In some embodiments, at each timestep other than the initial timestep and the final timestep, the decoder recurrent neural network is configured to process the natural language token and the hidden state generated by an immediately prior timestep in order to generate a natural language token and a hidden state for the timestep. In some embodiments, at each final timestep, the decoder recurrent neural network is configured to process the natural language token and the hidden state generated by a penultimate timestep in order to generate an end of sequence token for the plurality of natural language tokens.

In some embodiments, processing the feature vector using the decoder recurrent neural network in order to generate the natural language data comprises, at each current decoding timestep of the plurality of decoding timesteps: (i) identifying a prior hidden state of an immediately prior decoding timestep of the plurality of decoding timesteps; (ii) identifying a prior natural language token of the plurality of natural language token that is generated by the immediately prior decoding timestep; (iii) processing the prior hidden state and the prior natural language token to generate a current hidden state and a current natural language token; (iv) determining whether the current decoding timestep is a final decoding timestep of the plurality of decoding timesteps; and (v) in response to determining that the current decoding timestep is not the final decoding timestep, providing the current hidden state and the current natural language token to an immediately subsequent decoding timestep of the plurality of decoding timesteps.

In some embodiments, the decoder recurrent neural network is configured to generate the natural language tokens in an order that reflects the sequential ordering of natural language data. For example, the initial timestep of the decoder recurrent neural network may generate a first word of the natural language data, the second timestep of the decoder recurrent neural network may generate a second word of the natural language data, etc. In some embodiments, the decoder recurrent neural network is a conventional recurrent neural network or an LSTM recurrent neural network. In some embodiments, the parameters of the decoder recurrent neural network include one or more gate-specific parameters, such as gate-specific parameters associated with defined gates of an LSTM architecture.

FIG. 7depicts an operational example of an encoder-decoder architecture700configured to generate natural language data based on telemetry data. As depicted inFIG. 7, the encoder-decoder architecture700includes an encoder recurrent neural network701and a decoder neural network702. As further depicted inFIG. 7, the encoder recurrent neural network701is configured to perform data processing during various timesteps (e.g., encoding timesteps711-714) and based on input data provided at each timestep by the telemetry data703(e.g., telemetry data600ofFIG. 6) in order to generate the feature vector721. Moreover, the decoder recurrent neural network702is configured to perform data processing during various timesteps (e.g., decoding timesteps731-734) and based on the feature vector721in order to generate natural language tokens (e.g., natural language tokens741-744).

For example, in a first decoding timestep731, the decoder recurrent neural network702is configured to process the feature vector721, and the start token, in accordance with the parameters of the decoder recurrent network702in order to generate a hidden state, provide the generated hidden state to the second timestep732of the decoder recurrent neural network702, and generate (e.g., using one or more softmax operations in accordance with an available vocabulary of candidate natural language tokens) the natural language token741(i.e., “bag”) based on the generated hidden state of the first decoding timestep731.

As another example, in the second decoding timestep732of the decoder recurrent neural network702depicted inFIG. 7, the decoder recurrent neural network702is configured to process the hidden state generated by the first time step731and the encoded natural language token741generated by the first time step731in accordance with the parameters of the decoder recurrent network702in order to generate a hidden state, provide the generated hidden state to the third timestep733of the decoder recurrent neural network702, and generate (e.g., using one or more softmax operations in accordance with an available vocabulary of candidate natural language tokens) the natural language token742(i.e., “filter”) based on the generated hidden state of the second decoding timestep732.

As yet another example, in the third decoding timestep733of the decoder recurrent neural network702depicted inFIG. 7, the decoder recurrent neural network702is configured to process the hidden state generated by the second time step732and the encoded natural language token742generated by the second time step732in accordance with the parameters of the decoder recurrent network702in order to generate a hidden state, provide the generated hidden state to the fourth timestep734of the decoder recurrent neural network702, and generate (e.g., using one or more softmax operations in accordance with an available vocabulary of candidate natural language tokens) the natural language token743(i.e., “dirty”) based on the generated hidden state of the third decoding timestep733.

As a further example, in the fourth decoding timestep734, the decoder recurrent neural network702is configured to process the hidden state generated by the third time step733and the encoded natural language token743generated by the third time step733in accordance with the parameters of the decoder recurrent network702in order to generate a hidden state and generate (e.g., using one or more softmax operations in accordance with an available vocabulary of candidate natural language tokens) the natural language token744(i.e., the end-of-sequence natural language token) based on the generated hidden state of the fourth decoding timestep733.

Once generated in accordance with the process500ofFIG. 5, the natural language data can be utilized to perform various prediction-based actions. For example, in some embodiments, the predictive inference computing device106may be configured to determine a measure of similarity between the natural language data and representative description data for the telemetry data; determine whether the measure of similarity exceeds a similarity threshold; and in response to determining that the measure of similarity does not exceed the similarity threshold, adopt the natural language data as the representative description data for the telemetry data. For example, if the human-generated maintenance log for a maintenance period differs significantly from the automatically-generated maintenance log for the maintenance period that is determined based on the telemetry data for the maintenance period, the predictive inference computing device106may adopt the automatically-generated maintenance log as the maintenance log for the maintenance period and/or supplement the human-generated maintenance log for the maintenance period with at least some of the data associated with the automatically-generated maintenance log for the maintenance period. In some embodiments, if the human-generated maintenance log for a maintenance period differs significantly from the automatically-generated maintenance log for the maintenance period that is determined based on the telemetry data for the maintenance period, the predictive inference computing device106may generate feedback for maintenance personnel that indicates that their provided feedback was wrong or insufficient.

In some embodiments, feature vectors generated based on telemetry data can be used to determine similarities between time periods, which in turn can be utilized to assign predictive labels to the natural language data for the noted time periods. For example, if the feature vector for a first time period and the feature vector for a second time period are deemed sufficiently similar, the predictive inference computing device106may assign common predictive labels to the natural language data for the two time periods. In some embodiments, common predictive labels for two or more time periods may be determined based on preconfigured labeling data and/or based on output of a decoder recurrent neural network configured to generate predictive labels for time periods based on feature vectors for those time periods, where the feature vectors for time periods are in turn generated based on telemetry data for the noted time periods. In some embodiments, when the telemetry data is associated with a particular time period of a plurality of time periods, the predictive inference computing device106is configured to determine, based on the feature vector and each time period feature vector for a time period of the plurality of time periods, one or more related time periods of the plurality of time periods; determine a common predictive label for the plurality of time periods; and update description data for the particular time period to reflect the common predictive label.

In some embodiments, the predictive inference computing device106generates parameters of the encoder model and the decoder model using a training routine that seeks to minimize an error between: (i) inferred natural language tokens generated by the encoder model and the decoder model based on training telemetry data and (ii) representative telemetry description data (e.g., representative maintenance logs) for the training telemetry data. In some embodiments, the predictive inference computing device106determines, based on the similarity of the feature vector for the particular time period and each time period feature vector for a time period of a plurality of time periods, one or more related time periods of the plurality of time periods; and updates description data (e.g., maintenance logs) associated with the plurality of time periods to be associated with the particular time period instead.

In some embodiments, frequent lack of detection of particular issues referenced in human-generated natural language data can be used to make inferences about on observability of the particular issues given current sensory infrastructure. For example, if automatic analyses of telemetry data frequently fail to detect particular issues (e.g., noise or smell issues), the predictive inference computing device106may infer that the sensory infrastructure for detecting the particular issues is inadequate.

In some embodiments, the recall ratio of a predictive issue is determined based on a ratio of lack of detection of the predictive issue based on telemetry data for a particular time period when the predictive issue is referenced in human-generated natural language data for the particular time period. In some embodiments, subsequent to determining sensor adjustment actions, the predictive inference computing device106is configured to perform the sensor adjustment actions. Examples of sensor adjustment actions include automated ordering of new sensors, automated activations of already-installed sensors, automated generation of maintenance-related notifications, automated scheduling of maintenance appointments, etc. In some embodiments, the natural language data generated based on telemetry data can be used to predict upcoming maintenance needs associated with one or more monitored systems. This can in turn enable the predictive inference computing device106to perform prediction-based actions in anticipation of the predicted upcoming maintenance needs.

In some embodiments, the natural language generation concepts discussed herein may be utilized to detect one or more operational conditions of a heating, ventilation, and air-conditioning system. For example, based on telemetry data indicating low air supply temperatures, the predictive inference computing device106may detect a defect in heating capabilities of a heating, ventilation, and air-conditioning system. As another example, based on telemetry data indicating high air supply temperatures, the predictive inference computing device106may detect a defect in cooling capabilities of a heating, ventilation, and air-conditioning system. As a further example, based on telemetry data indicating high humidity inside a cooling plant of a heating, ventilation, and air-conditioning system, the predictive inference computing device106may detect a leaking inside the cooling plant.

Additional Implementation Details

Although example processing systems have been described in the figures herein, implementations of the subject matter and the functional operations described herein can be implemented in other types of digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them.

Embodiments of the subject matter and the operations described herein can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described herein can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded on computer-readable storage medium for execution by, or to control the operation of, information/data processing apparatus. Alternatively, or in addition, the program instructions can be encoded on an artificially-generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, which is generated to encode information/data for transmission to suitable receiver apparatus for execution by an information/data processing apparatus. A computer-readable storage medium can be, or be included in, a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them. Moreover, while a computer-readable storage medium is not a propagated signal, a computer-readable storage medium can be a source or destination of computer program instructions encoded in an artificially-generated propagated signal. The computer-readable storage medium can also be, or be included in, one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices).

The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some embodiments, a server transmits information/data (e.g., a Hypertext Markup Language (HTML) page) to a query-initiating computing device (e.g., for purposes of displaying information/data to and receiving user input from a user interacting with the query-initiating computing device). Information/data generated at the query-initiating computing device (e.g., a result of the user interaction) can be received from the query-initiating computing device at the server.

Conclusion

Many modifications and other embodiments of the inventions set forth herein will come to mind to one skilled in the art to which these inventions pertain having the benefit of the teachings presented in the foregoing description and the associated drawings. Therefore, it is to be understood that the inventions are not to be limited to the specific embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation, unless described otherwise.