Patent Description:
The present disclosure generally relates to remote analytics systems and more specifically to computing architectures providing low latency analytics and control of devices via edge nodes using edge communication links.

As network technologies continue to advance, both in terms of accessibility and connectivity, the utilization of networks has also expanded. As an example, mobile devices (e.g., cellular communication devices, tablet computing devices, or other handheld electronic devices) were initially limited to certain types of networks (e.g., cellular voice networks) but as cellular communication networks advanced, the capabilities of mobile devices also expanded to include data applications and other functionality. The expanded capabilities of cellular communication networks have become widely available in recent years in certain developed countries and continue to expand in other regions of the world, which have created new ways to utilize various types of network technologies.

Despite the increases in data rates provided by cellular communication and traditional data networks (e.g., broadband, fiber, and Wi-Fi networks), computing resources remain a limiting factor with respect to certain types of processing and functionality. For example, despite increases in computing hardware capabilities, edge computing devices typically remain limited with respect to computing resources (e.g., processor computational capabilities, memory, etc.) as compared to traditional types computing devices (e.g., servers, personal computing devices, laptop computing devices, and the like). As a result of the computing resources limitations of edge computing devices, the edge computing functionality has remained limited and resulted in use of more centralized, non-edge computing devices for many applications. While such computing devices and setups have benefited from the increases to speed and connectivity of existing networks, certain types of applications and functionality (e.g., computer vision-based applications) remain unacceptably slow due to latency and other factors associated with use of traditional network technologies despite the availability of powerful computing hardware. <CIT> relates to systems and methods that allow for dense depth map estimation given input images. According to the US application, embodiments of the deep neural network model comprise computationally efficient structures and fewer layers but still produce good quality results. Further, according to the US application, the deep neural network model is specially configured and trained to operate using a hardware accelerator component or components that can speed computation and produce good results, even if lower precision bit representations are used during computation at the hardware accelerator component.

Dependent claims define preferred embodiments. The present disclosure provides a computing architecture that enables computer vision and other analytical techniques to be provided in a manner that provides for low latency/rapid response by leveraging edge computing devices. In an aspect, sensor devices (e.g., cameras, temperature sensors, motion sensors, etc.) may be disposed in an environment and may capture information that may be analyzed to evaluate a state of the environment or a state of one or more devices and/or persons within the environment. Information recorded by the sensor devices may be transmitted to an edge node using an edge communication link, such as a communication link provided over a next generation network, such as a 5th Generation (<NUM>) communication network. The edge node may implement a computing architecture in accordance with the present disclosure that leverages multiple independent threads processing input data streams in parallel to perform analysis of the environment. The multiple independent threads may include threads executed by a central processing unit (CPU) of the edge node, such as to perform data reception and initial processing of the input data to prepare the input data streams for analysis via one or more machine learning models (e.g., computer vision models). Additionally, the multiple independent threads may include threads executed by a graphics processing unit (GPU) for evaluating model input data (i.e., the results of the pre-processing of the input data) against the one or more machine learning models. The one or more machine learning models may be configured to analyze the model input data according to one or more specific use cases (e.g., to determine whether a worker is wearing appropriate safety equipment or is operating machinery in an appropriate manner), and may generate model outputs for further analysis.

The model outputs may be evaluated using additional independent threads of the CPU and control logic configured to generate control data and outcome data. The control data may be used by one or more threads of a message broker service executing on the CPU to generate command messages for controlling remote devices or notifying users of situations within an environment (e.g., to slow or turn off a remote device or warn a user of unsafe conditions). The data utilized by the various analytics processes may be maintained locally at the edge node in cache memory to facilitate rapid access to the relevant data and a longer term storage may be used to store analytics data for a period of time. The relevant data stored in the longer term storage of the edge node may be used to present information in a graphical user interface and may be periodically transferred to an external system (e.g., a central server or other non-edge computing device).

The foregoing has outlined rather broadly the features and technical advantages of the present invention in order that the detailed description of the invention that follows may be better understood. Additional features and advantages of the invention will be described hereinafter which form the subject of the claims of the invention. It should be appreciated by those skilled in the art that the conception and specific embodiment disclosed may be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the present invention, as long as they do not depart from the invention which is defined by the appended claims. The novel features which are believed to be characteristic of the invention, both as to its organization and method of operation, together with further objects and advantages will be better understood from the following description when considered in connection with the accompanying figures. It is to be expressly understood, however, that each of the figures is provided for the purpose of illustration and description only and is not intended as a definition of the limits of the present invention.

For a more complete understanding of the disclosed methods and apparatuses, reference should be made to the embodiments illustrated in greater detail in the accompanying drawings, wherein:.

It should be understood that the drawings are not necessarily to scale and that the disclosed embodiments are sometimes illustrated diagrammatically and in partial views. In certain instances, details which are not necessary for an understanding of the disclosed methods and apparatuses or which render other details difficult to perceive may have been omitted. It should be understood, of course, that this disclosure is not limited to the particular embodiments illustrated herein.

Embodiments of the present disclosure provide a computing architecture that facilitates rapid analysis and control of an environment via edge computing nodes. Input data streams may be received at an edge node and prepared for processing by one or more machine learning models. The machine learning models may be trained according to different use cases to facilitate a multi-faceted and comprehensive analysis of the input data. The input data may be evaluated against the machine learning models to produce model outputs that are then evaluated using control logic to produce a set of outcomes and control data. The control data may be utilized to generate one or more command messages or control signals that may be used to provide feedback to a remote device or user regarding a state of a monitored environment or other observed conditions. To improve the throughput of the analytics process, the evaluation of the input data against the machine learning models may be performed on a separate processor than other computing processes. For example, the reception of the input data (and pre-processing of the input data for use with the machine learning models) may be performed using one or more threads running on first processor (e.g., a central processing unit (CPU)) while independent threads running on a second processor (e.g., a graphics processing unit (GPU)) may be utilized for each of the machine learning models. Additionally, independent threads running on the first processor may also be utilized to evaluate the model outputs and produce the control and outcome data, as well as to facilitate generation of command messages. As described in more detail below, the disclosed computing architecture enables computer vision-type analytics and other analytical processes to be performed via edge computing nodes in a manner that is significantly faster than existing techniques.

Referring to <FIG>, a block diagram illustrating a system for performing low latency edge computing analytics in accordance with aspects of the present disclosure is shown as a system <NUM>. The system <NUM> provides a system architecture that enables video and other types of analytics to be determined in a rapid fashion by leveraging edge nodes, such as edge node <NUM>. As shown in <FIG>, the edge node <NUM> includes one or more processors <NUM>, a memory <NUM>, a modelling engine <NUM>, one or more edge service modules <NUM>, and one or more communication interfaces <NUM>. The one or more processors <NUM> include a CPU or other computing circuitry (e.g., a microcontroller, one or more application specific integrated circuits (ASICs), and the like). The one or more processors <NUM> also include a GPU. As described in more detail with reference to <FIG>, the functionality provided by the modelling engine <NUM> may be executable by the GPU and the functionality provided by the one or more edge services <NUM> may be executable by the CPU.

The memory <NUM> may include read only memory (ROM) devices, random access memory (RAM) devices, one or more hard disk drives (HDDs), flash memory devices, solid state drives (SSDs), other devices configured to store data in a persistent or non-persistent state, or a combination of different memory devices. The memory <NUM> may store instructions <NUM> that, when executed by the one or more processors <NUM>, cause the one or more processors <NUM> to perform the operations described in connection with the edge device <NUM> with reference to <FIG>. For example, the instructions <NUM> may include instructions that correspond to the edge services <NUM> and are executable by the one or more CPUs to provide the functionality of the edge services <NUM>. The instructions <NUM> may additionally include instructions that correspond to the modelling engine <NUM> and are executable by the one or more GPUs to provide the functionality of the modelling engine <NUM>. Exemplary aspects of the functionality and operations of the modelling engine <NUM> and the edge services <NUM> are described in more detail below with reference to <FIG>. In addition to the instructions <NUM>, the memory <NUM> may also store information in one or more databases <NUM>. In some aspects, edge node <NUM> may include one or more I/O devices (e.g., one or more display devices, a keyboard, a stylus, one or more touchscreens, a mouse, a trackpad, a camera, one or more speakers, haptic feedback devices, or other types of devices that enable a user to receive information from or provide information to the edge node <NUM>).

The one or more communication interfaces <NUM> may communicatively couple the edge node <NUM> to remote computing devices <NUM>, <NUM> via one or more networks <NUM>. In an aspect, the edge node <NUM> may be communicatively coupled to the computing devices <NUM>, <NUM> via wired or wireless communication links according to one or more communication protocols or standards (e.g., an Ethernet protocol, a transmission control protocol/internet protocol (TCP/IP), an institute of electrical and electronics engineers (IEEE) <NUM> protocol, and an IEEE <NUM> protocol, and the like). In addition to being communicatively coupled to the computing devices <NUM>, <NUM> via the one or more networks <NUM>, the one or more communication interfaces <NUM> may communicatively couple edge node <NUM> to one or more sensor devices, such as sensor devices 150A-150C, or monitored devices, such as device <NUM>. The edge node <NUM> may be communicatively coupled to the sensor devices 150A-150C and the devices(s) <NUM> via an edge communication link (e.g., a communication link established according to a 4th Generation (<NUM>)/long term evolution (LTE) communication standard, a 5th Generation (<NUM>) communication standard).

As shown in <FIG>, the computing device <NUM> may include one or more processors <NUM> and a memory <NUM>. The one or more processors <NUM> include one or more CPUs, one or more GPUs, or other computing circuitry (e.g., a microcontroller, one or more ASICs, and the like). The memory <NUM> may include ROM devices, RAM devices, one or more HDDs, flash memory devices, SSDs, other devices configured to store data in a persistent or non-persistent state, or a combination of different memory devices. The memory <NUM> may store instructions that, when executed by the one or more processors <NUM>, cause the one or more processors <NUM> to perform the operations described in connection with the computing device <NUM> with reference to <FIG>. For example, the instructions may include instructions that correspond to the analytics engine <NUM> and the monitoring engine(s) <NUM>. In addition to the instructions, the memory <NUM> may also store information in one or more databases <NUM>. The information stored at database <NUM> may be similar to the information stored in the database <NUM>. Additionally or alternatively, the information stored at database <NUM> may be different from the information stored in the database <NUM>. In some aspects, the computing device <NUM> may include one or more I/O devices (e.g., one or more display devices, a keyboard, a stylus, one or more touchscreens, a mouse, a trackpad, a camera, one or more speakers, haptic feedback devices, or other types of devices that enable a user to receive information from or provide information to the computing device <NUM>). The computing device <NUM> may also include one or more analytics engines <NUM>, and one or more monitoring engines <NUM>, described in more detail below.

Sensor devices 150A-150C may include cameras (e.g., video cameras, imaging cameras, thermal cameras, etc.), temperature sensors, pressure sensors, acoustic sensors (e.g., ultrasound sensors, transducers, microphones, etc.), motion sensors (e.g., accelerometers, gyroscopes, etc.), or other types of devices capable of capturing and recording information associated with the device <NUM>. For example, device <NUM> may be a drill press, a saw, or other type of equipment and the sensor devices 150A-150C may monitor the state of the device <NUM>, the environment surrounding the device <NUM>, or other factors. The sensor devices 150A-150C may capture information that may be provided to the edge node <NUM> for analysis to determine whether a hazard condition is present in the vicinity of the device <NUM> (e.g., a user has a body part too close to the saw, etc.). The edge node <NUM> may evaluate the information captured by the sensor devices 150A-150C using the modelling engine <NUM> and may determine whether to transmit commands to the device <NUM> based the evaluating. For example, where a hazardous or dangerous condition is detected, the edge services <NUM> may transmit a command to the device <NUM> to cause the device <NUM> turn off or modify one or more operating parameters, thereby creating a safer environment and reducing the likelihood of an accident. Exemplary techniques for analyzing the information captured by the sensor devices 150A-150C and for exchanging commands with the device <NUM> via the edge services <NUM> are described in more detail below with reference to <FIG>.

In addition to leveraging edge node <NUM> to facilitate rapid analysis of data captured by sensor devices 150A-150C and providing feedback or commands to the device <NUM> (or other devices), the system <NUM> may also enable users to remotely monitor the status of one or more devices (e.g., one or more devices <NUM>) and environments where the devices are operating. For example, a user may utilize computing device <NUM> to access one or more graphical user interfaces supported by computing device <NUM>. The one or more graphical user interfaces may be configured to present information about the environment(s) and device(s) within the environment(s) to the user. Exemplary aspects of the types of information that may be provided to the user via the graphical user interface(s) and other functionality provided via the graphical user interfaces are described in more detail below.

As shown in <FIG>, the computing device <NUM> may include one or more processors <NUM> and a memory <NUM>. The one or more processors <NUM> include one or more CPUs, one or more GPUs, or other computing circuitry (e.g., a microcontroller, one or more ASICs, and the like). The memory <NUM> may include ROM devices, RAM devices, one or more HDDs, flash memory devices, SSDs, other devices configured to store data in a persistent or non-persistent state, or a combination of different memory devices. The memory <NUM> may store instructions <NUM> that, when executed by the one or more processors <NUM>, cause the one or more processors <NUM> to perform the operations described in connection with the computing device <NUM> with reference to <FIG>. In some aspects, computing device <NUM> may include one or more I/O devices (e.g., one or more display devices, a keyboard, a stylus, one or more touchscreens, a mouse, a trackpad, a camera, one or more speakers, haptic feedback devices, or other types of devices that enable a user to receive information from or provide information to the computing device <NUM>).

As briefly described above, the edge node <NUM> is configured to receive information about a monitored environment, such as information captured by the sensor devices 150A-150C. The monitored environment may include one or more devices, such as the device <NUM>, and the edge services <NUM> of the edge node <NUM> may be configured to analyze the information received from the sensor devices 150A-150C and determine whether to issue one or more commands to devices within the monitored environment. A computing architecture of the edge node <NUM> may be configured to enable rapid analysis of the received information and to enable the commands to be issued, where appropriate based on the analysis, to the devices of the monitored environment in real-time or near-real-time. For example, the computing architecture of the edge node <NUM> may enable the information to be received from the sensor devices 150A-150C, analyzed, and commands to be issued to and received at the device <NUM> within a threshold period of time. In an aspect, the threshold period of time may be less than <NUM> milliseconds (ms). In an additional or alternative aspect, the threshold period of time may be less <NUM>. In yet another additional or alternative aspect, the threshold period of time may be less than <NUM>. In some aspects, the threshold period of time may be between <NUM> and <NUM> (e.g., <NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>, and the like). In some aspects, the threshold period of time may be approximately <NUM>.

Referring to <FIG>, a block diagram illustrating exemplary aspects of a computing architecture facilitating rapid execution of computational services via an edge node in accordance with the present disclosure is shown as a computing architecture <NUM>. The exemplary computing architecture <NUM> shown in <FIG> may be utilized by an edge node, such as edge node <NUM> of <FIG>, to provide functionality in connection with monitoring an environment, such as an environment that includes sensor devices (e.g., the sensor devices 150A-150C of <FIG>) and devices (e.g., device <NUM> of <FIG>). The devices of the monitored environment may include tools (e.g., drill presses, saws, and the like) or other types of machinery and the functionality provided by the computing architecture <NUM> may enable various types of operations for monitoring and managing the environment, such as to monitor the status of the devices and safety of users within the environment.

As shown in <FIG>, the computing architecture <NUM> may provide services, such as a capture service <NUM> and a message broker service <NUM>. In an aspect, the capture service <NUM> and the message broker service <NUM> may be included in the edge services <NUM> of <FIG>. The capture service <NUM> may be configured to receive and process information from sensor devices (e.g., the sensor devices 150A-150C of <FIG>). For example, the sensor devices may include a camera providing video frame data <NUM>, a camera providing video frame data <NUM>, and a temperature sensor providing temperature data <NUM>. The video frame data <NUM> may include frames of video data (e.g., video frames A<NUM>-An) captured by the camera over a period of time (n); the video frame data <NUM> may include frames of video data (e.g., video frames B<NUM>-Bn) captured by the camera over the period of time (n); and the temperature data <NUM> may include temperature measurements (e.g., temperatures C<NUM>-Cn) captured by the temperature sensor over the period of time (n). It is noted that the video frames <NUM> and the video frames <NUM> may correspond to video streams captured from different angles and a device of interest (e.g., the device <NUM> of <FIG>) may be within the field of view of both video streams. The temperature data may include temperature information associated with the device of interest, which may enable high temperature or overheat conditions to be detected. It is noted that the information received from different sensor devices may have the same or different capture intervals. For example, the video frame data <NUM>, <NUM> may be captured at the same frame rate (e.g., <NUM> frames per second (fps), <NUM> fps) or at different frame rates (e.g., the video frame data <NUM> may be captured at <NUM> fps and the video frame data <NUM> may be captured at <NUM> fps). Additionally, the temperature information <NUM> may be captured once every second, once every minute, etc. In such a scenario, one instance of temperature data <NUM> may be associated with a time interval corresponding to multiple instances of video frame data <NUM>, <NUM> (e.g., one temperature measurement may provide temperature information for the device during multiple frames of video data).

As the various types of information are captured by the capture service <NUM>, information associated with the captured information may be stored in a cache memory <NUM> (e.g., a cache memory of the memory <NUM> of <FIG>). In some aspects, the captured information may be processed prior to being stored at the cache memory <NUM>. For example, the video frame data <NUM>, <NUM> may be processed to prepare the video frames for ingestion by a machine learning model, such as a computer vision model generated by the modelling engine <NUM> of <FIG>. The processing of the video frame data may include converting each frame of video data into an array or matrix of numeric values representing the pixels of the video frame (e.g., a numeric value representing to color or gray scale level of the pixels, luminance, and the like), normalization, down-sampling, scaling, or other processes that enable each of the video frames to be converted to a form that may be input into a computer vision model. The information received and processed by the capture service <NUM> may be stored in the cache memory <NUM>. For example, the video frame data <NUM> and the video frame data <NUM> may be stored as processed video frame data <NUM>' and <NUM>'. In some aspects, some of the information captured by the capture service <NUM> may include information that is not used as an input to the computer vision model(s), such as the temperature data <NUM>, and may be stored in the cache memory <NUM> without any further processing.

As shown in <FIG>, the computing architecture <NUM> includes a GPU module <NUM>. The GPU module <NUM> may be configured to evaluate at least a portion of the data captured by the capture service <NUM> using one or more machine learning models, such as the above-described computer vision models. For example, the GPU module <NUM> may include one or more models <NUM>. In the specific example shown in <FIG> the models <NUM> include y models (e.g., model (M<NUM>, M<NUM>,. It is noted that the particular number of models "y" may depend on the particular use case to which the computing architecture <NUM> is applied and that different use cases may utilize a different number and type of models.

Each of the models <NUM> may be configured to evaluate input data of a particular type (e.g., image or video frame data, etc.) according to a particular use case. Moreover, the models <NUM> configured to analyze image data may be trained using data captured from a particular viewing angle, such as the viewing angle associated with the video frame data <NUM> or the viewing angle associated with the video frame data <NUM>. Using training data captured from different viewing angles may enable the models <NUM> to be trained to identify relevant use case scenarios in a more accurate manner. For example, where the use case involves monitoring safety of a worker utilizing a drill press, the models <NUM> may be configured to evaluate whether the worker is safely operating the drill press and detect when an unsafe operating condition occurs. Information from the video frame data <NUM> and video frame data <NUM> may be captured from different angles to more effectively monitor the safety of the environment where the worker and drill press are located. For example, the viewing angle associated with the video frame data <NUM> may show normal/safe operation of the drill press by the worker but the viewing angle associated with the video frame data <NUM> may show unsafe operation of the drill press by the worker. In such a situation, the model evaluating the video frame data <NUM>' may determine that normal operating conditions are occurring and the model evaluating the video frame data <NUM>' may determine that an unsafe operating condition is occurring. It is noted that the models may not be configured to actually evaluate whether the video frame data indicates "safe" or "unsafe" operating conditions and instead may simply classify the scene depicted in the video frame data. For example, the models <NUM> may be configured to classify the video frame data into one of a plurality of classifications, such as drill press off, worker not present, worker's hands away from drill press, worker's hand(s) near drill press but not on handles of drill press, worker's hand(s) near drill press but on handles of drill press, etc..

It is noted that the models <NUM> of the GPU module <NUM> may include models configured to perform different types of analysis, which may include different types of analysis on a same dataset. For example, a set of video frame data may be analyzed by the GPU module <NUM> using two different models, each model trained to identify different scenario information (e.g., a worker's hand in an unsafe position with respect to a drill press and whether the worker is wearing appropriate safety gear, such as a hard hat, gloves, eyewear, etc.). Utilizing different models to analyze a same stream of video frame data may enable the models to be maintained in a more compact manner and provide for efficient processing of video frame data in a rapid fashion as compared to trying to use a single (larger) model to evaluate all potential types of information that may be derived from a particular set of video frame data. Accordingly, it should be understood that a single set of input video frame data (or another type of data) may be analyzed using a single model or multiple models depending on the particular configuration of the GPU module <NUM> and the use cases being considered.

As the cached data (e.g., the video frame data <NUM>', <NUM>') is evaluated against the models <NUM>, outputs associated with the classifications derived from analysis of the cached data may be produced. For example, evaluation of video frame data <NUM>' by model M<NUM> may produce a classification {A}, evaluation of video frame data <NUM>' by model M<NUM> may produce a classification {B}, and so on. The classifications output by the GPU module <NUM> may be stored at the cache memory <NUM> as classifications <NUM>. The classifications <NUM> may be evaluated by control logic <NUM> to determine a state of the monitored environment, such as whether the drill press in the above-described scenario is being operated safely. For example, the control logic <NUM> may configured with various logic parameters <NUM> (e.g., L<NUM>, L<NUM>,. , Lz) configured to evaluate the classifications <NUM>. In the example above, the control logic parameters <NUM> may be applied to or used to evaluate the classifications <NUM> (or other outputs of the GPU module <NUM>) to produce control data. The control data generated by control logic <NUM> may include different sets of data, such as a first set of data providing control information and a second set of data corresponding to analysis outcomes. In <FIG>, the first set of data (e.g., control data) is shown as " {A <NUM>} {B1} {C1}" (224A) and the second set of data (e.g., analysis outcomes) is shown as "{A2}{B2}{C2}" (224B) and may be stored in the cache memory <NUM> as control logic outputs <NUM>. The first set of data may be provided to the message broker service <NUM> where it may be used to generate one or more command messages, such as command message <NUM>. The command message <NUM> may be provided to an external device, such as the device <NUM> of <FIG> or the computing device <NUM> of <FIG>.

For example, in the above-example involving a drill press, the command message may be configured for delivery to the drill press or a device coupled to the drill press (e.g., a network enabled device configured to provide control functionality for the drill press) and may include command data to control operations the drill press. For example, where the control logic <NUM> determines, based on application of the logic parameters <NUM> to the classifications <NUM>, that the drill press is being operated in an unsafe manner, the command message <NUM> may include commands to slow or stop the drill press, a command to generate an auditory alert to the drill press operator, or other types of operations to address the unsafe operating conditions detected by the control logic <NUM>. The command message <NUM> may transmitted to the device by the message broker service <NUM> via the edge communication link. The second set of data (e.g., "{A2} {B2} {C2}") may be stored in a database <NUM>, which may be one of the one or more databases <NUM> of <FIG>. The information stored in the database <NUM> may maintained for a period of time, such as one hour, and after the period of time may be transferred to a long-term data storage, such as the one or more databases <NUM> of <FIG>. Storing the information in the database <NUM> may enable the data to be access rapidly directly from the edge node implementing the computing architecture <NUM> without requiring utilization of higher latency networks and systems, such as the computing device <NUM> of <FIG>. It is noted that the period of time for which data is retained in the database <NUM> may be configurable (e.g., a by a user or system administrator) or may be dynamic (e.g., based on available memory space).

In the exemplary flow shown in <FIG>, various hardware and software techniques may be utilized to increase the speed at which information is processed. For example, functionality provided by the capture service <NUM>, the control logic <NUM>, and the message broker service <NUM> may be executed on a CPU and the functionality provided by the GPU module <NUM> may be executed using a GPU. Utilizing the GPU to evaluate the machine learning models against the input data (e.g., the video frame data or other types of data) may enable the computer vision techniques or other artificial intelligence processes to be performed more rapidly. Additionally, utilizing a CPU to perform the functionality provided by the capture service <NUM>, the control logic <NUM>, and the message broker service <NUM> may also enable those functions to be performed more efficiently.

To further streamline the processing flow, multi-threaded processing may be utilized. For example, each incoming data stream (e.g., the data streams associated with the video frame data <NUM>, <NUM>, and the temperature information <NUM>) may be handled by processes performed by the CPU and/or the GPU via a separate thread. Utilizing different threads in the CPU and GPU enables parallel execution of various processes for different data streams and analysis, allowing multiple use cases or perspectives (e.g., different viewing angles for computer vision processes, etc.) to be considered simultaneously. Additionally, the functionality provided by the different threads executed in parallel produce optimized outputs that are appropriate for the next step of processing, such as pre-processing the video data to a form that is appropriate for the models <NUM>, outputting data objects (e.g., classifications, etc.) via the GPU module <NUM> that are suitable for handling by the CPU and the logic parameters <NUM>, and the like. Moreover, using the cache memory <NUM> to share data inputs and outputs between the different threads of the CPU and GPU enables rapid data transfer between the various stages of processing.

In addition to performance efficiencies provided by the computing architecture <NUM> described above, which enables edge nodes in accordance with the present disclosure to achieve low-latency control and messaging workflows, the computing architecture <NUM> of the present disclosure also leverages additional techniques to reduce latency and improve the flow and processing of data. For example, prioritization techniques may be utilized to allocate computing resources of the edge node <NUM> to workflow and processes in a manner that ensures sufficient computing resources (e.g., the CPU, GPU, cache memory, etc.) are allocated to critical workflows and capabilities so that those processes are not starved for computing resources by non-critical workflows and capabilities. To illustrate, the priority levels may include <NUM> priority levels, such as high, medium, and low. The high priority level may be associated with critical (e.g., in terms of latency or information) workflows and capabilities, such as data ingestion and model object detection and classification. The low priority level may be associated with workflows and capabilities that do not require or mandate real-time "ultra-low latency" operation, and the medium priority level may be associated with workflows and capabilities being used to process important workflows that do not require a lot of processing time (e.g., important micro tasks) and/or do not retrain or hold control of computing resources for a relatively long time (e.g., seconds, minutes, etc.).

As an example of applying the different priority levels described above, the high priority level may be utilized for workflows and capabilities involving ingestion and conditioning of data for analysis by the models and evaluating the conditioned data using the models, as well as allocation of resources in the cache memory for storing data generated and/or used by those processes. The medium priority level may be applied to workflows and capabilities associated with the control logic <NUM>, which may provide time sensitive functionality, such as determining whether to enable or disable devices (e.g., machinery, equipment, etc.) or other control functionality based on analysis of classifications output by the models <NUM>. It is noted that while the ability to control devices based on analysis of the control logic <NUM> may be time sensitive in certain ways, such as turning off a saw or drill press if requirements for worker safety are not met, as may be determined by the control logic <NUM>, using the medium priority for such tasks may be sufficient since evaluating the classifications output by the models may be performed quickly relative to the computational requirements and time requirements for ingesting, pre-processing, and analyzing the data streams using the models. Since the classifications resulting from the latter are inputs to the control logic <NUM>, applying the higher priority level to the data ingestion and modelling processes ensures that the information relied on by the (medium priority) processes of the control logic <NUM> is up-to-date or real-time data. Furthermore, when the control logic <NUM> makes a decision, such as to enable a piece of equipment or machinery when a worker is wearing all safety gear or to disable the piece of equipment when the worker is not wearing all required safety gear, it is not critical that the control logic <NUM> make additional decisions in real-time and a few ms (e.g., <NUM>-<NUM>) may be sufficient to ensure that the control signals are provided to enable/disable the piece of equipment (e.g., because the worker is not likely to be able to remove a piece of safety equipment in such a small time frame). The low priority level may be applied to non-critical tasks, such as storing the control data and/or analysis outcomes in a database.

It is also noted that while in the description above high priority levels are allocated to functionality of the capture service <NUM>, the cache memory <NUM>, and the GPU module <NUM>, medium priority levels are associated with the functionality of the control logic <NUM>, and low priority levels are associated with the message broker service <NUM> and the database(s) <NUM>, such priority level assignments have been provided by way of illustration, rather than by way of limitation. For example, certain input data streams and processing, as well as the models that analyze those data streams, may be assigned medium or low priority levels while other input data streams, processing, and associated models may be assigned the high priority level (e.g., worker safety models and associated processes may be the high priority level while models for evaluating performance of equipment may be assigned the medium or low priority level). Similarly, certain functionality provided by the control logic <NUM> and the message broker service <NUM> may be assigned the high priority level while other functionality of the control logic <NUM> and the message broker service <NUM> may be assigned low or medium priority levels (e.g., control logic for determining whether equipment should be enabled/disabled, as well as transmission of control signals to enable/disable the equipment may be assigned high or medium priority levels while other types of functionality by the control logic <NUM> and the message broker service <NUM> may be assigned low or medium priority levels).

It should be understood that the application and assignment of priority levels described above has been provided for purposes of illustration, rather than by way of limitation and that other combinations and configurations of the priority level assignments to the functionality of the edge node may be utilized. Moreover, it is noted that the priority levels may be assigned dynamically (i.e., change over time) depending on the state of the monitored environment. For example, in a worker safety use case involving machinery or equipment, models and control logic used to detect whether a worker is wearing required safety equipment may be assigned low or medium priority when a worker is not detected in the vicinity of the machinery or equipment, but may be assigned a higher priority level (e.g., high or medium) after a worker is detected in the vicinity of the machinery or equipment. Other functionality and processes of the computing architecture may similarly be assigned dynamic priority levels according to the particular use case and state of the environment or other target of the monitoring by the sensor devices, etc..

The various features described above enable the computing architecture <NUM> to compute, store, and share data in a rapid fashion. For example, the computing architecture <NUM> can complete a cycle of analysis (e.g., receive and process input data via the capture service <NUM>, analyze the input data via the GPU module <NUM>, evaluate the model outputs via the control logic <NUM>, and transmit a message via the message broker service <NUM> that is received by the target device) within the above-described threshold period of time.

Referring back to <FIG>, the edge node <NUM> analyzes information received from the sensor devices 150A-150C and issues commands to the device <NUM> based on the analysis, as described above with reference to <FIG>. As briefly described above, the computing device <NUM> includes an analytics engine <NUM> and a monitoring engine <NUM>. The analytics engine <NUM> may be configured to track various metrics associated with the environment where the device <NUM> is operating, such as to track the number of safety events that have occurred (e.g., the number of times an unsafe event is detected by functionality of the edge node <NUM>), a status of the various sensor devices 150A-150C, an amount of time elapsed since a last safety event, or other types of metrics associated with the monitored environment. The monitoring engine <NUM> may be configured to monitor the messages transmitted by the message broker service of the edge node <NUM> (e.g., the message broker service <NUM> of <FIG>) for certain types of events (e.g., unsafe operating conditions, etc.). When an event monitored by the monitoring engine <NUM> occurs, a user may be notified, such as a user operating the computing device <NUM>.

A user may monitor the environment where the device <NUM> is being operated via a graphical user interface provided by the computing device <NUM>. For example, the graphical user interface may be configured to present information associated with monitored devices and environments. The user may select one of the devices or environments and the graphical user interface may display information associated with a current status of the selected device(s) and environment. Additionally, the graphical user interface may also display information associated with a history of the device <NUM> or monitored environment. For example, the history information may include information associated with historical events within the environment or associated with the device <NUM>. The user can select events to view detailed information about the event, such as to view a clip of video content associated with the event, a time of the event, or other types of information. In some aspects, the graphical user interface may also provide functionality for recording notes associated with an event, such as to record whether an injury occurred, whether a cause of the event was resolved, or other types of information. In an aspect, the graphical user interface may present data from different data sources simultaneously. For example, a portion of the presented data may be obtained from the database(s) <NUM> of the edge node <NUM> (e.g., the database <NUM> of <FIG>) and another portion of the presented data may be stored in the database(s) <NUM>. The portions of the data presented from the database(s) <NUM> may correspond to more recent information while the portions of the data presented from the database(s) <NUM> may correspond to longer-term or older data.

As briefly described above, the edge services <NUM> may include a message broker service (e.g., the message broker service <NUM>) that is configured to provide commands to devices, such as the device <NUM>, based on analysis of input data provided by the sensor devices 150A-150C. The commands may include commands to change a mode of operation of the device <NUM>, such as to slow down an operating speed of the device <NUM>, increase the operating speed of the device <NUM>, stop or turn off the device <NUM>, or turn on the device <NUM>. The commands may additionally or alternatively include other types of commands, such as commands configured to play an alarm or audible alert to notify an operator of the device <NUM> of a particular environmental condition (e.g., the worker is not wearing gloves, a hardhat, eye protection, etc.), display an alert on a computing device (e.g., the computing device <NUM>), or other types of commands.

Referring to <FIG>, a flow diagram illustrating an exemplary method for performing low latency analysis of a monitored environment using edge computing in accordance with aspects of the present disclosure is shown as a method <NUM>. In an aspect, the method <NUM> may be performed by an edge computing device, such as edge node <NUM> of <FIG> having a computing architecture similar to computing architecture <NUM> of <FIG>. In some aspects, steps of the method <NUM> may be stored as instructions that, when executed by a plurality of processors (e.g., CPUs and GPUs of an edge node), cause the plurality of processors to perform the steps of the method <NUM> to provide for low latency analysis of a monitored environment using edge computing and machine learning techniques in accordance with the concepts disclosed herein.

At block <NUM>, the method <NUM> includes receiving, via a capture service executable by a first processor, input data from one or more data sources. As described above with reference to <FIG>, the input data may include information associated with a monitored environment, one or more monitored devices, or both. For example, the input data may include video stream data associated with one or more video streams captured by cameras disposed within the monitored environment. The cameras may provide different viewing angles of the monitored environment, which may include providing different viewing angles of the one or more monitored devices within the monitored environment or views of different monitored devices within the monitored environment. In some implementations, the input data may not include information associated with monitored devices and may depict other types of information associated with a monitored environment, such as: whether individuals present within the monitored environment are social distancing or wearing masks; real time asset tracking; in line quality inspection (e.g., monitoring manufacturing processes or other product processes to verify production quality); monitoring warehouse stock levels (e.g., monitoring on-hand quantities of products in real-time using computer vision or other techniques); real-time authentication and authorization (e.g., access control and managing allowed/not allowed zones); advanced preventive maintenance (e.g., monitoring component performance and use detecting or predicting when maintenance should be performed); real-time asset protection; and worker productivity tracking. It is noted that the concepts disclosed herein may also be readily adapted to many other types of use cases and scenarios which may or may not involve monitoring individuals or devices within an environment. Accordingly, it is to be appreciated that the exemplary use cases disclosed herein have been provided for purposes of illustration, rather than by way of limitation and that embodiments of the present disclosure may be utilized for other types of real-time or near-real-time monitoring.

At step <NUM>, the method <NUM> includes applying, by a modelling engine executable by a second processor, one or more machine learning models to at least a portion of the input data to produce model output data. In an aspect, the modelling engine may be the modelling engine <NUM> of <FIG> or the GPU module <NUM> of <FIG> and the second processor may be a GPU, as described above. The one or more machine learning models may include computer vision modules configured to evaluate video stream data, such as the video frame data <NUM>', <NUM>' of <FIG>. As described above, the portion of the input data to which the one or more machine learning models may be applied may include information extracted or derived from the input data, such as by converting frames of video data into a data structure that represents the video frame content as an array or matrix of numeric values (e.g., values derived from grey scale levels, luminance, etc.). Additionally, the video frame data may be subjected to other processing prior to generating the data structure, such as normalization, down-sampling, scaling, or other processes.

At step <NUM>, the method <NUM> includes executing, by control logic executable by the first processor, logic parameters against the model output data to produce control data. In an aspect, the logic parameters (e.g., the logic parameters <NUM> of the control logic <NUM> of <FIG>) may be configured to produce control data and outcome data, as described above. The control data may include information associated with operation of a remote device or another type of action item while the outcome data may include information associated with a state of the monitored environment, the monitored device, or both. For example, the state data may indicate a remote device (e.g., the device <NUM> of <FIG>) is being operated in a safe or unsafe manner.

At step <NUM>, the method <NUM> includes generating, via a message broker service executable by the first processor, at least one control message based on the control data and at step <NUM>, the method <NUM> includes transmitting, by the message broker service, the at least one control message to the remote device. In an aspect, the message broker service may be one of the edge services <NUM> of <FIG>, such as the message broker service <NUM> of <FIG>. As described above, the control message include one or more commands corresponding to a remote device (e.g., the device <NUM> of <FIG>). For example, the one or more commands may include commands to change a mode of operation of the remote device, such as to slow down an operating speed of the remote device, increase the operating speed of the remote device, stop or turn off the remote device, turn on the remote device, play an alarm or audible alert to notify an operator of the device <NUM> of a particular environmental condition (e.g., the worker is not wearing gloves, a hardhat, eye protection, etc.), or other types of commands.

As described above, the method <NUM> enables computer vision techniques to be leveraged from edge computing nodes, such as edge node <NUM> of <FIG>, while providing low latency and high performance. In some aspects, the method <NUM> enables processing cycles (e.g., a cycle includes receiving input data at step <NUM> through receiving (at the target device) the at least one control message generated at step <NUM> based on the input data) to be completed in under <NUM>, and in many use cases, between <NUM> and <NUM>. Such rapid computing and processing capabilities are orders of magnitude faster than presently available systems and techniques, which can take over <NUM> per cycle and more typically require almost <NUM>. Moreover, it is noted that the rapid and low latency capabilities of the method <NUM> are provided, at least in part, by the edge computing architecture of embodiments, as described and illustrated with reference to <FIG>.

Moreover, it is to be understood that method <NUM> and the concepts described and illustrated with reference to <FIG> and <FIG>, including utilization of priority levels, may be utilized to provide ultra-low latency and high performance analytics and analysis techniques that leverage <NUM> or other next generation network and edge architectures. The edge architectures may leverage devices or nodes with having limited computing resources as compared to traditional client-server systems or computing architectures and yet may achieve rapid and accurate analysis of input data streams as described above. Thus, embodiments of the present disclosure should be recognized as providing a framework and computing architecture for designing devices and systems that, despite having limited resources, are capable of performing "or solving for" real-time and mission critical use cases.

Table <NUM>, below, highlights exemplary use cases and examples of the applications and capabilities that may be realized using the computing architectures and functionality disclosed herein. It is noted that the exemplary use cases shown in Table <NUM> are provided for purposes of illustration, rather than by way of limitation and that the computing architecture and processes described herein may be applied to other use cases where edge devices and computer vision or other modelling techniques and low latency processing are advantageous.

In the non-limiting and exemplary use cases shown above in Table <NUM>, sensors and devices may be deployed in various types of environments to capture data that may be provided to one or more edge nodes, such as the edge node(s) <NUM> of <FIG> for analysis. The sensors and devices may include cameras (e.g., imaging camera, video cameras, infrared cameras, RGB-D cameras, etc.), temperature sensors, pressure sensors, global positioning system (GPS) devices, radio frequency identification (RFID) devices and sensors, radar sensors, proximity sensors, motion sensors, or types of sensors and devices (e.g., IoT devices). It is noted that the particular sensors and devices utilized to collect the data that is provided to the edge node(s) for analysis in accordance with the concepts disclosed herein may be different for different use cases. For example, in many of the use cases shown in Table <NUM> computer vision techniques may be utilized, but the image and/or video data, as well as the types of cameras utilized in those use cases may differ (e.g., a PPE monitoring use case may utilize video camera data, a predictive maintenance and remote diagnostics use case may utilize video camera data as well as infrared camera data, and a space utilization use case may utilize a still image camera (i.e., non-video data), video camera data, and RGB-D camera data). Other differences between various types of sensor devices and combinations of sensor devices that may be used for different use cases are also contemplated. Illustrative aspects of some of the above-identified use cases are described in more detail below.

Referring to <FIG>, a block diagram illustrating an exemplary system for monitoring an environment using edge node computing architectures in accordance with aspects of the present disclosure is shown as a system <NUM>. The system <NUM> may be designed to monitor safety in an environment where workers interact with various types of machinery <NUM> (e.g., drill presses, saws, welding tools, or other types of equipment). As shown in <FIG>, the system <NUM> includes the edge node <NUM>, computing device <NUM>, and computing device <NUM> of <FIG>. The system <NUM> also includes sensor devices <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, which may include cameras, proximity sensors, temperature sensors, or other types of sensors and devices. For example, the cameras may be disposed at various locations within the environment where the machinery <NUM> is located and each camera may have a field of view that includes the machinery <NUM>. The cameras may be communicatively coupled to the edge node <NUM> via an edge network communication link (e.g., a <NUM> communication link) and transmit data to the edge node <NUM> for analysis. The data transmitted by the cameras may include video data (e.g., a video stream), still image data (e.g., images captured by the cameras every "X" units of time, such as every <NUM> milliseconds (ms), <NUM>-<NUM>, <NUM>-<NUM>, <NUM> second (s), every <NUM>, every minute, or some other frequency), or both. In an aspect, the edge node <NUM> may transmit control signals to the cameras (or a device coupled to the cameras) to control the frequency with which the cameras provide data to the edge node <NUM> and/or to control whether the cameras provide data is still image data or video data. For example, during periods of time when no workers are present (e.g., as may be determined by the edge node <NUM> based on data from the cameras using the computer vision techniques described above), the edge node <NUM> may instruct, via the control signal, the cameras to transmit still image data periodically, such as once per minute. The edge node <NUM> may analyze the data provided by the cameras and upon detecting the presence of a worker within the environment where the machinery <NUM> is located, may provide a control signal to the cameras to switch to providing video data or providing still image data at a higher frequency (e.g., once per <NUM>).

As described above with reference to <FIG>, the edge node <NUM> may include one or more machine learning models providing functionality for analyzing data provided by the cameras. In the exemplary worker safety use case illustrated in <FIG>, the models may include one or more models configured to detect the presence of safety equipment for workers. For example, the model(s) may be trained to detect whether a worker is wearing eye protection, ear protection, gloves, a hardhat, a mask, or other safety equipment. Using these models, the edge node <NUM> may determine, based on analysis of the data provided by the cameras or other sensor devices, whether workers present in the environment where the machinery <NUM> is located are wearing all safety equipment required in order to use the machinery <NUM>. The edge node <NUM> may transmit information to the controller <NUM> to control the operational state (e.g., enable, disable, slow down, etc.) of the machinery <NUM> based on whether one or more workers detected in the environment where the machinery <NUM> is located are wearing or are not wearing appropriate safety equipment (e.g., missing gloves, missing eye protection, and the like), as described in more detail below.

To further illustrate the concepts of the system <NUM> described above, the edge node <NUM> may utilize a computing architecture in accordance with the concepts disclosed herein, such as the computing architecture <NUM> of <FIG>. In such an embodiment, sensor data (e.g., media content and other data) captured by the sensor devices <NUM>-<NUM> may be processed using a capture service (e.g., the capture service <NUM>) executable by a CPU. Processing the sensor data may include various operations to prepare the sensor data for analysis by the model(s) of the edge node <NUM>. For example, media content (e.g., frames of video data or image data) may be converted by the capture service into an array or matrix of numeric values representing the pixels representative of the media content (e.g., a numeric value representing to color or gray scale level of the pixels, luminance, and the like), normalization, down-sampling, scaling, or other processes that enable the media content to be converted to a form that may be input to the model(s) of the edge node <NUM>. The processed sensor data may then be stored in a cache memory that is shared between the CPU and a GPU, which enables the processed sensor data to be retrieved for processing by the model(s) of the edge node.

The model(s) may be used to evaluate the retrieved sensor data via a GPU module of the edge node <NUM> (e.g., the GPU module <NUM> of <FIG>) and the model(s) may output classifications (e.g., the classifications <NUM>) based on evaluation of the cached media content. In the example worker safety scenario described above, the classifications may include classifications indicating whether or not the worker is detected, as well as classifications indicating whether one or more pieces of protective equipment (e.g., eye protection, ear protection, gloves, a hardhat, a mask, and the like) are or are not being worn by the worker. The classifications output by the model(s) may be stored in a cache memory and subsequently retrieved for analysis by control logic <NUM>, which may be similar to the control logic <NUM> of <FIG>. During analysis of the classifications, shown as "{A} {B} {C}" in <FIG>, the control logic <NUM> may produce control data and analysis outcomes. In <FIG>, the control data is shown as "{A1}{B1}{C1}" and the analysis outcomes are shown as "{A2}{B2}{C2}. " The control data may be generated and stored in the cache memory as control logic outputs (e.g., the control logic outputs <NUM> of <FIG>). It is noted that the classifications may include multiple sequences of data, such as classifications derived from multiple time-sequenced pieces of sensor data, and the control logic <NUM> may be configured to output control data and/or analysis outcomes based on the sequences of classification data.

The control data and analysis outcomes may be stored in the cache memory for subsequent processing by a message broker service (e.g., the message broker service <NUM> of <FIG>) and/or storage in a database (e.g., the database <NUM> of <FIG>) or a remote database (e.g., a database of the computing device <NUM> and/or the computing <NUM>). The message broker service of the edge node <NUM> may be configured to generate one or more messages for transmission to the computing device <NUM> and/or the computing device <NUM>, such as messages <NUM>, <NUM>, respectively. The message <NUM>, <NUM> may provide information to users monitoring the environment, such as alerts to indicate workers are or are not wearing required protective equipment. In an aspect, the messages may be presented to the user in a textual format, such as to display a message indicating safety equipment is or is not being worn by one or more workers. The displayed message may only be changed when a change in the status of the safety equipment changes or the presence of workers changes. For example, as long as all detected workers are wearing appropriate safety equipment the message may indicate all worker complying with safety equipment requirements. If all workers leave the environment (or move a certain distance from the machinery <NUM>), the message may be updated to indicate no workers in vicinity of the machinery <NUM>. Similarly, the message may be updated to indicate when at least one worker in the vicinity of the machinery <NUM> is not wearing required safety equipment. In some aspects, a visible or audible alert may also be provided in certain circumstances, such as when a worker is detected in the vicinity of the machinery <NUM> that is not wearing all required safety equipment. In additional or alternative aspects, the information included in the messages <NUM>, <NUM> may not be presented to the user in a textual format and may instead be presented as a color graphic or other visual indicator that may be displayed on a user interface (e.g., green when workers in the monitored environment are wearing all protective equipment or red when workers are not wearing one or more pieces of protective equipment). Other types of information may also be provided to the users via the messages <NUM>, <NUM>, such as a state of the machinery <NUM> (e.g., whether the machinery <NUM> is being operated, is disabled, etc.). It is noted that some of the information presented to the user via the graphical user interface may be provided based on information stored in a database local to the edge node <NUM>, such as the database <NUM> of <FIG>, as described above.

Additionally, the messages <NUM>, <NUM> may also be used to store information at a remote database, such as to store information regarding the analysis outcomes (e.g., "{A2 B2 C2}") and/or the sensor data (e.g., A1-An, B1-Bn, C1-Cn, etc., or portions thereof) at a remote database (e.g., a database maintained at the computing device <NUM> or the computing device <NUM>). In some aspects, the sensor data may only be stored in the local and/or remote database when certain events occur, such as a state change with respect to the worker's safety equipment (e.g., one or more pieces of media content upon which a determination was made that the worker(s) is or is not wearing required safety equipment, a worker has been detected in the vicinity of the machinery <NUM>, etc.). In this manner the volume of data stored at the remote or local database(s) may be minimized while retaining a record of the state of certain key features being monitored within an environment. Similarly, control data may also be stored in the database(s) based on key events, such as when the machinery <NUM> is enabled, disabled, slowed, etc. based on the state of workers and their safety equipment. The records stored at the database(s) may be timestamped to enable time sequencing of the data, such as to enable a piece of media content to be associated with a control signal transmitted to the controller <NUM>, which may enable a user of the computing device <NUM> or the computing device <NUM> to review the control signals and associated media content from which the control signals were generated at a later time, such as during a safety or system audit.

In addition to the messages <NUM>, <NUM>, the message broker of the edge node <NUM> may also provide control signals <NUM> to the controller <NUM> to control the operational state (e.g., enable, disable, slow down, etc.) of the machinery <NUM> based on the analysis by the control logic <NUM>. For example, in the above-example involving a drill press, the edge node <NUM> may provide the control signals <NUM> to the controller <NUM> to control operations the drill press. The control signals may be generated based on application of the logic parameters <NUM> of the control logic <NUM> to the classifications output by the model(s). The logic parameters <NUM> may be configured to determine whether the drill press is being operated in a safe or unsafe manner based on the outputs of the model(s), and the control signals <NUM> may include commands to slow or stop the drill press, a command to generate an auditory alert to the drill press operator, or other types of operations to address any unsafe operating conditions detected by the control logic <NUM>. For example, logic parameters <NUM> are shown in <FIG> as including a plurality of logic parameters L<NUM>-Lz. Certain ones of the logic parameters <NUM> may be used to evaluate whether the worker is or is not wearing safety equipment (e.g., helmet, eye protection, ear protection, etc.) based on the classifications output by a first model (e.g., classifications {A}), other ones of the logic parameters <NUM> may be used to evaluate whether the worker is or is not wearing other pieces of safety equipment (e.g., gloves) based on the classifications output by a second model (e.g., classifications {B}), and other logic parameters may be configured to evaluate other aspects of the monitored environment (e.g., whether workers present in the environment, whether a worker is close to certain components of the machinery <NUM>, etc.) based on classifications by another model (e.g., classifications {C}).

For example, a first set of the logic parameters <NUM> may be used to determine whether workers are present in the environment and that required pieces of safety equipment are being worn and a second set of the logic parameters <NUM> may then determine whether to generate control signals based on the outputs of the evaluation by the certain logic parameters. Exemplary pseudocode illustrating aspects the first and second set of logic parameters described above is shown below:
<IMG>.

In the exemplary pseudocode above, worker_present() represents a logic parameter that uses classifications {C} as an input to determine whether a worker is present in the monitored environment; gloves_on() represents a logic parameter that uses classifications {B} as an input to determine whether gloves are being worn; and eye_protection_on(), ear_protection_on(), helmet_on() represent logic parameters that use classifications {A} as an input to determine whether eye protection, ear protection, and helmets are being worn. As can be appreciated from the pseudocode above, if no worker is present in the monitored environment (e.g., "if worker_present({C})" evaluates to no) the "else" statement is executed, which sets the "control_signal" variable to "disable" and outputs the "control_signal" variable (e.g., a control signal <NUM> is transmitted to controller <NUM> to disable the machinery <NUM>). If a worker is present in the monitored environment (e.g., "if worker_present({C})" evaluates to yes), the nested "if' statements are executed to confirm that required safety equipment is being worn by the worker(s). If gloves_on(), eye_protection_on(), ear_protection_on(), or helmet on() evaluates to "no", the "else" statement may be executed as described above. However, if gloves _on(), eye_protection_on(), ear_protection_on(), or helmet_on() evaluates to "yes" (i.e., all required safety equipment is being worn by the worker(s)), the "control signal" variable is set to "enable" and output (e.g., a control signal <NUM> is transmitted to controller <NUM> to enable the machinery <NUM>). In this manner, if a worker is not present or any piece of safety equipment is missing, a control signal <NUM> will be sent to the machinery <NUM> to disable operation of the machinery <NUM>, and the machinery will only be enabled if a worker is present and all required safety equipment is detected.

To reduce the number of control signals transmitted by the edge node <NUM>, the pseudocode could be modified to maintain state information and only send the control signal if the state of the machinery <NUM> is changing. For example:
if worker_present({C}) = yes {
if gloves_on({B}) = yes {
if eye_protection_on({A}) = yes {
if ear_protection_on({A}) = yes {
if helmet_on({A}) = yes {
if state = disabled {
control_signal = enable;
state = enabled;
output (control_signal)} } } } }
}
else {
if state = enabled {
control_signal = disable;
state = disabled;
output (control_signal)}}.

Using the modified pseudocode above, which maintains state information, the state of the machinery <NUM> is checked and the control signals are only sent when there is a state change. For example, if a worker is present and all required safety equipment is being worn then the machinery <NUM> should be in the enabled state. The "if state = disabled" checks to see if the current state of the machinery <NUM> is disabled, and if disabled (e.g., "if state = disabled" is true), the state is set to enabled, the control_signal variable is set to enable, and the control_signal is output. Similarly, if a worker is not present or all required safety equipment is being worn, the machinery <NUM> should be in the disabled state. In the "else" clause, the state is first checked to see if the machinery <NUM> is already in the enabled state, and if enabled, the control_signal variable is set to disable, the state variable is set to disabled, and the control_signal is transmitted to the controller <NUM>. In this manner, the number of control signals transmitted by the edge node <NUM> may be reduced. It is noted that the exemplary pseudocode described above has been provided for purposes of illustration, rather than by way of limitation and that other techniques may be used to evaluate the classifications and generate control signals in accordance with the concepts disclosed herein. It is noted that the control signals <NUM> may be transmitted to the controller <NUM> by a message broker service of the edge node <NUM> via an edge communication link, as described above.

It is noted that while control logic <NUM> is shown in <FIG> as analyzing or evaluating <NUM> different types of classifications (e.g., classifications {A} {B} {C}), control logic <NUM> may be configured to analyze or evaluate less than <NUM> types of classifications or more than <NUM> different types of classifications if desired depending on the particular use case involved and the configuration of the control logic and/or models of the system <NUM>. Furthermore, it is noted that the control logic <NUM> may include logic parameters <NUM> that evaluate information other than classifications. For example, in addition to monitoring the environment to ensure that workers are only able to operate the machinery <NUM> when wearing required safety equipment, the edge node <NUM> may provide other types of functionality for monitoring worker safety. For example, suppose that the worker is operating the machinery <NUM> for a period of time and takes off his/her helmet while operating the machinery <NUM>. By analyzing the data provided by the cameras (e.g., one or more of the sensor devices <NUM>-<NUM>) over time using the processing described above, the edge node <NUM> may detect that the worker has taken the helmet off and may transmit a control signal <NUM> to the controller <NUM> to turn off the machinery <NUM>. As described above, due to the computing architecture disclosed herein, the determination that the worker took the helmet off may be made in fractions of a second, thereby ensuring that the control signal <NUM> to turn the machinery <NUM> off may occur very soon after the worker takes the helmet off, which may prevent an accident or injury while the worker's helmet is off. If the worker puts the helmet back on, the edge node <NUM> may again detect that the worker is wearing the appropriate safety equipment and provide a control signal <NUM> to the controller <NUM> that enables operation of the machinery <NUM> to controller <NUM>. As described above, the determinations to transmit the control signals <NUM> in response to the worker removing or putting back on the helmet may be generated based on analysis of the outputs of the model(s) (e.g., the classifications) by the control logic <NUM>, and multiple outputs may be generated by the control logic (e.g., the control data and analysis outcomes).

As another example, suppose that the machinery <NUM> is intended to be operated by a worker that is not wearing gloves (e.g., to provide improved interaction with certain controls of the machinery <NUM> that may be impeded when the worker is wearing gloves). Suppose that the worker is operating the machinery <NUM> and then puts on a pair of gloves to pick up an item the worker is working on (e.g., a welded item) and reposition the item for further processing using the machinery <NUM> or to start working on a new item. The edge node <NUM> may detect that the user has put on gloves and may transmit a control signal to turn the machinery <NUM> off. When the worker finishes repositioning the item or has positioned the new item appropriately, the worker may then remove the gloves. The edge node <NUM> may detect the worker has removed the gloves and provide a control signal to the controller <NUM> that places the machinery <NUM> back in the operational state, thereby allowing the worker to continue using the machinery <NUM>.

In addition to models for detecting whether the worker is wearing safety equipment, the models of the edge node <NUM> may also be configured to provide computer vision-based functionality for monitoring other aspects of worker safety. For example, the models of the edge node <NUM> may include models configured to detect whether the worker is using the machinery <NUM> in a safe manner, such as to detect whether a portion of the worker's body (e.g., hands, legs, arms, etc.) is close to one or more moving parts of the machinery <NUM> (e.g., a saw blade, a drill bit of a drill press, and the like). If the edge node <NUM> detects that the machinery <NUM> is being operated in an unsafe manner by the worker, the edge node <NUM> may provide a control signal to the controller <NUM> to turn off a particular portion of the machinery <NUM> (e.g., stop rotation or oscillation of a saw blade, etc.) or turn off the machinery <NUM> completely. In some aspects, the edge node <NUM> may provide control signals to the controller <NUM> that may be used to provide feedback to the worker regarding detection of unsafe operation of the machinery <NUM>. For example, where the machinery <NUM> is a saw, a first control signal may be transmitted from the edge node <NUM> to the controller <NUM> to change a characteristic of the rotation or oscillation of the saw blade, such as to slow down the saw blade or to pulse the saw blade (e.g., speed up and slow down the saw blade multiple times). The changing of the characteristic of the rotation or oscillation of the saw blade may inform the worker of an unsafe operating condition, such as to indicate that the worker's hand(s) are approaching a position considered too close to the blade (e.g., once the worker's hand(s) reach the position deemed too close to the blade the saw may be turned off) or that another worker is present in the environment in the vicinity of the machinery <NUM>.

As an additional example, the models of the edge node <NUM> may include a model configured to detect movement of workers in the environment where the machinery is located, and the control logic <NUM> may be configured to selectively turn off the machinery <NUM> based on detection of the worker. For example, in <FIG>, a region <NUM> surrounding the machinery <NUM> is shown. The region <NUM> may be an area surrounding the machinery <NUM> that corresponds to a space where workers are typically located when using the machinery <NUM>. The model(s) of the edge node <NUM> may be configured to determine whether a worker is present in the region <NUM> or not and provide control signals to the controller <NUM> to enable/disable the machinery <NUM> based on the presence of the worker in the region <NUM>. For example, suppose that a worker wearing all required PPE is present in the region <NUM> and using the machinery <NUM>, but then leaves the region <NUM> (e.g., to obtain additional material(s) for use with the machinery <NUM> or some other reason). The model may classify video data received from one or more of the sensor devices as indicating the worker has left the region <NUM> and the control logic of the edge node <NUM> may determine to send a control signal to the controller <NUM> to turn off the machinery <NUM> based on detecting the worker is not present at the machinery <NUM> (e.g., not within the region <NUM>). If the worker subsequently returns to the region <NUM>, the video data received from one or more of the sensors may be classified by the model(s) as indicating the worker is present in the region <NUM> and the control logic may determine to send a control signal to the controller <NUM> to enable the machinery <NUM> to be turned on again. It is noted that the models of the edge node <NUM> may worker in a coordinated manner, rather than in isolation. To illustrate, in the example above the models of the edge node <NUM> may also determine whether the worker is wearing all of the PPE required for using the machinery <NUM>. If the worker is wearing the PPE, the control logic may determine that the worker is present in the region <NUM> and is wearing the required PPE and provide the control signal to enable the machinery <NUM>, but if the worker is not present in the region <NUM> or is not wearing all required PPE upon returning to the region <NUM>, the control logic may not enable operation of the machinery <NUM>.

In addition to monitoring worker safety, the exemplary configuration of the system <NUM> of <FIG> may also be utilized to facilitate other types of use cases from Table <NUM> above. For example, the system <NUM> may also include capabilities to perform operations supporting predictive maintenance and/or remote diagnostics. In particular, the sensor devices <NUM>-<NUM> may include acoustic sensors, temperature sensors, pressure sensors, or other types of sensors that may be used to monitor performance of the machinery <NUM>. Information received from such sensors may be provided to one or more models of the edge node <NUM> for analysis, such as to determine if sounds picked up by the acoustic sensors indicate potential problems with bearings of the machinery <NUM> (e.g., a squeaking noise is detected), detecting overheating conditions based on temperature data received from temperature sensors, or other types of abnormalities that may be detected by models of the edge node <NUM>. The control logic <NUM> may be configured to provide control signals <NUM> to the controller <NUM> to turn off the machinery <NUM> and/or to a user of the computing devices <NUM>, <NUM> when a potential issue related to performance and/or operation of the machinery <NUM> is detected (e.g., to prevent further damage or failure of the machinery <NUM> or potential injury resulting from the damage or failure, as well as notify maintenance personnel associated with one of the computing devices <NUM>, <NUM>).

Additionally, the control logic <NUM> may provide a notification to the computing device <NUM> and/or the computing device <NUM> indicating the detection of a problem condition with respect to operation of the machinery <NUM>. For example, the computing device <NUM> may be associated with maintenance personnel and the notification may indicate that a potential problem has been detected with respect to the machinery <NUM>. The notification may include information associated with a predicted problem with the machinery <NUM>, which may be predicted based on a classification of the sensor data by the one or more models. The maintenance personnel may subsequently inspect the machinery <NUM> to confirm the existence of a problem with the machinery <NUM> and make any necessary repairs. As described above, information associated with the analysis performed by the edge node <NUM> may also be stored in a database and presented to a user via a graphical user interface, such as a graphical user interface presented at a display device associated with the computing device <NUM> and/or a display device associated with the computing device <NUM>. Presenting the information at the graphical user interface may facilitate real-time monitoring of the environment where the machinery <NUM> is located. The graphical user interface may also enable the user to view historic information associated with the environment where the machinery <NUM> is located, as described above.

In addition to utilizing the computing architectures disclosed herein to achieve low-latency control and messaging, the edge node <NUM> may utilize additional techniques to improve the flow and processing of data, which may further improve the low latency capabilities of the edge node <NUM>. For example, prioritization techniques may be utilized to prioritize memory cache streams and control priority of computing and processing resources of the edge node <NUM>. As explained above, the edge node <NUM> may provide functionality to support different workflows and capabilities, such as processes to condition sensor data for ingestion by the model(s), evaluating the conditioned sensor data by the model(s), evaluation of classifications generated by the model(s) by the control logic, and transmission of control signals and messages. The prioritization techniques may include multiple priority levels for different processing and data streams. For example, the priority levels may include <NUM> priority levels: high, medium, and low. High priority levels may be associated with critical (e.g., in terms of latency or information) workflows and capabilities, such as data ingestion and model object detection and classification. Medium priority levels may be associated with streams currently being used to process important workflows that do not require a lot of processing time (e.g., important micro tasks) and/or do not get a hold of the resource for a long time, such as applying control logic <NUM> to classification data to extract meaningful outcomes. Low priority levels may be associated with processes that do not require or mandate a real-time "ultra-low latency" action or processing. As a non-limiting example, the <NUM> priority levels may be applied in the above-described use case as follows: low priority may be assigned to processes and streams used to store data to a local and/or remote database, serve data to dashboards (e.g., provide data to GUIs or other devices via APIs, data syncs, etc.), or other tasks (e.g., workflows and processes related to analysis of sensor data related to performance of the machinery <NUM>, which may be useful but lower priority than worker safety processes); medium priority may be assigned to processes for evaluating classification data for detection of worker safety issues and conditioning and modelling processes for processing ; and high Priority may be used for processing ingesting sensor data, pre-processing the sensor data for analysis by the models, and evaluating the processed or conditioned data by the models. As explained above with reference to <FIG>, other assignments of the priority levels and/or dynamic assignment of the priority levels may be utilized if desired.

As shown above, systems incorporating edge nodes configured in accordance with the computing architectures and techniques disclosed herein enable monitoring of environments via analysis of data streams provided by various sensors using one or more machine learning models. The machine learning models may characterize or classify events occurring within the monitored environment based on the information included in the data streams and control logic may evaluate the events occurring within the environment based on the outputs of the machine learning models to provide feedback to the monitored environment (e.g., to control operations of machinery or other devices in the monitored environment) and/or users associated with the monitored environment (e.g., workers within the environment, maintenance personnel, a supervisor, and the like). Due to the utilization of edge nodes for analysis of the data streams and the computing architectures of the present disclosure, the feedback (e.g., the messages <NUM>, <NUM> and the control signals <NUM>) may be provided in real-time or near-real-time (if desired), which may prevent injury to individuals within the environment (e.g., in a worker safety use case) and/or mitigate a likelihood of damage or failure of machinery and equipment within the environment (e.g., in a predictive maintenance and/or remote diagnostics use case).

It is noted that while <FIG> shows the system <NUM> as including a single edge node <NUM> and one piece of machinery <NUM>, the system <NUM> may be readily implemented with more than one edge node <NUM> and more than one piece of machinery <NUM>. Moreover, it should be understood that the system <NUM> may also be implemented with additional sensors and/or types of sensors than those described above and that the edge nodes <NUM> may be configured with other types of models and control logic suitable for a desired set of monitoring and control operations. Accordingly, it is to be understood that the exemplary details regarding the system <NUM> described above have been provided for purposes of illustration, rather than by way of limitation and that the system <NUM> may include less devices (e.g., less sensors, computing devices, etc.), more devices (e.g., more sensors, computing devices, etc.), different devices, and/or be used to support other use cases and operations depending on the particular needs of the environment being monitored and the use cases involved.

Referring to <FIG>, a block diagram illustrating another exemplary system for monitoring an environment using edge node computing architectures in accordance with aspects of the present disclosure is shown as a system <NUM>. The system <NUM> may be designed to monitor an environment in which various manufacturing processes take place or other environments where items are moved (e.g., warehouse facilities, packaging facilities, and the like). As shown in <FIG>, the system <NUM> includes the edge node <NUM>, computing device <NUM>, and computing device <NUM> of <FIG>. The system <NUM> also includes sensor devices <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, which may include cameras, proximity sensors, temperature sensors, motion sensors, or other types of sensors and devices.

The sensors <NUM>-<NUM> may be configured to monitor various portions of a production infrastructure <NUM>. The production infrastructure <NUM> may include components or machinery to facilitate movement of items or products <NUM> in the direction shown by arrows <NUM>, <NUM> (e.g., from left to right in <FIG>), such as conveyors, rollers, robotic arms or assemblies, and the like. As the products <NUM> are moved along various stages of the production infrastructure <NUM> the sensors <NUM>-<NUM> may capture various types of data that may be provided to the edge node <NUM> for analysis using the computing architectures of the present disclosure. To illustrate, camera sensors may provide media content (e.g., video and/or image data streams) to the edge node <NUM> for analysis. The edge node <NUM> may utilize one or more models to analyze the media content and the model(s) may be trained to detect and/or identify defects or other types of issues (e.g., dents, scratches, cracks, misaligned components, and the like) as the products move through the production infrastructure <NUM>. In such an implementation, the one or more models may include different models for different types of defects or issues (e.g., one or more models for scratches, one or more models for cracks, one or more models for dents, and so on) or some of the models may be configured to detect multiple types of defects (e.g., a model configuration to detect scratches and cracks).

The model(s) of the edge node <NUM> and/or the control logic <NUM> may additionally or alternatively be configured to determine a cause of at least some of the defects identified by the edge node <NUM>. For example, the production infrastructure <NUM> may involve heating and/or cooling processes and certain types of defects may be more prevalent when the heating and/or cooling processes occur too rapidly or result in temperatures that too high or too low for current environmental conditions (e.g., ambient temperature, humidity, etc.). The sensors <NUM>-<NUM> may include devices that provide environmental data regarding the environment where the production infrastructure (or a portion thereof) is located, such as ambient temperature data, humidity data, temperature data associated with heating or cooling processes, temperature data associated with products moving through the production infrastructure, and the like. The environmental data may be analyzed by the model(s) and/or the control logic to predict causes of one or more types of defects. For example, if one or more of the models classify detected defects as cracks, another model and/or the control logic may evaluate the environmental data to determine whether a cooling process is occurring too rapidly or slowly (e.g., due to a temperature of the cooling processes being too cold or too hot or because a conveyor is moving the product(s) through the cooling process too slow or fast). When a potential cause for the cracks is determined based on the environmental data, one or more of the messages <NUM>, <NUM> may be provided to the computing devices <NUM>, <NUM> to indicate the cause of identified defects. In an aspect, one or more of the messages <NUM>, <NUM> transmitted by the edge node <NUM> may include other types of information, such as information that indicates a possible cause of the detected or predicted defects (e.g., the defect is being caused by one or more processes or functionality of the production infrastructure <NUM>, other environmental conditions, and the like). As described above with reference to <FIG> and <FIG>, the messages <NUM>, <NUM> may be transmitted by a message broker service of the edge node <NUM> (e.g., the message broker service <NUM> of <FIG>).

In some aspects, control signals <NUM> may also be sent to one or more controller devices <NUM>, which may be configured to control operations of the production infrastructure <NUM>. To illustrate, a control signal <NUM> may be sent to a controller <NUM> configured to control a cooling temperature used by a cooling process of the production infrastructure <NUM> to modify the temperature (e.g., increase or decrease the temperature) of the cooling process. Additionally or alternatively, a control signal <NUM> may be provided to a controller <NUM> configured to control a rate or speed at which products are moved through the cooling process (e.g., to speed up or slow down the cooling process). Other types of control signals <NUM> may also be provided to controllers <NUM> of the production infrastructure to minimize further occurrences of defects detected by the edge node <NUM>. In additional or alternative aspects, the messages <NUM>, <NUM> transmitted to one or more of the computing devices <NUM>, <NUM> may include recommended modifications to the operations of the production infrastructure <NUM> and the control signals <NUM> may be provided to the controller(s) <NUM> by the computing device(s) after review by a user, such as in response to inputs provided by the user to a graphical user interface (e.g., a dashboard or other application).

Using the system <NUM>, users of the computing devices <NUM>, <NUM> may monitor the production infrastructure <NUM> and receive information in real-time or near-real-time (e.g., less than <NUM>, less than <NUM>, less than <NUM>, approximately <NUM>, etc.) regarding defects or other abnormalities detected with respect to products moving through the production infrastructure <NUM>. Moreover, the functionality provided by the edge node <NUM> of the system <NUM> may enable actions to mitigate detected defects and anomalies to be implemented automatically (e.g., via control signals provided from the edge node(s) <NUM> to the controller(s) <NUM>) or recommendations regarding actions to mitigate detected defects and anomalies to be provided to the users of the computing devices <NUM>, <NUM>. Using such capabilities may enable the production infrastructure <NUM> to be controlled and operated in a more efficient manner and enable mitigation of defects or other issues to be addressed more quickly as compared to currently available production management solutions. Moreover, in some implementations the models of the edge node(s) <NUM> may be configured to predict the occurrence of defects or other production anomalies prior to the widespread occurrence of the defects based on information provided by one or more of the sensor devices <NUM>-<NUM>, which may enable mitigation actions to be implemented (automatically or at the direction of the user(s)) in a pre-emptive, rather than reactive manner.

To further illustrate the concepts of the system <NUM> described above, the edge node <NUM> may utilize a computing architecture in accordance with the concepts disclosed herein, such as the computing architecture <NUM> of <FIG>. In such an embodiment, sensor data (e.g., media content and other data) captured by the sensor devices <NUM>-<NUM> may be processed using a capture service (e.g., the capture service <NUM>) executable by a CPU. Processing the sensor data may include various operations to prepare the sensor data for analysis by the model(s) of the edge node <NUM>. For example, media content (e.g., frames of video data or image data) may be converted by the capture service into an array or matrix of numeric values representing the pixels representative of the media content (e.g., a numeric value representing to color or gray scale level of the pixels, luminance, and the like), normalization, down-sampling, scaling, or other processes that enable the media content to be converted to a form that may be input to the model(s) of the edge node <NUM>. In some aspects, temperature data or other types of non-media content data (e.g., pressure data, humidity data, etc.) received from the sensor devices <NUM>-<NUM> by the capture service may also be processed (e.g., rounded, normalized, etc.). In additional or alternative aspects, these other types of sensor data may simply be stored in the cache memory without any processing (e.g., because the sensor data may already be in format suitable for use by the models, such as numeric data). As described above, the cache memory may be shared by processes utilizing the computing resources of the CPU and a GPU, such as the capture service, a GPU module, and the control logic <NUM>, which enables the sensor data to be stored by the capture service and retrieved for processing by the GPU module more quickly.

As in the examples described above, the model(s) of the GPU module may be used to evaluate the retrieved sensor data, and one or more classifications may be output based on evaluation of the cached media content. In the example use case above where the system <NUM> is used to monitor the production infrastructure <NUM>, the classifications may include classifications indicating whether defects are or are not detected, as well as other types of classifications associated with the processes of the production infrastructure <NUM>, such as classifications associated with a speed at which products are moving through the production infrastructure <NUM>, temperature classifications (e.g., classification of temperatures of cooling or heating processes, ambient environment temperatures, and the like), or other classifications. The classifications output by the model(s) may be stored in the cache memory and may be subsequently retrieved for analysis by control logic <NUM>, which may be similar to the control logic <NUM> of <FIG> and the control logic <NUM> of <FIG>. During analysis of the classifications, shown as "{A} {B} {C}" in <FIG>, the control logic <NUM> may produce control data and analysis outcomes, shown in <FIG> as "{A1}{B1}{C1}" and "{A2}{B2}{C2}", respectively. The control data and analysis outcomes may be stored in the cache memory as control logic outputs (e.g., the control logic outputs <NUM> of <FIG>). It is noted that the classifications may include multiple sequences of data, such as classifications derived from multiple time-sequenced pieces of sensor data, and the control logic <NUM> may be configured to output control data and/or analysis outcomes based on the sequences of classification data.

The control data and analysis outcomes may be subsequently retrieved from the cache memory for processing by a message broker service (e.g., the message broker service <NUM> of <FIG>) and/or stored in a database local to the edge node <NUM> (e.g., the database <NUM> of <FIG>) or a remote database (e.g., a database of the computing device <NUM> and/or the computing <NUM>). The message broker service of the edge node <NUM> may be configured to generate one or more messages for transmission to the computing device <NUM> and/or the computing device <NUM>, such as the messages <NUM>, <NUM>, respectively. The messages <NUM>, <NUM> may provide information to users monitoring the environment, such as alerts to indicate defects or improper process parameters (e.g., temperature, speed, etc.) have been detected. In an aspect, the messages may be presented to the user in a textual format, such as to display a message indicating defects are or are not being detected. In an aspect, the displayed message(s) may only change when a change in the status of detection of defects changes. For example, as long as no defects are detected the message may indicate no defects detected, but may be changed when a defect is detected. Similar messages may be displayed for other aspects of the production infrastructure <NUM> being monitored by the edge node <NUM>. In some aspects, one or more visible or audible alerts may be provided, rather than or in addition to a textual alert (e.g., green to indicate no defects and/or all processes operating within tolerable ranges, and red to indicate defects present and/or one or more processes operating outside of tolerable ranges). It is noted that some of the information presented to the user via the graphical user interface may be provided based on information stored in a database local to the edge node <NUM>, such as the database <NUM> of <FIG>, as described above.

Additionally, the messages <NUM>, <NUM> may also be used to store information at a remote database, such as to store information regarding the analysis outcomes (e.g., "{A2 B2 C2}") and/or the sensor data (e.g., A1-An, B1-Bn, C1-Cn, etc., or portions thereof) at a remote database (e.g., a database maintained at the computing device <NUM> or the computing device <NUM>). In some aspects, the sensor data may only be stored in the local and/or remote database when certain events occur, such as to store one or more pieces of media content upon which a determination was made that a defect has occurred). In this manner the volume of data stored at the remote or local database(s) may be minimized while retaining a record of the state of certain key features being monitored within an environment. Similarly, control data may also be stored in the database(s) based on key events, such as when defects are detected or operations of the production infrastructure <NUM> are outside of tolerable ranges. The records stored at the database(s) may be timestamped to enable time sequencing of the data, such as to enable a piece of sensor data to be associated with a control signal transmitted to the controller <NUM>, which may enable a user of the computing device <NUM> or the computing device <NUM> to review the control signals and associated sensor data from which the control signals were generated at a later time, such as during a system or performance audit.

In addition to the messages <NUM>, <NUM>, the message broker of the edge node <NUM> may also provide control signals <NUM> to the controller <NUM> to control operations of the production infrastructure <NUM> based on the analysis by the control logic <NUM>. To illustrate, in the above-example involving monitoring the production infrastructure <NUM> for defects, the edge node <NUM> may provide the control signals <NUM> to the controller <NUM> to control operations the production infrastructure <NUM>. The control signals <NUM> may be generated based on application of the logic parameters <NUM> of the control logic <NUM> to the classifications output by the model(s). The logic parameters <NUM> may be configured to determine whether defects are present, whether operational parameters are within tolerable ranges, or other features related to the production infrastructure <NUM>. For example, the logic parameters <NUM> are shown in <FIG> as including a plurality of logic parameters L<NUM>-Lz. Certain ones of the logic parameters <NUM> may be used to evaluate whether certain defects are present in products moving through the production infrastructure <NUM> based on the classifications output by a first model (e.g., classifications {A}), other ones of the logic parameters <NUM> may be used to evaluate whether the production infrastructure <NUM> is operating within tolerable ranges based on the classifications output by a second model (e.g., classifications {B}), and other logic parameters may be configured to evaluate other features of the production infrastructure <NUM> or potentially other types of use cases (e.g., predicting equipment failures, etc.) based on classifications by another model (e.g., classifications {C}). It is noted that while control logic <NUM> is shown in <FIG> as analyzing or evaluating <NUM> different types of classifications (e.g., classifications {A} {B} {C}), the control logic <NUM> may be configured to analyze or evaluate less than <NUM> types of classifications or more than <NUM> different types of classifications if desired depending on the particular use case involved and the configuration of the control logic and/or models of the system <NUM>. Furthermore, it is noted that a prioritization scheme may be utilized to further optimize the functionality provided by the edge node <NUM> and reduce latency within the system <NUM>, as described in more detail above with reference to <FIG> and <FIG>.

While the description of <FIG> and <FIG> above illustrate features provided by the computing architectures and edge nodes of the present disclosure with respect to several of the use cases from Table <NUM> above, it should be understood that the description of <FIG> and <FIG> are provided for purposes of illustration, rather than by way of limitation and should not be understood to be an exhaustive description of how edge nodes and the computing architectures disclosed herein may be utilized with respect to the illustrated use cases. Furthermore, while <FIG> and <FIG> show a single edge node <NUM>, it is to be understood that more than one edge node <NUM> may be utilized depending on the particular use case(s), the number of sensor devices, the features of the system or environment being monitored, or other factors. For example, in a system incorporating edge nodes in accordance with the computing architectures disclosed herein (e.g., the system <NUM> of <FIG>, the system <NUM> of <FIG>, the system <NUM> of <FIG>, or another system) different portions of the monitored environment (e.g., stages of the production infrastructure <NUM> of FIG. <NUM>) may be associated with a different edge node, thereby providing dedicated edge nodes for each different portion of the monitored environment. Additionally or alternatively, edge nodes may be associated with multiple portions of the monitored environment, which may reduce the number of edge nodes needed to support a particular use case. Furthermore, sensor devices utilized to capture data that is provided to the edge nodes may be specific to one edge node (e.g., each sensor device only provides its data to one edge node), may support multiple edge nodes (e.g., one or more of the sensor devices may provide data to multiple edge nodes), may support all edge nodes (e.g., one or more of the sensor devices may provide data to all edge nodes), or combinations thereof (e.g., some sensor devices only provide data to one edge node, some sensor devices may provide data to multiple edge nodes, other sensor devices may provide data to all edge nodes).

In an aspect, sensor devices utilized by systems in accordance with the present disclosure may also be used to trigger analysis by the edge nodes. For example, in an asset tracking and warehouse management use case a sensor device (e.g., an RFID device) may detect items as they pass a certain location (e.g., an entry way to a warehouse, an aisle, a loading dock, etc.) and information associated with the detected items may be transmitted to an edge node(s). The edge node may then use media content received from other sensor devices (e.g., cameras) and models to track movement of the items to particular locations within the warehouse. Information associated with the locations of the items may then be stored in a database (e.g., a database stored at a memory of the computing device <NUM>, the computing device <NUM>, and/or another data storage device). It is noted that the description above where RFID devices are used as triggering events to detect movement of items has been provided by way of illustration, rather than by way of limitation and the asset tracking and warehouse management systems operating in accordance with the present disclosure may utilize different techniques to detect and track items.

Claim 1:
A method (<NUM>) for monitoring and controlling remote devices and/or systems comprising:
receiving (<NUM>), via a capture service (<NUM>) executable by a first processor of an edge device (<NUM>), input data from one or more data sources, the input data comprising information associated with a monitored environment (<NUM>), one or more monitored devices (<NUM>; <NUM>; <NUM>), or both, wherein the first processor comprises a central processing unit, wherein the input data includes video stream data associated with one or more video streams captured by cameras disposed within the monitored environment, wherein each camera has a field of view that includes a remote device (<NUM>);
applying (<NUM>), by a modelling engine (<NUM>) executable by a second processor of the edge device (<NUM>), two or more machine learning models (<NUM>) to at least a portion of the input data to produce model output data (<NUM>), wherein the second processor comprises a graphics processing unit;
executing, by control logic (<NUM>) executable by the first processor, control logic against the model output data (<NUM>) to produce control data (224A);
generating, via a message broker service (<NUM>) executable by the first processor, at least one control message (<NUM>; <NUM>; <NUM>) based on the control data (224A), the control message (<NUM>; <NUM>; <NUM>) comprising one or more commands corresponding to the remote device (<NUM>; <NUM>; <NUM>); and
transmitting, by the message broker service (<NUM>), the at least one control message (<NUM>; <NUM>; <NUM>) to the remote device (<NUM>; <NUM>; <NUM>).