Patent ID: 12260649

DETAILED DESCRIPTION

The following detailed description of example implementations refers to the accompanying drawings. The same reference numbers in different drawings may identify the same or similar elements.

A smart city may include an active surveillance system with video cameras that assist various services, such as intelligent traffic monitoring, emergency vehicle detection and monitoring, locating a person in a video using facial recognition, and/or the like. For example, conventional traffic monitoring may require the presence of traffic police during rush hours or may utilize a fixed timer-based operation to avoid a human presence all the time. A problem occurs when there is a need to allow an emergency vehicle (e.g., an ambulance, a fire truck, a police car, and/or the like) to pass through an intersection. Video analytics of images received from the surveillance system may enable identification and tracking of emergency vehicles by the surveillance system. However, machine learning models utilized by the surveillance system may fail to explain predictions of the machine learning models. Furthermore, the machine learning models process a huge quantity (e.g., tens of thousands or more) of images every day and may generate incorrect predictions. Without explanations of the predictions of the machine learning models, it is impossible to identify the incorrect predictions generated by the machine learning models.

Therefore, current techniques for processing video data captured by a surveillance system consume computing resources (e.g., processing resources, memory resources, communication resources, and/or the like), networking resources, and/or the like associated with failing to generate explanations for machine learning model predictions, failing to identify incorrect predictions by machine learning models, utilizing the incorrect predictions from the machine learning models to incorrectly manage a smart city (e.g., cause traffic congestion, traffic accidents, and/or the like), and/or the like.

Some implementations described herein relate to a prediction system that determines incorrect predictions by machine learning models and generates explanations for predictions of the machine learning models. For example, the prediction system may receive surveillance video data captured by a surveillance system in a geographic location, and may process the surveillance video data, with a regression analysis model, to derive traffic signal timing in the geographic location. The prediction system may process the surveillance video data, with a deep learning model, to identify objects in the geographic location, and may utilize a segmentation guided attention network model with the objects to determine traffic density count data in the geographic location. The prediction system may utilize a curriculum loss model with the objects to determine crowd count data in the geographic location, and may process the surveillance video data, with a deep learning video analytics model, to identify first events associated with emergency vehicles and accidents in the geographic location. The prediction system may process the surveillance video data, with a classifier model and a deep network model, to identify second events associated with facial recognition in the geographical location. The prediction system may process one or more of the objects, the traffic density count data, the crowd count data, the first events, or the second events, with a dynamic text-based explanation model, to generate a text-based explanation and/or a failure prediction for one or more of the models. The prediction system may perform one or more actions based on the text-based explanation and/or the failure prediction.

In this way, the prediction system determines incorrect predictions by machine learning models and generates explanations for predictions of the machine learning models. The prediction system may utilize deep learning model-based video analytics to detect events, such as emergency vehicle detection and tracking, facial recognition, accident detection, traffic density counting, and/or the like for intelligent operation in a smart city. The prediction system may provide explanations for machine learning model predictions (e.g., an explanation of a facial recognition) by providing textual information that is dynamically generated through inspection of attributes. The prediction system may identify incorrect machine learning model predictions and may utilize the identification of the incorrect predictions to retrain and/or improve the machine learning models. This, in turn, conserves computing resources, networking resources, and/or the like that would otherwise have been consumed in failing to generate explanations for machine learning model predictions, failing to identify incorrect predictions by machine learning models, utilizing the incorrect predictions from the machine learning models to incorrectly manage a smart city (e.g., cause traffic congestion, traffic accidents, and/or the like), and/or the like.

FIGS.1A-1Gare diagrams of an example100associated with determining incorrect predictions by machine learning models and generating explanations for predictions of the machine learning models. As shown inFIGS.1A-1G, example100includes a prediction system associated with a surveillance system. The prediction system may include a system that determines incorrect predictions by machine learning models and generates explanations for predictions of the machine learning models. The surveillance system may include system that perform surveillance of a geographical location. Further details of the prediction system and the surveillance system are provided elsewhere herein.

As shown inFIG.1A, and by reference number105, the prediction system may receive surveillance video data captured by a surveillance system in a geographic location. For example, the surveillance system may surveille and manage assets, resources, services, and/or the like in the geographic location (e.g., an urban area) based on data collected from different types of electronic methods, sensors, and/or the like. For example, the surveillance system may monitor and manage traffic and transportation systems, power plants, utilities, water supply networks, waste systems, criminal investigations, information systems, schools, libraries, hospitals, emergency vehicles, and other community services. In some implementations, the surveillance system may capture surveillance video data associated with citizens, buildings, assets, vehicles, and/or the like of the geographic location. The surveillance video data may include videos, images, and/or the like associated with the geographic location.

In some implementations, the prediction system may continuously receive the surveillance video data from the surveillance system, may periodically receive the surveillance video data from the surveillance system, may receive the surveillance video data from the surveillance system based on providing a request for the surveillance video data to the surveillance system, and/or the like. In some implementations, the prediction system may store the surveillance video data in a data structure (e.g., a database, a table, a list, and/or the like) associated with the prediction system.

As further shown inFIG.1A, and by reference number110, the prediction system may process the surveillance video data, with a deep network model followed by a regression analysis model, to derive traffic signal timing in the geographical location. For example, the prediction system may utilize the surveillance video data for traffic density detection in the geographic location, detection and tracking of emergency vehicles in the geographic location, accident identification in the geographic location, facial recognition of a person of interest, and/or the like. In some implementations, the prediction system may process the surveillance video data to identify a vehicle count associated with traffic in the geographic location. The prediction system may process the vehicle count, with a deep network model followed by a regression analysis model, to derive a traffic signal timing in the geographical location. For example, the prediction system may detect an emergency vehicle in traffic, and may modify the traffic signal timing so that the emergency vehicle may quickly travel through the geographic location (e.g., to a hospital or an accident scene), which may save lives. In another example, the prediction system may detect a traffic accident in the geographic location, and may generate an alarm based on detecting the traffic accident. The prediction system may also modify the traffic signal timing so that an emergency vehicle may quickly travel through the geographic location to the traffic accident scene.

As shown inFIG.1B, and by reference number115, the prediction system may process the surveillance video data, with a deep learning model, to identify objects in the geographic location. For example, the prediction system may train a deep learning model for object detection based on the surveillance video data. In some implementations, the deep learning model may include a YOLO (you only look once) object detection model that detects and recognizes various objects in an image (e.g., in real time). Object detection in the YOLO model may be executed as a single regression problem and may provide class probabilities of detected images. In some implementations, the prediction system may train the deep learning model with video data identifying non-emergency vehicles, emergency vehicles, accident scenes, pedestrians, and/or the like. Further details of training a model are provided below in connection withFIG.2. In some implementations, once the deep learning model is trained, the prediction system may process the surveillance video data, with the deep learning model, to identify objects (e.g., emergency vehicles, persons of interest, buildings, and/or the like) in the geographic location.

As shown inFIG.1C, and by reference number120, the prediction system may utilize a segmentation guided attention network (SGANet) model with the objects to determine traffic density count data in the geographic location and utilize a curriculum loss model with the objects to determine crowd count data in the geographic location. For example, a SGANet model may include a deep convolutional neural network (CNN) model that may be utilized to count objects in images. The prediction system may process the objects identified in the surveillance video data, with the SGANet model, to determine the traffic density count data in the geographic location. The traffic density count data may include data identifying a quantity of traffic (e.g., vehicles) in the geographic location or a portion of the geographic location (e.g., a street, a sidewalk, and/or the like). In some implementations, the prediction system may utilize the SGANet model, with an Inception model (e.g., a CNN model for assisting in image analysis and object detection), to process the objects and determine the traffic density count data.

In some implementations, the prediction system may process the objects, with a curriculum loss model, to determine the crowd count data in the geographic location. The crowd count data may include data identifying a quantity of people in the geographic location or a portion of the geographic location, a quantity of vehicles on a road, and/or the like. Generalization may be important for many deep network models and may become more challenging when high robustness is required for learning with noisy labels. A “0-1” loss has monotonic relationship between empirical adversary (reweighted) risk, and is robust to outliers. To efficiently optimize “0-1” loss while keeping its robust properties, a deep network model may include a curriculum loss. Thus, the curriculum loss model may include a deep network model with the curriculum loss. In some implementations, the prediction system may utilize the traffic density count data and the crowd count data to automate traffic monitoring for the geographic location and to assist administrative agencies through real time detection of various events from the surveillance video data using computer vision and deep learning.

As shown inFIG.1D, and by reference number125, the prediction system may process the surveillance video data, with a deep learning video analytics model, to identify first events associated with emergency vehicles and accidents in the geographic location. For example, the prediction system may train a deep learning video analytics model to identify the first events associated with emergency vehicles and accidents in the geographic location. In some implementations, the deep learning video analytics model may include a deep neural network (DNN) object detection model that detects and recognizes various objects in an image (e.g., in real time). Object detection in the DNN model may be executed as a regression problem and may provide class probabilities of detected images. In some implementations, the prediction system may train the deep learning video analytics model with video data identifying non-emergency vehicles, emergency vehicles, accident scenes, pedestrians, and/or the like. Further details of training a model are provided below in connection withFIG.2. In some implementations, once the deep learning video analytics model is trained, the prediction system may process the surveillance video data, with the deep learning video analytics model, to identify the first events associated with emergency vehicles and accidents in the geographic location.

As shown inFIG.1E, and by reference number130, the prediction system may process the surveillance video data, with a classifier model and a deep network model, to identify second events associated with facial recognition in the geographical location. For example, the prediction system be associated with a classifier model (e.g., a Haar cascade classifier model) and a deep network model (e.g., a visual geometry group (VGG) CNN model). For facial recognition of a person of interest, the prediction system may localize a face of the person from the surveillance video data using the classifier model, and may utilize the deep network model to perform facial recognition of the localized face of the person. In some implementations, the prediction system may train the classifier model and the deep network model with video data identifying people, human faces, non-human faces, animals, and/or the like. Further details of training a model are provided below in connection withFIG.2.

In some implementations, the prediction system may generate alarms for the second events associated with facial recognition in the geographical location and when the second events identify people of interest (e.g., are recognized by the prediction system). A person of interest may include an escapee, a criminal, a missing person, and/or the like associated with a facial image that is uploaded by an administrative agency (e.g., a police department, a government, and/or the like) to a data structure associated with the prediction system.

As shown inFIG.1F, and by reference number135, the prediction system may process one or more of the objects, the traffic density count data, the crowd count data, the first events, or the second events, with a dynamic text-based explanation model, to generate a text-based explanation and/or a failure prediction for one or more of the models. For example, the prediction system may provide robust methods to explain outputs by each of the models described above. In some implementations, the prediction system may process the traffic signal timing, with the dynamic text-based explanation model, to generate a text-based explanation and/or a failure prediction for an object detection or classification model; may process the objects, with the dynamic text-based explanation model, to generate a text-based explanation and/or a failure prediction for the deep learning model; may process the traffic density count data, with the dynamic text-based explanation model, to generate a text-based explanation and/or a failure prediction for the segmentation guided attention network model; may process the crowd count data, with the dynamic text-based explanation model, to generate a text-based explanation and/or a failure prediction for the curriculum loss model; may process the first events, with the dynamic text-based explanation model, to generate a text-based explanation and/or a failure prediction for the deep learning video analytics model; may process the second events, with the dynamic text-based explanation model, to generate a text-based explanation and/or a failure prediction for the classifier model and the deep network model; and/or the like.

In some implementations, the dynamic text-based explanation model may include a local interpretable model-agnostic explanations (LIME) model that generates text-based explanations of facial recognitions based on feature similarity and in human understandable way. The text-based explanations and/or the failure predictions generated by the prediction system may enable the one or more models to be improved and/or audited by providing insights into the predictions of the one or more models.

In some implementations, the prediction system may utilize the LIME model to explain facial recognitions through pixel attribution and by deriving a closest matching image from a data structure. In this way, the prediction system enables a user to judge model predictions and to assess trustworthiness of the models.

In some implementations, when processing the one or more of the objects, the traffic density count data, the crowd count data, the first events, or the second events, with the dynamic text-based explanation model, to generate the text-based explanation and/or the failure prediction, the prediction system may process the one or more of the objects, the traffic density count data, the crowd count data, the first events, or the second events, with the LIME model, to generate a set of features. The prediction system may process the set of features, with a VGG CNN model, to generate a processed set of features. The prediction system may map the processed set of features to a vector space of a deep long short-term memory (LSTM) model, and may generate the text-based explanation via the deep LSTM model.

In some implementations, when processing the one or more of the objects, the traffic density count data, the crowd count data, the first events, or the second events, with the dynamic text-based explanation model, to generate the text-based explanation and/or the failure prediction, the prediction system may compare a golden dataset of images with images of the surveillance video data utilized by the one or more of the regression analysis model, the deep learning model, the segmentation guided attention network model, the curriculum loss model, the deep learning video analytics model, the classifier model or the deep network model. The prediction system may identify a set of images, associated with a failure by the one or more of an object detection or classification model, the regression analysis model, the deep learning model, the segmentation guided attention network model, the curriculum loss model, the deep learning video analytics model, the classifier model or the deep network model, based on comparing the golden dataset of images with images of the surveillance video data, and may generate the failure prediction based on the set of images.

The failure prediction may include identification of a set of images associated with a failure by the one or more of the object detection or classification model, the regression analysis model, the deep learning model, the segmentation guided attention network model, the curriculum loss model, the deep learning video analytics model, the classifier model or the deep network model. In some implementations, the failure prediction may include a textual explanation of a failure by the one or more of the regression analysis model, the deep learning model, the segmentation guided attention network model, the curriculum loss model, the deep learning video analytics model, the classifier model or the deep network model.

Video surveillance of a smart city requires processing of a large quantity of images every day, making it impossible to manually check explanations for all the images and to identify a failure scenario. The prediction system provides an approach for identifying a set of images for which a model has higher chances of failure (e.g., by identifying a closest match from the golden dataset).

In this way, the prediction system may integrate deep learning-based video analytics to detect events, such as emergency vehicle detection and tracking, facial recognition, accident detection, traffic density counting, and/or the like for intelligent operation in a smart city. The prediction system may explain a facial recognition by providing textual information that is dynamically generated through inspection of facial attributes generated by explainable methods. The prediction system may identify images associated with a most probable failure prediction for a model based on explainable feature attribution in connection with a golden dataset.

As shown inFIG.1G, and by reference number140, the prediction system may perform one or more actions based on the text-based explanation and/or the failure prediction. In some implementations, performing the one or more actions includes the prediction system retraining one or more of the models based on the failure prediction. For example, the prediction system may utilize the failure prediction as additional training data for retraining the one or more of the models, thereby increasing the quantity of training data available for training the one or more of the models. Accordingly, the prediction system may conserve computing resources associated with identifying, obtaining, and/or generating historical data for training the one or more of the models relative to other systems for identifying, obtaining, and/or generating historical data for training machine learning models.

In some implementations, performing the one or more actions includes the prediction system providing a text-based explanation of problem video frames associated with the failure prediction. For example, the prediction system may identify problem video frames associated with a failed prediction by one of the models, and may generate a text-based explanation of the problem video frames (e.g., the model identified a dog as a person). The prediction system may provide the problem video frames and the text-based explanation for display to a user of the prediction system. In this way, the prediction system conserves computing resources, networking resources, and/or the like that would otherwise have been consumed in failing to generate explanations for machine learning model predictions.

In some implementations, performing the one or more actions includes the prediction system alerting authorities when the text-based explanation is associated with facial recognition of a person of interest. For example, when the prediction system identifies a person of interest (e.g., a criminal), the prediction system may generate a text-based explanation of the person of interest. The text-based explanation may indicate that the identified person matches the person of interest because of specific facial attributes (e.g., eye color, bone structure, and/or the like). The prediction system may alert authorities about a last known location of the person of interest as determined from the surveillance video data. In this way, the prediction system conserves computing resources, networking resources, and/or the like that would otherwise have been consumed in failing to provide understandable machine learning model predictions.

In some implementations, performing the one or more actions includes the prediction system providing problem video frames associated with the failure prediction. For example, the prediction system may identify problem video frames associated with a failed prediction by one of the models, and may provide the problem video frames for display to a user of the prediction system. In this way, the prediction system conserves computing resources, networking resources, and/or the like that would otherwise have been consumed in failing to generate explanations for machine learning model predictions.

In some implementations, performing the one or more actions includes the prediction system providing the text-based explanation and/or the failure prediction for display. For example, the prediction system may provide the text-based explanation and/or the failure prediction for display to a user of the prediction system. The user may utilize the text-based explanation to further understand a prediction of one of the models, and may utilize the failure prediction to modify and/or improve the one of the models. In some implementations, the text-based explanation may be replaced with an audible explanation, may be utilized by other computer-implemented models to make decisions and/or to improve performance of the models described in connection withFIGS.1A-1G, and/or the like. In this way, the prediction system conserves computing resources, networking resources, and/or the like that would otherwise have been consumed in failing to identify incorrect predictions by machine learning models, utilizing the incorrect predictions from machine learning models to incorrectly manage a smart city, and/or the like.

In this way, the prediction system determines incorrect predictions by machine learning models and generates explanations for predictions of the machine learning models. The prediction system may utilize deep learning model-based video analytics to detect events, such as emergency vehicle detection and tracking, facial recognition, accident detection, traffic density counting, and/or the like for intelligent operation in a smart city. The prediction system may provide explanations for machine learning model predictions (e.g., an explanation of a facial recognition) by providing textual information that is dynamically generated through inspection of attributes. The prediction system may identify incorrect machine learning model predictions and may utilize the identification of the incorrect predictions to retrain and/or improve the machine learning models. This, in turn, conserves computing resources, networking resources, and/or the like that would otherwise have been consumed in failing to generate explanations for machine learning model predictions, failing to identify incorrect predictions by machine learning models, utilizing the incorrect predictions from the machine learning models to incorrectly manage a smart city (e.g., cause traffic congestion, traffic accidents, and/or the like), and/or the like.

As indicated above,FIGS.1A-1Gare provided as an example. Other examples may differ from what is described with regard toFIGS.1A-1G. The number and arrangement of devices shown inFIGS.1A-1Gare provided as an example. In practice, there may be additional devices, fewer devices, different devices, or differently arranged devices than those shown inFIGS.1A-1G. Furthermore, two or more devices shown inFIGS.1A-1Gmay be implemented within a single device, or a single device shown inFIGS.1A-1Gmay be implemented as multiple, distributed devices. Additionally, or alternatively, a set of devices (e.g., one or more devices) shown inFIGS.1A-1Gmay perform one or more functions described as being performed by another set of devices shown inFIGS.1A-1G.

FIG.2is a diagram illustrating an example200of training and using a machine learning model for determining incorrect predictions by machine learning models and generating explanations for predictions of the machine learning models. The machine learning model training and usage described herein may be performed using a machine learning system. The machine learning system may include or may be included in a computing device, a server, a cloud computing environment, and/or the like, such as the prediction system described in more detail elsewhere herein.

As shown by reference number205, a machine learning model may be trained using a set of observations. The set of observations may be obtained from historical data, such as data gathered during one or more processes described herein. In some implementations, the machine learning system may receive the set of observations (e.g., as input) from the prediction system, as described elsewhere herein.

As shown by reference number210, the set of observations includes a feature set. The feature set may include a set of variables, and a variable may be referred to as a feature. A specific observation may include a set of variable values (or feature values) corresponding to the set of variables. In some implementations, the machine learning system may determine variables for a set of observations and/or variable values for a specific observation based on input received from the prediction system. For example, the machine learning system may identify a feature set (e.g., one or more features and/or feature values) by extracting the feature set from structured data, by performing natural language processing to extract the feature set from unstructured data, by receiving input from an operator, and/or the like.

As an example, a feature set for a set of observations may include a first feature of objects, a second feature of first events, a third feature of second events, and so on. As shown, for a first observation, the first feature may have a value of objects1, the second feature may have a value of first events1, the third feature may have a value of second events1, and so on. These features and feature values are provided as examples and may differ in other examples.

As shown by reference number215, the set of observations may be associated with a target variable. The target variable may represent a variable having a numeric value, may represent a variable having a numeric value that falls within a range of values or has some discrete possible values, may represent a variable that is selectable from one of multiple options (e.g., one of multiple classes, classifications, labels, and/or the like), may represent a variable having a Boolean value, and/or the like. A target variable may be associated with a target variable value, and a target variable value may be specific to an observation. In example200, the target variable may be a text-based explanation and may include a value of text-based explanation1for the first observation.

The target variable may represent a value that a machine learning model is being trained to predict, and the feature set may represent the variables that are input to a machine learning model to predict a value for the target variable. The set of observations may include target variable values so that the machine learning model can be trained to recognize patterns in the feature set that lead to a target variable value. A machine learning model that is trained to predict a target variable value may be referred to as a supervised learning model.

In some implementations, the machine learning model may be trained on a set of observations that do not include a target variable. This may be referred to as an unsupervised learning model. In this case, the machine learning model may learn patterns from the set of observations without labeling or supervision, and may provide output that indicates such patterns, such as by using clustering and/or association to identify related groups of items within the set of observations.

As shown by reference number220, the machine learning system may train a machine learning model using the set of observations and using one or more machine learning algorithms, such as a regression algorithm, a decision tree algorithm, a neural network algorithm, a k-nearest neighbor algorithm, a support vector machine algorithm, and/or the like. After training, the machine learning system may store the machine learning model as a trained machine learning model225to be used to analyze new observations.

As shown by reference number230, the machine learning system may apply the trained machine learning model225to a new observation, such as by receiving a new observation and inputting the new observation to the trained machine learning model225. As shown, the new observation may include a first feature of objects X, a second feature of first events Y, a third feature of second events Z, and so on, as an example. The machine learning system may apply the trained machine learning model225to the new observation to generate an output (e.g., a result). The type of output may depend on the type of machine learning model and/or the type of machine learning task being performed. For example, the output may include a predicted value of a target variable, such as when supervised learning is employed. Additionally, or alternatively, the output may include information that identifies a cluster to which the new observation belongs, information that indicates a degree of similarity between the new observation and one or more other observations, and/or the like, such as when unsupervised learning is employed.

As an example, the trained machine learning model225may predict a value of text-based explanation A for the target variable for the new observation, as shown by reference number235. Based on this prediction, the machine learning system may provide a first recommendation, may provide output for determination of a first recommendation, may perform a first automated action, may cause a first automated action to be performed (e.g., by instructing another device to perform the automated action), and/or the like.

In some implementations, the trained machine learning model225may classify (e.g., cluster) the new observation in a cluster, as shown by reference number240. The observations within a cluster may have a threshold degree of similarity. As an example, if the machine learning system classifies the new observation in a first cluster (e.g., an objects cluster), then the machine learning system may provide a first recommendation. Additionally, or alternatively, the machine learning system may perform a first automated action and/or may cause a first automated action to be performed (e.g., by instructing another device to perform the automated action) based on classifying the new observation in the first cluster.

As another example, if the machine learning system were to classify the new observation in a second cluster (e.g., a first events cluster), then the machine learning system may provide a second (e.g., different) recommendation and/or may perform or cause performance of a second (e.g., different) automated action.

In some implementations, the recommendation and/or the automated action associated with the new observation may be based on a target variable value having a particular label (e.g., classification, categorization, and/or the like), may be based on whether a target variable value satisfies one or more thresholds (e.g., whether the target variable value is greater than a threshold, is less than a threshold, is equal to a threshold, falls within a range of threshold values, and/or the like), may be based on a cluster in which the new observation is classified, and/or the like.

In this way, the machine learning system may apply a rigorous and automated process to determine incorrect predictions by machine learning models and generate explanations for predictions of the machine learning models. The machine learning system enables recognition and/or identification of tens, hundreds, thousands, or millions of features and/or feature values for tens, hundreds, thousands, or millions of observations, thereby increasing accuracy and consistency and reducing delay associated with determining incorrect predictions by machine learning models and generating explanations for predictions of the machine learning models relative to requiring computing resources to be allocated for tens, hundreds, or thousands of operators to manually determine incorrect predictions by machine learning models and generate explanations for predictions of the machine learning models.

As indicated above,FIG.2is provided as an example. Other examples may differ from what is described in connection withFIG.2.

FIG.3is a diagram of an example environment300in which systems and/or methods described herein may be implemented. As shown inFIG.3, the environment300may include a prediction system301, which may include one or more elements of and/or may execute within a cloud computing system302. The cloud computing system302may include one or more elements303-313, as described in more detail below. As further shown inFIG.3, the environment300may include a network320and/or a surveillance system330. Devices and/or elements of the environment300may interconnect via wired connections and/or wireless connections.

The cloud computing system302includes computing hardware303, a resource management component304, a host operating system (OS)305, and/or one or more virtual computing systems306. The resource management component304may perform virtualization (e.g., abstraction) of the computing hardware303to create the one or more virtual computing systems306. Using virtualization, the resource management component304enables a single computing device (e.g., a computer, a server, and/or the like) to operate like multiple computing devices, such as by creating multiple isolated virtual computing systems306from the computing hardware303of the single computing device. In this way, the computing hardware303can operate more efficiently, with lower power consumption, higher reliability, higher availability, higher utilization, greater flexibility, and lower cost than using separate computing devices.

The computing hardware303includes hardware and corresponding resources from one or more computing devices. For example, the computing hardware303may include hardware from a single computing device (e.g., a single server) or from multiple computing devices (e.g., multiple servers), such as multiple computing devices in one or more data centers. As shown, the computing hardware303may include one or more processors307, one or more memories308, one or more storage components309, and/or one or more networking components310. Examples of a processor, a memory, a storage component, and a networking component (e.g., a communication component) are described elsewhere herein.

The resource management component304includes a virtualization application (e.g., executing on hardware, such as the computing hardware303) capable of virtualizing the computing hardware303to start, stop, and/or manage the one or more virtual computing systems306. For example, the resource management component304may include a hypervisor (e.g., a bare-metal or Type1hypervisor, a hosted or Type2hypervisor, and/or the like) or a virtual machine monitor, such as when the virtual computing systems306are virtual machines311. Additionally, or alternatively, the resource management component304may include a container manager, such as when the virtual computing systems306are containers312. In some implementations, the resource management component304executes within and/or in coordination with a host operating system305.

A virtual computing system306includes a virtual environment that enables cloud-based execution of operations and/or processes described herein using computing hardware303. As shown, a virtual computing system306may include a virtual machine311, a container312, a hybrid environment313that includes a virtual machine and a container, and/or the like. A virtual computing system306may execute one or more applications using a file system that includes binary files, software libraries, and/or other resources required to execute applications on a guest operating system (e.g., within the virtual computing system306) or the host operating system305.

Although the prediction system301may include one or more elements303-313of the cloud computing system302, may execute within the cloud computing system302, and/or may be hosted within the cloud computing system302, in some implementations, the prediction system301may not be cloud-based (e.g., may be implemented outside of a cloud computing system) or may be partially cloud-based. For example, the prediction system301may include one or more devices that are not part of the cloud computing system302, such as device400ofFIG.4, which may include a standalone server or another type of computing device. The prediction system301may perform one or more operations and/or processes described in more detail elsewhere herein.

The network320includes one or more wired and/or wireless networks. For example, the network320may include a cellular network, a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a private network, the Internet, and/or the like, and/or a combination of these or other types of networks. The network320enables communication among the devices of the environment300.

The surveillance system330may include one or more devices capable of receiving, generating, storing, processing, providing, and/or routing information, as described elsewhere herein. The surveillance system330may include a communication device and/or a computing device. For example, the surveillance system330may include a server, such as an application server, a client server, a web server, a database server, a host server, a proxy server, a virtual server (e.g., executing on computing hardware), or a server in a cloud computing system. In some implementations, the surveillance system330may include computing hardware used in a cloud computing environment.

The number and arrangement of devices and networks shown inFIG.3are provided as an example. In practice, there may be additional devices and/or networks, fewer devices and/or networks, different devices and/or networks, or differently arranged devices and/or networks than those shown inFIG.3. Furthermore, two or more devices shown inFIG.3may be implemented within a single device, or a single device shown inFIG.3may be implemented as multiple, distributed devices. Additionally, or alternatively, a set of devices (e.g., one or more devices) of the environment300may perform one or more functions described as being performed by another set of devices of the environment300.

FIG.4is a diagram of example components of a device400, which may correspond to the prediction system301and/or the surveillance system330. In some implementations, the prediction system301and/or the surveillance system330may include one or more devices400and/or one or more components of the device400. As shown inFIG.4, the device400may include a bus410, a processor420, a memory430, an input component440, an output component450, and a communication component460.

The bus410includes a component that enables wired and/or wireless communication among the components of device400. The processor420includes a central processing unit, a graphics processing unit, a microprocessor, a controller, a microcontroller, a digital signal processor, a field-programmable gate array, an application-specific integrated circuit, and/or another type of processing component. The processor420is implemented in hardware, firmware, or a combination of hardware and software. In some implementations, the processor420includes one or more processors capable of being programmed to perform a function. The memory430includes a random-access memory, a read only memory, and/or another type of memory (e.g., a flash memory, a magnetic memory, and/or an optical memory).

The input component440enables the device400to receive input, such as user input and/or sensed inputs. For example, the input component440may include a touch screen, a keyboard, a keypad, a mouse, a button, a microphone, a switch, a sensor, a global positioning system component, an accelerometer, a gyroscope, an actuator, and/or the like. The output component450enables the device400to provide output, such as via a display, a speaker, and/or one or more light-emitting diodes. The communication component460enables the device400to communicate with other devices, such as via a wired connection and/or a wireless connection. For example, the communication component460may include a receiver, a transmitter, a transceiver, a modem, a network interface card, an antenna, and/or the like.

The device400may perform one or more processes described herein. For example, a non-transitory computer-readable medium (e.g., the memory430) may store a set of instructions (e.g., one or more instructions, code, software code, program code, and/or the like) for execution by the processor420. The processor420may execute the set of instructions to perform one or more processes described herein. In some implementations, execution of the set of instructions, by one or more processors420, causes the one or more processors420and/or the device400to perform one or more processes described herein. In some implementations, hardwired circuitry may be used instead of or in combination with the instructions to perform one or more processes described herein. Thus, implementations described herein are not limited to any specific combination of hardware circuitry and software.

The number and arrangement of components shown inFIG.4are provided as an example. The device400may include additional components, fewer components, different components, or differently arranged components than those shown inFIG.4. Additionally, or alternatively, a set of components (e.g., one or more components) of the device400may perform one or more functions described as being performed by another set of components of the device400.

FIG.5is a flowchart of an example process500for determining incorrect predictions by machine learning models and generating explanations for predictions of the machine learning models. In some implementations, one or more process blocks ofFIG.5may be performed by a device (e.g., the prediction system301). In some implementations, one or more process blocks ofFIG.5may be performed by another device or a group of devices separate from or including the device, such as a surveillance system (e.g., the surveillance system330). Additionally, or alternatively, one or more process blocks ofFIG.5may be performed by one or more components of the device400, such as the processor420, the memory430, the input component440, the output component450, and/or the communication component460.

As shown inFIG.5, process500may include receiving surveillance video data captured by a surveillance system in a geographic location (block510). For example, the device may receive surveillance video data captured by a surveillance system in a geographic location, as described above.

As further shown inFIG.5, process500may include processing the surveillance video data, with a deep learning model, to identify objects in the geographic location (block520). For example, the device may process the surveillance video data, with a deep learning model, to identify objects in the geographic location, as described above. In some implementations, processing the surveillance video data, with the deep learning model, to identify the objects in the geographic location includes processing the surveillance video data, with a you only look once object detection model, to identify the objects in the geographic location.

As further shown inFIG.5, process500may include utilizing a segmentation guided attention network model with the objects to determine traffic density count data in the geographic location (block530). For example, the device may utilize a segmentation guided attention network model with the objects to determine traffic density count data in the geographic location, as described above. In some implementations, utilizing the segmentation guided attention network model with the objects to determine the traffic density count data in the geographic location includes utilizing the segmentation guided attention network model, and a convolutional neural network as backbone, with the objects to determine traffic density count data in the geographic location.

As further shown inFIG.5, process500may include processing the surveillance video data, with a regression analysis model, to derive traffic signal timing in the geographic location (block540). For example, the device may process the surveillance video data, with a regression analysis model, to derive traffic signal timing in the geographic location, as described above.

As further shown inFIG.5, process500may include utilizing a curriculum loss model with the objects to determine crowd count data in the geographic location (block550). For example, the device may utilize a curriculum loss model with the objects to determine crowd count data in the geographic location, as described above.

As further shown inFIG.5, process500may include processing the surveillance video data, with a deep learning video analytics model, to identify first events associated with emergency vehicles and accidents in the geographic location (block560). For example, the device may process the surveillance video data, with a deep learning video analytics model, to identify first events associated with emergency vehicles and accidents in the geographic location, as described above.

As further shown inFIG.5, process500may include processing the surveillance video data, with a classifier model and a deep network model, to identify second events associated with facial recognition in the geographical location (block570). For example, the device may process the surveillance video data, with a classifier model and a deep network model, to identify second events associated with facial recognition in the geographical location, as described above. In some implementations, processing the surveillance video data, with the classifier model and the deep network model, to identify the second events associated with the facial recognition in the geographical location includes processing the surveillance video data, with a Haar cascade classifier model and a visual geometry group convolutional neural network model, to identify the second events associated with the facial recognition in the geographical location.

As further shown inFIG.5, process500may include processing one or more of the objects, the traffic density count data, the crowd count data, the first events, or the second events, with a dynamic text-based explanation model, to generate a text-based explanation and/or a failure prediction for one or more of the regression analysis model, the deep learning model, the segmentation guided attention network model, the curriculum loss model, the deep learning video analytics model, the classifier model or the deep network model (block580). For example, the device may process one or more of the objects, the traffic density count data, the crowd count data, the first events, or the second events, with a dynamic text-based explanation model, to generate a text-based explanation and/or a failure prediction for one or more of the regression analysis model, the deep learning model, the segmentation guided attention network model, the curriculum loss model, the deep learning video analytics model, the classifier model or the deep network model, as described above.

In some implementations, processing the one or more of the objects, the traffic density count data, the crowd count data, the first events, or the second events, with the dynamic text-based explanation model, to generate the text-based explanation and/or the failure prediction includes processing the one or more of the objects, the traffic density count data, the crowd count data, the first events, or the second events, with a local interpretable model-agnostic explanations model, to generate the text-based explanation and/or the failure prediction.

In some implementations, processing the one or more of the objects, the traffic density count data, the crowd count data, the first events, or the second events, with the dynamic text-based explanation model, to generate the text-based explanation and/or the failure prediction includes processing the one or more of the objects, the traffic density count data, the crowd count data, the first events, or the second events, with a randomized input sampling for explanation model, to generate the text-based explanation and/or the failure prediction.

In some implementations, processing the one or more of the objects, the traffic density count data, the crowd count data, the first events, or the second events, with the dynamic text-based explanation model, to generate the text-based explanation and/or the failure prediction includes processing the one or more of the objects, the traffic density count data, the crowd count data, the first events, or the second events, with a local interpretable model-agnostic explanations model, to generate a set of features, processing the set of features, with a visual geometry group convolutional neural network model, to generate a processed set of features, mapping the processed set of features to a vector space of a deep long short-term memory (LSTM) model, and generating the text-based explanation and/or the failure prediction via the deep LSTM model.

In some implementations, processing the one or more of the objects, the crowd count data, the first events, or the second events, with the dynamic text-based explanation model, to generate the failure prediction includes comparing a golden dataset of images with images of the surveillance video data utilized by the one or more of the regression analysis model, the deep learning model, the segmentation guided attention network model, the curriculum loss model, the deep learning video analytics model, the classifier model or the deep network model, identifying a set of images, associated with a failure by the one or more of the regression analysis model, the deep learning model, the segmentation guided attention network model, the curriculum loss model, the deep learning video analytics model, the classifier model or the deep network model, based on comparing the golden dataset of images with images of the surveillance video data, and generating the failure prediction based on the set of images.

In some implementations, the failure prediction includes identification of a set of images associated with a failure by the one or more of an object detection or classification model, the deep learning model, the segmentation guided attention network model, the curriculum loss model, the deep learning video analytics model, the classifier model or the deep network model. In some implementations, the failure prediction includes a textual explanation of a failure by the one or more of the regression analysis model, the deep learning model, the segmentation guided attention network model, the curriculum loss model, the deep learning video analytics model, the classifier model or the deep network model.

As further shown inFIG.5, process500may include performing one or more actions based on the text-based explanation and/or the failure prediction (block590). For example, the device may perform one or more actions based on the text-based explanation and/or the failure prediction, as described above. In some implementations, performing the one or more actions includes retraining one or more of the regression analysis model, the deep learning model, the segmentation guided attention network model, the curriculum loss model, the deep learning video analytics model, the classifier model or the deep network model based on the failure prediction. In some implementations, performing the one or more actions includes one or more of providing a text-based explanation of problem video frames associated with the failure prediction, or providing the problem video frames associated with the failure prediction. In some implementations, performing the one or more actions includes one or more of alerting authorities when the text-based explanation is associated with facial recognition of a person of interest, or providing the text-based explanation and/or the failure prediction for display.

AlthoughFIG.5shows example blocks of process500, in some implementations, process500may include additional blocks, fewer blocks, different blocks, or differently arranged blocks than those depicted inFIG.5. Additionally, or alternatively, two or more of the blocks of process500may be performed in parallel.

The foregoing disclosure provides illustration and description but is not intended to be exhaustive or to limit the implementations to the precise form disclosed. Modifications may be made in light of the above disclosure or may be acquired from practice of the implementations.

As used herein, the term “component” is intended to be broadly construed as hardware, firmware, or a combination of hardware and software. It will be apparent that systems and/or methods described herein may be implemented in different forms of hardware, firmware, and/or a combination of hardware and software. The actual specialized control hardware or software code used to implement these systems and/or methods is not limiting of the implementations. Thus, the operation and behavior of the systems and/or methods are described herein without reference to specific software code—it being understood that software and hardware can be used to implement the systems and/or methods based on the description herein.

As used herein, satisfying a threshold may, depending on the context, refer to a value being greater than the threshold, greater than or equal to the threshold, less than the threshold, less than or equal to the threshold, equal to the threshold, and/or the like, depending on the context.

Although particular combinations of features are recited in the claims and/or disclosed in the specification, these combinations are not intended to limit the disclosure of various implementations. In fact, many of these features may be combined in ways not specifically recited in the claims and/or disclosed in the specification. Although each dependent claim listed below may directly depend on only one claim, the disclosure of various implementations includes each dependent claim in combination with every other claim in the claim set.

No element, act, or instruction used herein should be construed as critical or essential unless explicitly described as such. Also, as used herein, the articles “a” and “an” are intended to include one or more items and may be used interchangeably with “one or more.” Further, as used herein, the article “the” is intended to include one or more items referenced in connection with the article “the” and may be used interchangeably with “the one or more.” Furthermore, as used herein, the term “set” is intended to include one or more items (e.g., related items, unrelated items, a combination of related and unrelated items, and/or the like), and may be used interchangeably with “one or more.” Where only one item is intended, the phrase “only one” or similar language is used. Also, as used herein, the terms “has,” “have,” “having,” or the like are intended to be open-ended terms. Further, the phrase “based on” is intended to mean “based, at least in part, on” unless explicitly stated otherwise. Also, as used herein, the term “or” is intended to be inclusive when used in a series and may be used interchangeably with “and/or,” unless explicitly stated otherwise (e.g., if used in combination with “either” or “only one of”).

In the preceding specification, various example embodiments have been described with reference to the accompanying drawings. It will, however, be evident that various modifications and changes may be made thereto, and additional embodiments may be implemented, without departing from the broader scope of the invention as set forth in the claims that follow. The specification and drawings are accordingly to be regarded in an illustrative rather than restrictive sense.