Epistemic and aleatoric deep plasticity based on sound feedback

Simulating uncertainty in an artificial neural network is provided. Aleatoric uncertainty is simulated to measure what the artificial neural network does not understand from sensor data received from an object operating in a real-world environment by adding random values to edge weights between nodes in the artificial neural network during backpropagation of output data of the artificial neural network and measuring impact on the output data by the added random values to the edge weights between the nodes. Epistemic uncertainty is simulated to measure what the artificial neural network does not know by dropping out a selected node from each respective layer of the artificial neural network during forward propagation of the sensor data and measuring impact of dropped out nodes on the output data of the artificial neural network. An action corresponding to the object is performed based on the impact of simulating the aleatoric and epistemic uncertainty.

BACKGROUND

The disclosure relates generally to deep learning and more specifically to simulating uncertainty while training an artificial neural network so as to predict an ability of the neural network to produce accurate results amidst uncertainty during later real-world application.

2. Description of the Related Art

Deep learning is a branch of machine learning based on a set of algorithms that attempts to model high-level abstractions in data by using artificial neural network architectures composed of multiple non-linear transformations. This machine learning can be supervised or unsupervised. Deep learning architectures have been applied to fields, such as, for example, audio recognition, computer vision, speech recognition, and natural language processing, where these architectures have produced results comparable to and in some cases superior to human experts.

Deep learning uses a cascade of multiple layers of processing nodes for feature extraction and transformation. Each successive layer of nodes uses the output from the previous layer of nodes as input. The deep architecture is organized as a graph. The nodes in the graph are connected by edges or links to propagate activation, calculated at the origin, to the destination nodes. Each edge has a weight that determines the relative strength and sign of the connection and each node applies an activation function to all of the weighted sum of incoming activations. The activation function is given as a hard threshold, a sigmoid function, or a hyperbolic tangent, for example.

Such deep architectures learn progressively and improve performance on tasks by considering examples, generally without task-specific programming. For example, in audio recognition, these deep architectures might learn to identify sounds that indicate panic in a crowd of people by analyzing sound samples that have been manually labeled as “panic” or “no panic” and using the results to identify panicked crowds in other sound inputs. These deep architectures are able to do this without any prior knowledge about crowds of people. Instead, these deep architectures evolve their own set of relevant characteristics from the training data that they process. However, current deep architecture models do not perform well in real-world environments due to errors in measurements or not accounting for what the current models don't know. In other words, these current deep architecture models do not perform well under high levels and ever-changing magnitudes of uncertainty in real-world environments.

SUMMARY

According to one illustrative embodiment, a computer-implemented method for simulating uncertainty in an artificial neural network is provided. A computer simulates aleatoric uncertainty to measure what the artificial neural network does not understand from sensor data received from an object operating in a real-world environment by adding random values to edge weights between nodes in the artificial neural network during backpropagation of output data of the artificial neural network and measuring impact on the output data by the added random values to the edge weights between the nodes. The computer simulates epistemic uncertainty to measure what the artificial neural network does not know by dropping out a selected node from each respective layer of the artificial neural network during forward propagation of the sensor data and measuring impact of dropped out nodes on the output data of the artificial neural network. The computer performs an action corresponding to the object sending the sensor data and operating in the real-world environment based on the impact of simulating the aleatoric uncertainty and the epistemic uncertainty. According to other illustrative embodiments, a computer system and computer program product for simulating uncertainty in an artificial neural network are provided. Thus, illustrative embodiments are able to perform during data and sensor uncertainty across many various types of platforms, such as autonomous vehicles, robotic assistants, drones, and the like, which operate in real-world environments. As a result, illustrative embodiments are able to increase performance of these various types of platforms while performing during data and sensor uncertainty in their respective environments.

According to yet another illustrative embodiment, a computer-implemented method for simulating uncertainty in an artificial neural network is provided. A computer simulates aleatoric uncertainty to measure what the artificial neural network does not understand from sensor data received from an object operating in a real-world environment by adding random values to edge weights between nodes in the artificial neural network during backpropagation of output data of the artificial neural network and measuring impact on the output data by the added random values to the edge weights between the nodes. The computer performs an action corresponding to the object sending the sensor data and operating in the real-world environment based on the impact of simulating the aleatoric uncertainty.

According to yet another illustrative embodiment, a computer-implemented method for simulating uncertainty in an artificial neural network is provided. A computer simulates epistemic uncertainty to measure what the artificial neural network does not know by dropping out a selected node from each layer of the artificial neural network during forward propagation of data corresponding to an object operating in a real-world environment and measuring impact of dropped out nodes on an output of the artificial neural network. The computer performs an action corresponding to the object operating in the real-world environment based on the impact of simulating the epistemic uncertainty.

These alternative illustrative embodiments decrease computer resource usage by only simulating either aleatoric uncertainty or epistemic uncertainty. In addition, both sensor uncertainty and data uncertainty may not exist in a real-world application. As a result, these alternative illustrative embodiments are better suited for performing in circumstances when only sensor uncertainty or only data uncertainty exists.

DETAILED DESCRIPTION

With reference now to the figures, and in particular, with reference toFIG.1andFIG.2, diagrams of data processing environments are provided in which illustrative embodiments may be implemented. It should be appreciated thatFIG.1andFIG.2are only meant as examples and are not intended to assert or imply any limitation with regard to the environments in which different embodiments may be implemented. Many modifications to the depicted environments may be made.

FIG.1depicts a pictorial representation of a network of data processing systems in which illustrative embodiments may be implemented. Network data processing system100is a network of computers, data processing systems, and other devices in which the illustrative embodiments may be implemented. Network data processing system100contains network102, which is the medium used to provide communications links between the computers, data processing systems, and other devices connected together within network data processing system100. Network102may include connections, such as, for example, wire communication links, wireless communication links, and fiber optic cables.

In the depicted example, server104and server106connect to network102, along with storage108. Server104and server106may be, for example, server computers with high-speed connections to network102. In addition, server104and server106may provide a set of services to clients110,112, and114. For example, server104and server106may simulate aleatoric and epistemic uncertainty in an artificial neural network to produce accurate outputs during uncertainty in real-world environments where clients110,112, and114operate. Further, it should be noted that server104and server106may each represent a cluster of servers in a datacenter. Alternatively, server104and server106may be servers in a cloud environment. Further, server104and server106may provide other information, such as applications and programs, to clients110,112, and114.

Client110, client112, and client114also connect to network102. Clients110,112, and114are clients of server104and/or server106. In this example, clients110,112, and114are illustrated as a vehicle, an aircraft, and a robot, respectively, with wireless and wire communication links to network102. However, it should be noted that clients110,112, and114are meant as examples only. In other words, clients110,112, and114may include other types of devices or machines, such as, for example, watercraft, computers, smart phones, smart watches, smart televisions, smart appliances, gaming devices, kiosks, and the like, with wire or wireless communication links to network102.

Furthermore, client110, client112, and client114include sensor116, sensor118, and sensor120, respectively. Sensor116, sensor118, and sensor120each represent a set of one or more sensors. The set of sensors may include, for example, imaging sensors, such as still and video cameras, sound capturing sensors, such as microphones, geolocation sensors, such as global positioning system (GPS) transceivers, light detection and ranging (LiDAR) sensors, radar sensors, and the like.

Clients110,112, and114transmit sensor data obtained from sensors116,118, and120regarding their respective real-world operating environments to server104and/or server106for evaluation of unknown events or circumstances occurring within their respective environments. Server104and/or server106input the sensor data into a trained artificial neural network, which simulates the aleatoric and epistemic uncertainties, to produce an accurate output. Based on the output generated by the trained artificial neural network, server104and/or server106may perform an action based on the output being outside of normal ranges for a particular environment. An action may be for server104to send an alert to an operator of client110. Another action may be for server106to automatically take control of client112.

Storage108is a network storage device capable of storing any type of data in a structured format or an unstructured format. In addition, storage108may represent a plurality of network storage devices. Further, storage108may store identifiers for a plurality of clients; artificial neural networks; probability density functions; and the like. Furthermore, storage unit108may store other types of data, such as authentication or credential data that may include user names, passwords, and biometric data associated with system users and administrators, for example.

In addition, it should be noted that network data processing system100may include any number of additional servers, clients, storage devices, and other devices not shown. Program code located in network data processing system100may be stored on a computer readable storage medium and downloaded to a computer or other data processing device for use. For example, program code may be stored on a computer readable storage medium on communications server104and downloaded to client110over network102for use on client110.

In the depicted example, network data processing system100may be implemented as a number of different types of communication networks, such as, for example, an internet, an intranet, a local area network (LAN), and a wide area network (WAN).FIG.1is intended as an example only, and not as an architectural limitation for the different illustrative embodiments.

With reference now toFIG.2, a diagram of a data processing system is depicted in accordance with an illustrative embodiment. Data processing system200is an example of a computer, such as server104inFIG.1, in which computer readable program code or instructions implementing processes of illustrative embodiments may be located. Alternatively, data processing system200may be implemented in a device or machine, such as, for example, client110inFIG.1. In this illustrative example, data processing system200includes communications fabric202, which provides communications between processor unit204, memory206, persistent storage208, communications unit210, input/output (I/O) unit212, and display214.

Processor unit204serves to execute instructions for software applications and programs that may be loaded into memory206. Processor unit204may be a set of one or more hardware processor devices or may be a multi-processor core, depending on the particular implementation. Further, processor unit204may include a graphics processing unit.

Memory206and persistent storage208are examples of storage devices216. A computer readable storage device is any piece of hardware that is capable of storing information, such as, for example, without limitation, data, computer readable program code in functional form, and/or other suitable information either on a transient basis and/or a persistent basis. Further, a computer readable storage device excludes a propagation medium. Memory206, in these examples, may be, for example, a random-access memory, or any other suitable volatile or non-volatile storage device. Persistent storage208may take various forms, depending on the particular implementation.

In this example, persistent storage208stores uncertainty manager218. However, it should be noted that even though uncertainty manager218is illustrated as residing in persistent storage208, in an alternative illustrative embodiment uncertainty manager218may be a separate component of data processing system200. For example, uncertainty manager218may be a hardware component coupled to communication fabric202or a combination of hardware and software components. In another alternative illustrative embodiment, a first portion of uncertainty manager218may be located in data processing system200and a second portion of uncertainty manager218may be located in a second data processing system, such as server106or client112inFIG.1. In yet another alternative illustrative embodiment, uncertainty manager218may be located in client devices instead of, or in addition, to data processing system200.

Uncertainty manager218controls the process of simulating aleatoric and epistemic uncertainty in artificial neural network220to produce an output with increased accuracy during uncertainty in a real-world environment. Uncertainty describes a situation involving ambiguous and/or unknown information. In other words, uncertainty is the lack of certainty or a state of limited knowledge where it is impossible to exactly describe the current state, a future outcome, or more than one possible outcome. Uncertainty quantification is the quantitative characterization and reduction of uncertainties in both computational and real-world applications to determine how likely certain outcomes are if some aspects of the system are not exactly known. Measurement of uncertainty may include a set of possible states or outcomes where probabilities are assigned to each possible state or outcome. This also may include the application of a probability density function to continuous variables.

Uncertainty may be classified into two categories. One category is aleatoric uncertainty and the other category is epistemic uncertainty. Aleatoric uncertainty is related to statistical uncertainty and is representative of unknowns that differ each time the same process or experiment is run. Aleatoric uncertainty measures what cannot be understood from the data, but can be explained by unlimited sensing. For example, occlusions in a sensor image may occur so a model, such as artificial neural network220, does not have all the needed information from the sensors. For example, an autonomous vehicle may need to know the distance to a tollbooth, but the autonomous vehicle's imaging sensor cannot detect the tollbooth because a semi-tractor trailer is in front of the vehicle occluding the tollbooth from the imaging sensor. As a result, what the model does not understand from the data needs to be measured and unlimited sensing needs to be emulated.

Epistemic uncertainty is related to systemic errors or measuring inaccuracy. Epistemic uncertainty measures what a model does not know, but can be explained by unlimited data. For example, a model may not know certain data, such as drag (e.g., air resistance) on an object falling to earth at 32 feet per second squared. As another example, a model may receive inaccurate data from a sensor because the sensor is not functioning properly due to malfunction or interference. As a result, what the model does not know from the data needs to be measured and unlimited data needs to be emulated.

In real-life applications, both kinds of uncertainty are typically present. In other words, typically in real-life applications both data uncertainty and sensor uncertainty exist. Uncertainty quantification works toward reducing epistemic uncertainties to aleatoric uncertainties. The quantification of aleatoric uncertainties can be relatively straightforward to perform depending on the application. Techniques such as Monte Carlo methods are frequently used. Monte Carlo methods are computational algorithms that randomly select a data subset from a set of data. Monte Carlo methods are useful for modeling phenomena with significant uncertainty in inputs. In other words, the objective of Monte Carlo sampling is to better understand a system through random sampling of data. Illustrative embodiments may utilize Monte Carlo methods to generate random draws from probability distributions. In this case, Monte Carlo sampling provides data for simulating the epistemic uncertainty.

A probability distribution provides probabilities of occurrence of different possible outcomes in a process or experiment. In other words, a probability distribution is a description of a random phenomenon in terms of the probabilities of events. A probability distribution is defined in terms of the sample space, which is the set of all possible outcomes of the random phenomenon being observed. A continuous probability distribution is typically described by probability density functions.

A probability density function of a continuous random variable is a function whose value at any given sample or point in the sample space (i.e., set of possible values taken by the continuous random variable) can be interpreted as providing a relative likelihood that the value of the random variable would equal that sample. In other words, while the absolute likelihood for a continuous random variable to take on any particular value is zero (0) since there are an infinite set of possible values to begin with, the value of the probability density function at two different samples can be used to infer, in any particular draw of the random variable, how much more likely it is that the random variable would equal one sample compared to the other sample. Thus, the probability density function is used to specify the probability of the random variable falling within a particular range of values instead of taking on any one value.

Uncertainty manager218simulates aleatoric uncertainty to measure what artificial neural network220cannot understand from the sensor data by emulating unlimited sensing. Uncertainty manager218emulates unlimited sensing by adding random values to edge weights between nodes in the artificial neural network220during backpropagation of output data of artificial neural network220and measuring the impact on the output data by the added random edge weight values between the nodes. In other words, uncertainty manager218makes weight adjustments to edges between nodes to compensate for lack of sensor data.

Uncertainty manager218simulates epistemic uncertainty to measure what artificial neural network220does not know by emulating unlimited data. Uncertainty manager218emulates unlimited data by dropping out random nodes from each layer in artificial neural network220during forward propagation of the sensor data and measuring the impact of the dropped out random nodes on outputs of artificial neural network220. In other words, uncertainty manager218takes out nodes creating “internal lesions” in artificial neural network220to compensate for lack of data. Uncertainty manager218selects nodes to be randomly dropped out from each layer in artificial neural network220by using input sensor data, such as a set of one or more sounds or noises and/or a set of one or more images from a set of one or more sensors, corresponding to a real-world environment for Monte Carlo dropout sampling. For each respective layer of artificial neural network220, uncertainty manager218identifies a probability density function in probability density functions236that corresponds to a particular layer and selects a node to be dropped out in that particular layer based on applying an intensity level of the sensor data (e.g., sound) input into the identified probability density function.

Probability density functions236include a plurality of different types of probability density functions. In addition, each respective hidden layer in artificial neural network220corresponds to a different type of probability density function in probability density functions236. The different types of probability density functions in probability density functions236may include, for example, a Wrapped Cauchy probability density function (PDF), a Weibull_max PDF, a Weibull_min PDF, a Wald PDF, a Venmises PDF, a TukeyLambda PDF, a T PDF, a Semicircular PDF, a Rayleigh PDF, an Rdist PDF, a Pearson3 PDF, a Mielke PDF, a Maxwell PDF, a Logistic PDF, a Laplace PDF, a Hyesecant PDF, a Gamma PDF, a Gennorm PDF, a Foldnorm PDF, a Fisk PDF, a Cosine PDF, a Chi PDF, an Arcsine PDF, an Anglit PDF, an Alpha PDF, and the like.

Uncertainty manager218generates an output of artificial neural network220based on the simulated aleatoric and epistemic uncertainties. Uncertainty manager218utilizes the output of artificial neural network220to determine whether to perform an action based on the output, which takes into account the uncertainty in the real-world environment. The action performed by uncertainty manager218may be, for example, issuing an alert to an operator or user of an object, such as a device or machine, which is sending the sensor data as input to artificial neural network220and is operating in the real-world environment. Alternatively, the action performed by uncertainty manager218may be to take control of the device or machine operating in the real-world environment to manage, control, mitigate, or eliminate effects of the uncertainty and, therefore, increase performance of the device or machine in the real-world environment. The device or machine may be, for example, an autonomous or semi-autonomous vehicle, such as a car, truck, van, bus, train, and the like, an autonomous or semi-autonomous aircraft, such as a plane, jet, helicopter, drone, and the like, an autonomous or semi-autonomous watercraft, such as a boat, ship, submarine, and the like, an autonomous or semi-autonomous robotic assistant, such as an industrial robotic assistant, a surgical robotic assistant, a military robotic assistant, a household robotic assistant, and the like.

Artificial neural network220is a data processing algorithm. In an alternative illustrative embodiment, artificial neural network220is a hardware processing device. Artificial neural network220may represent any type of artificial neural network, such as, for example, a residual neural network, a convolutional neural network, a recurrent neural network, a stochastic neural network, and the like.

Artificial neural network220is organized into a plurality of layers, such as layers222. Layers222include input layer224, hidden layers226, and output layer228. Each layer in layers222consists of a number of interconnected nodes, such as nodes230, which contain an activation function. An activation function of a node defines the output of that node given the input. Data patterns are presented to artificial neural network220via input layer224, which communicates to one or more hidden layers in hidden layers226where the actual data processing is performed via weighted edges or connections, such as edges232that include weights234, between nodes230. Hidden layers226are internal representations of the input data. Hidden layers226then link to output layer228where an answer is output.

Artificial neural network220modifies weights234of edges232according to the input data patterns that artificial neural network220is presented with. Artificial neural network220learns via a supervised process that occurs with each epoch or cycle (i.e., each time uncertainty manager218runs artificial neural network220with a new input data pattern) through a forward activation flow of outputs, and the backpropagation of weight adjustments. In other words, this is a two-step process where uncertainty manager218feeds the data inputs into artificial neural network220via forward propagation and multiplied with initially random weights before they are transformed via an activation function. Once the forward propagation is complete, the backpropagation step measures the error from the final output to the expected output by calculating the derivatives of weights234generating the error and adjusts them. In other words, backpropagation adjusts the weights by calculating the gradient of a loss function. Once the weights are adjusted, artificial neural network220repeats the process of the forward propagation and backpropagation steps to minimize the error rate until convergence.

More simply, when artificial neural network220is initially presented with a data pattern, artificial neural network220makes a random guess as to what it might be. Artificial neural network220then determines how far its answer was from the actual one and makes an appropriate adjustment to weights234of edges232. Within each hidden layer node is an activation function, such as a sigmoid function, linear function, or rectified linear unit, which polarizes network activity and helps it to stabilize. Once artificial neural network220is trained, uncertainty manager218utilizes artificial neural network220as an analytical tool to generate simulated aleatoric uncertainty238and simulated epistemic uncertainty240. Uncertainty manager218utilizes Monte Carlo method242to generate simulated epistemic uncertainty240. The output of a forward propagation run is then the predicted model for the data, which replaces previous values.

Object244represents the device or machine operating in real-world environment246. Object244is connected to data processing system200via a network, such as network102inFIG.1. Object244may be, for example, client114inFIG.1, which is illustrated as a robotic device. In this case, real-world environment246may be an industrial production line, for example. Object244includes at least one sensor, such as sensor120inFIG.1, which generates sensor data248. Sensor data248is information corresponding to real-world environment246.

In addition, object244sends sensor data248as input to artificial neural network220. Sensor data248may be, for example, sound or audio data picked up by a sensor, such as a microphone, connected to object244, which is operating in real-world environment246. Sensor data248also includes intensity level250. Intensity level250represents a level of intensity corresponding to sensor data248. For example, a sound intensity level or acoustic intensity level is the level of the intensity of a sound relative to a reference value.

Based on the output of artificial neural network220, uncertainty manager218performs action252in response to the output being outside normal or expected values for real-world environment246. For example, uncertainty manager218may send an alert to an operator or user of object244. Alternatively, uncertainty manager218may take over operational control of object244to, for example, prevent or reduce damage to object244or other objects or humans in real-world environment246, increase performance of object244, decrease a level of risk to object244or humans in real-world environment246, warn humans of an unknow event in real-world environment246, and the like.

Program code254is located in a functional form on computer readable media256that is selectively removable and may be loaded onto or transferred to data processing system200for running by processor unit204. Program code254and computer readable media256form computer program product258. In one example, computer readable media256may be computer readable storage media260or computer readable signal media262. Computer readable storage media260may include, for example, an optical or magnetic disc that is inserted or placed into a drive or other device that is part of persistent storage208for transfer onto a storage device, such as a hard drive, that is part of persistent storage208. Computer readable storage media260also may take the form of a persistent storage, such as a hard drive, a thumb drive, or a flash memory that is connected to data processing system200. In some instances, computer readable storage media260may not be removable from data processing system200.

Alternatively, program code254may be transferred to data processing system200using computer readable signal media262. Computer readable signal media262may be, for example, a propagated data signal containing program code254. For example, computer readable signal media262may be an electro-magnetic signal, an optical signal, and/or any other suitable type of signal. These signals may be transmitted over communication links, such as wireless communication links, an optical fiber cable, a coaxial cable, a wire, and/or any other suitable type of communications link. In other words, the communications link and/or the connection may be physical or wireless in the illustrative examples. The computer readable media also may take the form of non-tangible media, such as communication links or wireless transmissions containing the program code.

In some illustrative embodiments, program code254may be downloaded over a network to persistent storage208from another device or data processing system through computer readable signal media262for use within data processing system200. For instance, program code stored in a computer readable storage media in a data processing system may be downloaded over a network from the data processing system to data processing system200. The data processing system providing program code254may be a server computer, a client computer, or some other device capable of storing and transmitting program code254.

As another example, a computer readable storage device in data processing system200is any hardware apparatus that may store data. Memory206, persistent storage208, and computer readable storage media260are examples of physical storage devices in a tangible form.

The fields of data science and artificial intelligence are entering into a golden age ushered in by deep learning. Deep learning neural networks are just beginning to leverage techniques of reinforcement to learn from their environment. However, current models are not able to perform well in real-world environments due to errors in measurement or accounting for what current models don't know. Illustrative embodiments provide life-like training environments that resemble real-world application. Using illustrative embodiments, artificial neural networks are capable of reasoning under high levels and ever-changing magnitudes of uncertainty and humans will have a better idea of how these artificial neural networks will perform during uncertainty.

Illustrative embodiments provide a way to simulate aleatoric and epistemic uncertainty through deep learning plasticity. Further, illustrative embodiments determine the type of unknown event or disruption in the real-world environment using sensor feedback, such as sound data, obtained from the real-world environment. Thus, illustrative embodiments are able to perform during data and sensor uncertainty across many various types of platforms, such as autonomous vehicles, robotic assistants, drones, and the like. In fact, each training epoch of artificial neural networks using illustrative embodiments will uncover new results or outcomes due to uncertainty.

With reference now toFIG.3, a flowchart illustrating a process for performing an action based on output of a trained artificial neural network simulating uncertainty is shown in accordance with an illustrative embodiment. The process shown inFIG.3may be implemented in a computer, such as, for example, server104inFIG.1or data processing system200inFIG.2. Alternatively, the process shown inFIG.3may be implemented in a device or machine, such as client110inFIG.1, that includes some type of data processing system or computer.

The process begins when the computer simulates aleatoric uncertainty to measure what an artificial neural network does not understand from sensor data received from an object operating in a real-world environment by adding random values to edge weights between nodes in the artificial neural network during backpropagation of output data of the artificial neural network and measuring impact on the output data by the added random values to the edge weights between the nodes (step302). The computer selects a node to be randomly dropped from each layer of the artificial neural network by using the sensor data corresponding to the real-world environment for Monte Carlo dropout sampling and, for each respective layer of the artificial neural network, the computer identifies a probability density function corresponding to a particular layer and selects the node to be randomly dropped from that particular layer based on applying an intensity level of the sensor data to the probability density function corresponding to that particular layer (step304).

The computer simulates epistemic uncertainty to measure what the artificial neural network does not know by dropping out the selected node from each respective layer of the artificial neural network during forward propagation of the sensor data and measuring impact of dropped out nodes on the output data of the artificial neural network (step306). The computer generates an output of the artificial neural network based on simulating the aleatoric uncertainty and the epistemic uncertainty (step308). The computer performs an action corresponding to the object sending the sensor data and operating in the real-world environment based on the output of the artificial neural network simulating the aleatoric uncertainty and the epistemic uncertainty (step310). Thereafter, the process terminates.

However, it should be noted that both data uncertainty and sensor uncertainty may not exist in all real-life applications. As a result, one alternative illustrative embodiment may only simulate aleatoric uncertainty to measure what an artificial neural network does not understand from sensor data received from an object operating in a real-world environment when only sensor uncertainty exists. Another alternative illustrative embodiment may only simulate epistemic uncertainty to measure what an artificial neural network does not know from data corresponding to an object operating in a real-world environment when only data uncertainty exists. Further, by alternative illustrative embodiments only simulating either aleatoric uncertainty or epistemic uncertainty, these alternative illustrative embodiments conserve processor, memory, and network resources as compared to simulating both aleatoric and epistemic uncertainties.

With reference now toFIG.4, a flowchart illustrating a process for simulating aleatoric uncertainty is shown in accordance with an illustrative embodiment. The process shown inFIG.4may be implemented in a computer, such as, for example, server106inFIG.1or data processing system200inFIG.2. Alternatively, the process shown inFIG.4may be implemented in a device or machine, such as client112inFIG.1, that includes some type of data processing system or computer.

An artificial neural network learns by backpropagating a loss function or error, which is the difference between a target output and an obtained output, through the artificial neural network. Illustrative embodiments update the weights on each of the edges between nodes in the artificial neural network based on the amount of contribution that each node had on the error. Through simulating aleatoric uncertainty, illustrative embodiments add a weight adjustment (e.g., epsilon) to the updated weights using the following formula:

wnew[L]⁢(l)=wold[L]⁢(l)+α⁢dJd+ϵ⁢dJd.
In other words, illustrative embodiments utilize epsilon as a multiplier for the error. Illustrative embodiments retrieve the value for epsilon from each different probability density function corresponding to each respective hidden layer of the artificial neural network. Illustrative embodiments may utilize a plurality of different types of probability density functions, such as, for example, thirty different types corresponding to thirty different hidden layers of the artificial neural network.

The process begins when the computer runs an artificial neural network that includes a plurality of hidden layers using labeled sensor data samples corresponding to a real-world environment (step402). The computer utilizes an obtained output of the artificial neural network to determine model error based on a delta between a target output and the obtained output (step404). The computer inputs the obtained output of the artificial neural network into each different type of probability density function corresponding to each respective hidden layer in the plurality of hidden layers to generate edge weight adjustments between nodes based on probabilities of occurrence of the obtained output in the real-world environment (step406).

The computer backpropagates the model error through the artificial neural network to update edge weights between nodes in the plurality of hidden layers based on a level of contribution by each respective node to the model error (step408). The computer adds the edge weight adjustments to the updated edge weights between nodes in each respective hidden layer in the plurality of hidden layers to simulate aleatoric uncertainty (step410). Thereafter, the process terminates.

With reference now toFIG.5, a flowchart illustrating a process for simulating epistemic uncertainty is shown in accordance with an illustrative embodiment. The process shown inFIG.5may be implemented in a computer, such as, for example, server104inFIG.1or data processing system200inFIG.2. Alternatively, the process shown inFIG.5may be implemented in a device or machine, such as client114inFIG.1, that includes some type of data processing system or computer.

Illustrative embodiments may utilize, for example, sound-based dropout sampling to model epistemic uncertainty. Illustrative embodiments may utilize a sound print obtained from a real-world environment to set the dropout learning rate for each hidden layer in the artificial neural network. For example, illustrative embodiments may utilize a sound analyzer to measure the noise from a crowd of people in the real-world environment to generate multiple tags corresponding to the noise. Illustrative embodiments may input the tags into a word2vector neural network to generate word vectors and then enter the word vectors into a sound intensity classifier, along with a phonograph of the crowd noise. The resultant noise intensity score generated by the sound intensity classifier provides the magnitude of sampling during Monte Carlo dropout sampling. The magnitude of sampling is first related to the number of nodes to be dropped out of the artificial neural network and then to how many samples to take. As a result, during training, the artificial neural network will emulate a loss of nodes that simulate what the artificial neural network does not know. The output of the trained artificial neural network will contain epistemic error evaluation.

The process begins when the computer receives sensor data from an object operating in a real-world environment (step502). The computer determines an intensity level of the sensor data (step504). The computer determines whether the intensity level of the sensor data is greater than an intensity level threshold level indicating occurrence of an unknown event (step506).

If the computer determines that the intensity level of the sensor data is less than the intensity level threshold level indicating occurrence of an unknown event, no output of step506, then the process returns to step502where the computer continues to receive sensor data. If the computer determines that the intensity level of the sensor data is greater than or equal to the intensity level threshold level indicating occurrence of an unknown event, yes output of step506, then the computer inputs the sensor data into an artificial neural network that includes a plurality of hidden layers, each hidden layer including a plurality of nodes (step508).

Afterward, the computer performs Monte Carlo dropout sampling on the sensor data to determine which node in each respective hidden layer in the plurality of hidden layers is to be randomly dropped out to simulate the unknown event (step510). The computer selects a hidden layer in the plurality of hidden layers (step512). The computer identifies a probability density function corresponding to the selected hidden layer in the plurality of hidden layers that models an output of the artificial neural network (step514).

The computer selects a node within the selected hidden layer to be randomly dropped out based on applying the intensity level of the sensor data to the identified probability density function (step516). The computer drops out the selected node within the selected hidden layer to simulate epistemic uncertainty associated with the unknown event (step518). Subsequently, the computer makes a determination as to whether another hidden layer exists in the plurality of hidden layers (step520).

If the computer determines that another hidden layer does exist in the plurality of hidden layers, yes output of step520, then the process returns to step512where the computer selects another hidden layer. If the computer determines that another hidden layer does not exist in the plurality of hidden layers, no output of step520, then the process terminates thereafter.

With reference now toFIG.6, a flowchart illustrating a process for training an artificial neural network is shown in accordance with an illustrative embodiment. The process shown inFIG.6may be implemented in a computer, such as, for example, server104inFIG.1or data processing system200inFIG.2. Alternatively, the process shown inFIG.6may be implemented in a device or machine, such as client114inFIG.1, that includes some type of data processing system or computer.

The process begins when the computer receives training exemplars for an artificial neural network (step602). The computer forward propagates the training exemplars through the artificial neural network (step604). The computer backpropagates an output of the artificial neural network (step606).

In addition, the computer receives sound data from a real-world environment (step608). The computer determines a sampling rate based on the sound data (step610). Afterward, the computer sets a Monte Carlo sampling parameter and a diffusion parameter based on backpropagation and the determined sampling rate (step612).

For each layer of the artificial neural network, the computer identifies a corresponding probability density function (PDF) (step614). Using the diffusion parameter, the computer randomly selects nodes in each layer of the artificial neural network (step616). Using the Monte Carlo sampling parameter, the computer removes an area in each probability density function that was not sampled (step618). The computer calculates a ratio of new area to old area in each probability density function after area removal (step620). The computer adds the ratio to all input edge weights for each node that were randomly selected (step622).

Further, the computer determines training error using a loss function (step624). The computer inputs the training error into each probability density function for data occlusion (step626). The computer calculates an extra error adjustment for edge weights based on each output of each probability density function (step628). The computer adjusts the edge weights between nodes using the extra error adjustment (step630).

The computer continues training the artificial neural network (step632). The process terminates thereafter.