Patent Description:
In recent years, artificial intelligence techniques, such as machine learning, deep learning, etc., have become more prevalent in solving problems and/or performing tasks, which include, but are not limited to image recognition, pattern recognition, autonomous vehicle navigation, protein folding analysis, etc. Such techniques employ different types of hardware resources to accomplish the various tasks.

<CIT> discloses Logic which may determine a physical resource assignment via a neural network logic trained to determine an optimal policy for assignment of the physical resources in source code. Logic may generate training data to train a neural network by generating multiple instances of machine code for one or more source codes in accordance with different policies. Logic may generate different policies by adjusting, combining, mutating, and/or randomly changing a previous policy. Logic may execute and measure and/or statically determine measurements for each instance of a machine code associated with a source code to determine a reward associated with each state in the source code. Logic may apply weights and biases to the training data to approximate a value function. Logic may determine a gradient descent of the approximated value function and may backpropagate the output from the gradient descent to adjust the weights and biases to determine an optimal policy. <NPL> discloses a deep reinforcement learning approach to optimizing the execution cost of computation graphs in a static compiler. A neural network policy is combined with a genetic algorithm, the Biased Random-Key Genetic Algorithm (BRKGA). The policy is trained to predict, given an input graph to be optimized, the node-level probability distributions for sampling mutations and crossovers in BRKGA. The approach uses the policy's ability to transfer to new graphs to significantly improve the solution quality of the genetic algorithm for the same objective evaluation budget.

The figures are not to scale. In general, the same reference numbers will be used throughout the drawing(s) and accompanying written description to refer to the same or like parts.

Efforts to map a deep learning workload to resources (e.g., hardware) is a challenging and time-consuming task. Typically, personnel chartered with the responsibility of implementing a deep learning (DL) workload must analyze a neural network algorithm, break the neural network down into individual layers, apply configuration characteristics to each layer, and observe results (e.g., results from hardware resources, results from hardware simulators, etc.) of applying the characteristics to identify whether particular configuration characteristics should be used. Additionally, the personnel must select particular resources to accomplish computational tasks for each layer of a neural network. Resources include, but are not limited to computer processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), dedicated accelerators (e.g., matrix accelerators, matrix multiplication accelerators), inference computation engines (ICEs), systems on chip (SOCs), etc. Additionally, resources include different types of storage devices, such as particular types of memory (e.g., dynamic randomaccess memory (DRAM), scratchpad memory, last level cache (LLC), etc.).

While some known compilers provide a degree of automation to select resources for individual neural network layers, such compilers employ rule-based approaches and/or heuristics to make such selections. However, rule-based approaches fail to scale to deep learning neural networks as complexity increases. Indeed, mere application of heuristics is shortsighted in view of the vast number of permutations of layers, the vast number of corresponding resources and the vast number of corresponding memory device assignments for the respective layers. Such approaches are further constrained by the requirement of a human to evaluate trial-and-error results, and apply manual fine-tuning. Furthermore, discretionary errors are introduced when humans apply judgement to decisions regarding resource allocation on a layer-by-layer basis. Even where a human develops a particular skill set in making such layer-to-resource mappings, such skill sets are generally not transferrable to other personnel so that application can be applied in a predictable manner.

Examples disclosed herein employ reinforcement learning (RL) to compile neural networks in a manner that improves (e.g., optimizes) resource mappings on a layer-by-layer basis in view of particular improvement (e.g., optimization) objectives. In some examples disclosed herein, a latency constraint is identified as an improvement objective to improve (e.g., maximize). In response to receiving and/or otherwise retrieving a neural network input, resource mappings are assigned for each layer of the neural network to form an output mapping framework that can be applied to a compiler. As such, the output mapping framework generated by examples disclosed herein overrides any default mapping strategy(ies) employed by the compiler (e.g., rule-based mappings, heuristics, etc.) to cause improved (e.g., optimized) performance of the neural network when measured against the particular improvement (e.g., optimization) objectives. However, examples disclosed herein are not limited to a single improvement (e.g., optimization) objective, but identify resource mappings on a layer-by-layer basis in connection with any number of improvement objectives (e.g., lower latency, faster throughput, lower power consumption, etc.).

Artificial intelligence (AI), including machine learning (ML), deep learning (DL), neural networks (NNs), deep NNs, convolutional NNs (CNNs), and/or other artificial machine-driven logic, enables machines (e.g., computers, logic circuits, etc.) to use a model to process input data to generate an output based on patterns and/or associations previously learned by the model via a training process. For instance, the model may be trained with data to recognize patterns and/or associations and follow such patterns and/or associations when processing input data such that other input(s) result in output(s) consistent with the recognized patterns and/or associations.

Many different types of machine learning models and/or machine learning architectures exist. In examples disclosed herein, a reinforcement model (reinforcement learning) is used. Using a reinforcement model enables behaviors (e.g., arbitrary behaviors) to play-out scenarios such that an agent can identify how to act/perform in an effort to improve (e.g., maximize) a reward (or reduce (e.g., minimize) a punishment). As used herein, an agent is a representation (e.g., an executable) of the influence of making a change, such as a network directive that, when executed, causes particular hardware performance activity and/or a change in state. In general, machine learning models/architectures that are suitable to use in the example approaches disclosed herein will be reinforcement learning techniques. However, other types of machine learning models/techniques could additionally or alternatively be used.

In general, implementing a ML/AI/DL system involves two phases, a learning/training phase and an inference phase. In the learning/training phase, a training algorithm is used to train a model to operate in accordance with patterns and/or associations based on, for example, training data. In general, the model includes internal parameters that guide how input data is transformed into output data, such as through a series of nodes and connections within the model to transform input data into output data. Additionally, in some examples hyperparameters are used as part of the training process to control how the learning is performed (e.g., a learning rate, a number of layers to be used in the machine learning model, a discount factor, etc.). Hyperparameters are defined to be training parameters that are determined, for example, prior to initiating the training process.

Different types of training may be performed based on the type of NIL/AI/DL model/technique and/or the expected output. For example, supervised training uses inputs and corresponding expected (e.g., labeled) outputs to select parameters (e.g., by iterating over combinations of select parameters) for the ML/AI/DL model that reduce model error. Generally speaking, supervised learning/training is particularly useful when predicting values based on labeled data. As used herein, labelling refers to an expected output of the machine learning model (e.g., a classification, an expected output value, etc.) Alternatively, unsupervised training/learning (e.g., used in deep learning, a subset of machine learning, etc.) involves inferring patterns from inputs to select parameters for the ML/AI/DL model (e.g., without the benefit of expected (e.g., labeled) outputs). Generally speaking, unsupervised learning is particularly useful when attempting to identify relationships in unlabeled data.

In examples disclosed herein, ML/AI/NN/DL models are trained using reinforcement learning. However, any other training algorithm/technique may additionally or alternatively be used. In examples disclosed herein, training is performed until convergence, which is aided through the use of neural networks. Training is performed using hyperparameters that control how the learning is performed (e.g., a learning rate, a number of layers to be used in the machine learning model, etc.). In examples disclosed herein, hyperparameters that control the discount factor enable different degrees of learning experimentation and attempts to "try. " Such hyperparameters are selected by, for example, empirical observation, time constraints, etc. In some examples re-training may be performed.

For some ML/AI/NN/DL approaches, training is performed using training data. In examples disclosed herein, the training data originates from a code corpus of code samples deemed to be particularly useful and error free (e.g., industry standard code). Because supervised training may be used, the training data is labeled. However, labelled data may also be useful in reinforcement learning to provide additional states and/or corresponding actions of particular code functions.

In some examples, once training is complete, the model is deployed for use as an executable construct that processes an input and provides an output based on the network of nodes and connections defined in the model. The model is stored at local storage devices (e.g., databases) and/or network-accessible storage devices (e.g., cloud-based storage services).

Once trained, the deployed model may be operated in an inference phase to process data. In the inference phase, data to be analyzed (e.g., live data) is input to the model, and the model executes to create an output. This inference phase can be thought of as the AI (e.g., an ML model) "thinking" to generate the output based on what it learned from the training (e.g., by executing the model to apply the learned patterns and/or associations to the live data). In some examples, input data undergoes pre-processing before being used as an input to the machine learning model. Moreover, in some examples, the output data may undergo post-processing after it is generated by the model (e.g., ML model) to transform the output into a useful result (e.g., a display of data, an instruction to be executed by a machine, etc.).

In some examples, output of the deployed model may be captured and provided as feedback. By analyzing the feedback, an accuracy of the deployed model can be determined. If the feedback indicates that the accuracy of the deployed model does not satisfy (e.g., is less than) a threshold or fails to satisfy some other criterion, training of an updated model can be triggered using the feedback and an updated training data set, hyperparameters, etc., to generate an updated, deployed model.

<FIG> illustrates a portion of an example mapping process <NUM>. In the illustrated example of <FIG>, the mapping process <NUM> includes a neural network <NUM> having any number of layers, in which a layer of interest <NUM> is shown as a current layer being analyzed (layer "t"). The illustrated example of <FIG> also includes an example reinforcement learning (RL) agent <NUM>, which includes an example state definer <NUM>, an example reward determiner <NUM>, and an example action determiner <NUM>. The illustrated example of <FIG> also includes example resources <NUM>. The example resources <NUM> of <FIG> include a circuit board <NUM> having any number and/or type of resources thereon, such as an example CPU, an example GPU, example logic analyzers, example accelerators, etc. In some examples, a platform and/or other communicatively connected resources are scanned to perform an audit of available hardware and/or hardware simulators that can be used to execute the neural network and/or layers therein. While the illustrated example of <FIG> includes a physical circuit board <NUM> (e.g., a printed circuit board (PCB)), examples disclosed herein are not limited thereto.

In the illustrated example of <FIG>, the circuit board <NUM> is associated with the current layer of the neural network being analyzed (layer "t"), in which the example RL agent <NUM> has determined to employ a first resource <NUM> for layer "t. " However, at a prior iteration of the example RL agent <NUM>, a prior layer of the neural network <NUM> was analyzed (layer "t-<NUM>") in which the example RL agent <NUM> has determined to employ a second resource <NUM> of the example circuit board <NUM>. Additionally, at a subsequent iteration of the example RL agent <NUM>, a subsequent layer of the neural network <NUM> may be analyzed (layer "t+<NUM>") in which the example RL agent <NUM> may be determined to employ a third resource <NUM> of the example circuit board <NUM>. Generally speaking, the illustrated example of <FIG> shows that the example circuit board <NUM> may have any number of different/unique resources thereon, and examples disclosed herein identify which ones of those unique resources best improve (e.g., optimize) respective layers of the neural network <NUM>. The set of resources that best improves one layer may be completely different from the set of resources that improves another layer.

<FIG> illustrates another portion of an example mapping process <NUM>. In the illustrated example of <FIG>, the mapping process <NUM> includes a neural network <NUM> having any number of layers, in which a layer of interest <NUM> is shown as a 3x3 convolution layer (a convolution operator) <NUM> being analyzed. An RL agent, such as the example RL agent <NUM> of <FIG>, propagates (e.g., evaluates, executes, simulates, etc.) through the example neural network <NUM> layer by layer to map memory and computational resources for each layer of interest. As described in further detail below, the mapping is based on previous layer mappings, remaining utilization capabilities of the resources, and cost/reward function results. In the illustrated example of <FIG>, an example input feature map (IFM) <NUM>, which includes tensor (e.g., vector space object) dimensions, is mapped to an ICE block <NUM>. In the illustrated example of <FIG>, the example ICE block <NUM> includes an example matrix accelerator <NUM> (e.g., hardware circuitry) and an example digital signal processor (DSP) <NUM> (e.g., hardware circuitry). More specifically, the example IFM <NUM> is mapped to a scratchpad (SP) memory <NUM> of the example IFM <NUM>. The example mapping process <NUM> of <FIG> also illustrates the convolution operator <NUM> (e.g., hardware circuitry) mapped to the example matrix accelerator <NUM> of the example ICE block <NUM>, and that an example output feature map (OFM) <NUM>, which includes tensor dimensions, is mapped to an example last level cache <NUM>. However, other candidate layers of the example neural network <NUM> may utilize different hardware and/or memory for the example convolution operator <NUM>, the example IFM <NUM>, and/or the example OFM <NUM>.

<FIG> illustrates an example system <NUM> to map workloads (e.g., tasks to be completed by a neural network). In the illustrated example of <FIG>, the system <NUM> includes an example neural network (NN) mapper <NUM> communicatively connected to an example network <NUM> to facilitate communication and/or control with/over an example workload data store <NUM>, example hardware <NUM>, an example hardware simulator <NUM>, and an example compiler <NUM>. In some examples, the system includes both example hardware <NUM> and the example hardware simulator <NUM>, while in some examples the system includes one or the other. In some examples, the hardware <NUM> is implemented by one or more hardware circuits (e.g., the example ICE <NUM>, the example resources <NUM>, the example circuit board <NUM>, etc.). In some examples, the hardware simulator <NUM> is implemented by one or more processors. In some examples, the compiler is implemented by one or more processors. In some examples, the neural network mapper scans the example network <NUM> and/or a platform communicatively connected to the network <NUM> to identify candidate resources that could execute the target neural network (e.g., the example hardware <NUM> and/or the example hardware simulator <NUM>). In some examples, the NN mapper <NUM> is directly connected to one or more of the aforementioned structures of the example system <NUM> without any need for the example network <NUM>. In some examples, the network <NUM> includes an intranet, the Internet, a local area network (LAN), a wide area network (WAN), etc. As described above, the example hardware <NUM> may include any number and/or types of resource, such as physical hardware (e.g., CPUs, GPUs, accelerators, etc.) and/or virtual machines (VMs). In some examples, when physical hardware is unavailable, the example hardware simulator <NUM> simulates hardware on a layer-by-layer basis. In response to the example NN mapper <NUM> evaluating a layer of a NN, the example NN mapper <NUM> generates and/or updates a mapping file. As described in further detail below, the mapping file includes specific resource assignments on a layer-by-layer basis, which is provided to the example compiler <NUM>. While some compilers utilize one or more different techniques to assign resources to NN layers, examples disclosed herein generate improved (e.g., optimized) resource assignments and facilitate compiler override instructions to bypass and/or otherwise prevent such conventional resource assignments from being implemented. Instead, examples disclosed herein employ the example mapping file to control and/or otherwise direct the resource assignment activities of the example compiler <NUM>.

In operation, the example NN mapper <NUM> retrieves and/or otherwise receives a NN model from the example workload data store <NUM>. The example NN model may be in a device agnostic format, such as the Open Neural Network eXchange (ONNX) format to represent deep learning models. <FIG> illustrates an example NN input model <NUM> having information associated with respective layers of the model <NUM>. In the illustrated example of <FIG>, the NN input model <NUM> includes rows corresponding to respective layers <NUM> of the NN input model <NUM>. The example NN input model <NUM> of <FIG> also identifies particular operator types <NUM> (e.g., rectified linear units (ReLUs), Reshape, Softmax, etc.), tensor size information, layer hierarchy information, etc..

Returning to the illustrated example of <FIG>, the NN mapper <NUM> evaluates a neural network on a layer-by-layer basis. To do so, the candidate layers are executed by the example hardware <NUM> (or the example hardware simulator <NUM>) with a different combination of hardware elements (e.g., processing devices, memory devices, circuits, etc.) on a layer by layer basis. For each layer, one combination of hardware elements will exhibit a relatively improved (e.g., optimum) performance metric that is saved as a final resource directive for that particular layer being analyzed. In other words, different resource configurations are compared based on their relative scores, and the best score is deemed "optimum. " When a current layer is finished being analyzed (e.g., based on a threshold number of attempted hardware configurations, based on detection of a convergence indicator, etc.), the example NN mapper <NUM> moves on to analyzing a next layer of the neural network, and so on until each layer has been analyzed as an "optimum" resource configuration has been identified for each layer.

<FIG> is a schematic illustration of the example NN mapper <NUM> of <FIG>. In the illustrated example of <FIG>, the NN mapper <NUM> includes an example NN input detector <NUM>, an example layer selector <NUM>, an example mapping configuration storage <NUM>, and an example layer map generator <NUM>. The aforementioned structures are communicatively connected via an example NN mapper bus <NUM>. The example layer map generator <NUM> includes an example iteration tracker <NUM>, an example constraint definer <NUM>, an example agent generator <NUM>, the example state definer <NUM>, the example action determiner <NUM>, and the example reward determiner <NUM>. The aforementioned structures of the example layer map generator <NUM> are communicatively connected via an example layer map generator bus <NUM>. In some examples, all structures within the illustrated example of <FIG> is communicatively connected via the example NN mapper bus <NUM> and/or the example layer map generator bus <NUM>, without limitation. These structures may be implemented by circuitry.

In some examples, the constraint definer <NUM> implements means for performance characteristic defining. The performance characteristic defining means may be implemented by a processor, such as the processor <NUM> of <FIG> executing instructions, such as the instructions of <FIG> and/or <NUM>. In some examples, the action determiner <NUM> implements means for action applying. The action applying means may be implemented by the processor <NUM> of <FIG> executing instructions, such as the instructions of <FIG> and/or <NUM>. In some examples, the reward determiner <NUM> implements the means for results calculating. The results calculating means may be implemented by the processor <NUM> of <FIG> executing instructions, such as the instructions of <FIG> and/or <NUM>. In some examples, the layer map generator <NUM> implements the means for map generating. The map generating means may be implemented by the processor <NUM> of <FIG> executing instructions, such as the instructions of <FIG> and/or <NUM>. In some examples, the neural network mapper <NUM> implements the means for neural network mapping. The neural network mapping means may be implemented by the processor <NUM> of <FIG> executing instructions, such as the instructions of <FIG> and/or <NUM>. In some examples, the layer selector <NUM> implements the means for layer selection. The layer selection means may be implemented by the processor <NUM> of <FIG> executing instructions, such as the instructions of <FIG> and/or <NUM>. In some examples, the state definer implements the means for state defining. The state defining means may be implemented by the processor <NUM> of <FIG> executing instructions, such as the instructions of <FIG> and/or <NUM>.

In operation, the example NN input detector <NUM> determines whether a NN input model has been received. As described above, examples disclosed herein evaluate a NN model, such as the device agnostic model <NUM> shown in the illustrated example of <FIG>. In response to a NN analysis request, the example layer selector <NUM> selects one of any number of layers associated with the received and/or otherwise retrieved NN model. As described above, each layer of a NN model can utilize particular resources in a manner that best satisfies desired operating characteristics of the NN model. For instance, selecting a first processing circuit (e.g., an element, such as a CPU) for a first layer may exhibit improved performance characteristics over a second processing circuit (e.g., a matrix accelerator). However, selecting that same first processing circuit for a second layer may not necessarily exhibit improved (e.g., optimized) performance characteristics. As such, examples disclosed herein evaluate each layer in view of a particular combination of desired performance characteristics with which to improve (e.g., maximize).

The example iteration tracker <NUM> determines whether the selected layer of interest has been evaluated on a prior occasion and, if not, the example constraint definer <NUM> defines resource constraints to be associated with the selected layer of interest. As used herein, resource constraints, performance characteristics and performance characteristic targets are referred-to interchangeably and are metrics that can be improved when particular resources (e.g., processing resources, memory resources) are utilized with particular ones of layers in a neural network. Resource constraints to be defined include, but are not limited to improved (e.g., maximized) throughput metrics, improved (e.g., maximized) response time (e.g., latency) metrics, reduced (e.g., minimized) power metrics, and/or non-continuous weight values for respective constraints to be applied in one or more reward/cost functions (sometimes referred to herein as a value function). Examples disclosed herein employ actor-critic reinforcement learning (RL) to converge permutations of an action space that, in some examples, is too large for reasonable human processing. As used herein, the action space includes a particular number of combinations of hardware selections and/or memory selections for respective layers in a NN.

The example reward function may, for example, seek to improve (e.g., maximize) a throughput metric in connection with compute, memory and latency constraints. In some examples, the reward function considers mutual interactions between particular actions and mappings of previously analyzed layers of the deep NN. As used herein, the actor-critic RL framework includes a critic (e.g., reward determiner) to evaluate the merits of an action (e.g., how good an action is to take), and updates action-value parameters to seek and/or otherwise suggest by an actor (e.g., action determiner). As used herein, the actor instructs a particular action and updates policy parameters as suggested by the example critic. The actor may, for example, map a layer to a particular memory location (e.g., SRAM, LLC, DDR, etc.), map the layer to a particular processor (e.g., a matrix accelerator, a DSP, a CPU, etc.) and/or identify a particular core of a multi-core system. In some examples, the critic identifies parameters to guide a degree of candidate action exploration rather than merely attempting to execute all possible permutations of an action space, which would be computationally and temporally expensive.

<FIG> illustrates pseudo code of an example actor-critic RL framework <NUM>. Other inputs to the actor-critic RL framework include state values, such as current layer parameters associated with the IFM, the OFM, a kernel size of the layer of interest, an amount of remaining (e.g., available) memory, memory bandwidth capability information and/or mapping information associated with previous layers of the deep NN.

During a first iteration of the actor-critic RL framework for the selected layer of interest, because no prior mapping permutations have been attempted, the example constraint definer <NUM> identifies a heuristic and/or rule-based mapping configuration. For example, the constraint definer <NUM> may utilize a greedy algorithm to make a resource mapping selection that employs the fastest available processing device and a memory device having the lowest relative latency. While some applications of the greedy algorithm may exhibit desired performance characteristics, simple application of the greedy algorithm may cause particular problems. For instance, in the event the memory having the lowest latency is selected, but that memory is also near full capacity, then such a selection causes future bottlenecks and relatively poor results. Nonetheless, applying the heuristic and/or rule-based mapping configuration serves as a helpful starting point for the actor-critic RL framework. In particular, such a starting point allows the generation of a reward calculation with performance data, which allows the RL framework to iterate with alternate mapping permutations that are guided by the performance characteristics. Stated differently, the RL framework benefits from poor mapping choices to guide future mapping configuration permutations to attempt.

The example agent generator <NUM> generates a reinforcement learning agent (e.g., the example RL agent <NUM> of <FIG>) (e.g., a container for the example state definer <NUM>, the example reward determiner <NUM> and the example action determiner <NUM>). The example state definer <NUM> generates a current state (St) representation based on the selected heuristics/rules and prior layer state information, if any, in which t reflects the selected layer of interest. In some examples, the current state is represented in a manner consistent with example Equation <NUM>. <MAT> In the illustrated example of Equation <NUM>, IFM refers to the input feature map (having tensor dimensions), OFM refers to the output feature map (having tensor dimensions), BW refers to a memory bandwidth metric (e.g., in Gigabits per second (GB/s)), MEM refer to a memory capacity metric (e.g., in Megabits (MB)), UTIL refers to a vector of a hardware component utilization for the selected layer of interest (e. g, a <NUM>% utilization of a DSP, a <NUM>% utilization for an accelerator, a <NUM>% utilization for a CPU, etc.), and LATENCY refers to a latency metric for the layer of interest.

The example action determiner <NUM> applies the mapping configuration (based on the heuristics/rules) to the example resources <NUM> so that state data may be generated. As the generated state data results from every permutation of resources applied to the selected layer of interest, relative performance capabilities can be realized when comparing respective instances of the state data. In some examples, results from each iteration of applied resources (either actual physical resources <NUM> or simulated hardware <NUM>) are stored in the example mapping configuration storage <NUM>.

In response to at least one iteration of the example layer map generator <NUM> applying a particular combination of resources, the example state definer <NUM> updates the state metrics of the mapping effort. <FIG> illustrates an example state report <NUM> generated by the example state definer <NUM>. In the illustrated example of <FIG>, the state report <NUM> includes embedded state information associated with the layers <NUM>, actions most recently taken <NUM> and corresponding results <NUM> (e.g., metrics calculated from the reward function). For instance, an example first row <NUM> illustrates an array of integer values indicative of different state details (<NUM>, <NUM>, <NUM>, <NUM>, etc.). A first integer may indicate a first batch (<NUM>), a second integer may indicate a number of input channels (<NUM>), third and fourth integers may indicate an image size (224x224), etc. The example reward determiner <NUM> evaluates the metrics in connection with one or more reward functions. In some examples, the reward determiner <NUM> calculates a results metric that is based on measured performance metrics and corresponding performance characteristic targets. Such results metric data is used on a relative comparison basis to determine which particular resource assignments exhibit the relatively best (e.g., optimized) performance as compared to the other resource assignments. The reward determiner <NUM> generates an iteration decision based on the results metric. In the event the example reward determiner <NUM> determines that additional resource mapping iterations should occur (e.g., the iteration decision indicates at least one more hardware configuration permutation to try), the example action determiner <NUM> updates resource mapping directives and applies those directives to a subsequent mapping configuration to be tested. In some examples, the reward determiner <NUM> identifies an indication of convergence to signal when iterations of the actor-critic RL framework should cease, while in some examples the reward determiner <NUM> conducts a threshold number of iterations.

In response to the action determiner <NUM> determining that no further iterations are needed (e.g., a threshold degree of convergence has been detected by the example action determiner <NUM>), the example layer map generator <NUM> stores the mapping configurations for the selected layer in the example mapping configuration storage <NUM>. The example layer selector <NUM> determines whether additional layers still require analysis and, if so, the example layer selector <NUM> selects the next layer in the example NN.

On the other hand, in the event the example layer selector <NUM> determines that all layers of the NN have been analyzed, the example layer map generator <NUM> generates a mapping file for the NN. <FIG> illustrates an example mapping file <NUM>, sometimes referred to herein as a resource map. In the illustrated example of <FIG>, the mapping file <NUM> includes a first mapping of a particular layer and a corresponding hardware feature <NUM>. The example mapping file <NUM> also includes a mapping of that same layer to a particular inference computation engine (ICE) processor <NUM>, such as one of any number of devices on the ICE (e.g., a particular accelerator, a particular DSP, etc.). The example mapping file <NUM> also includes a mapping of that same layer to a particular memory and/or memory location to which layer tensors are to be mapped <NUM>. While the illustrated example of <FIG> includes five (<NUM>) layers of a NN, examples disclosed herein are not limited thereto. After the example layer map generator <NUM> completes building and/or otherwise generating the example mapping file <NUM>, the example NN mapper <NUM> overrides one or more directives of the example compiler <NUM> that are associated with resource selection. Stated differently, because the example mapping file <NUM> includes particular resources for each layer that are selected based on improving (e.g., optimizing) one or more reward functions of an actor-critic RL framework, such mappings cause improved performance of the NN on such hardware resources when compared to shortsighted heuristics and/or rule-based selections of the example compiler <NUM>.

While an example manner of implementing the neural network mapper <NUM> of <FIG> is illustrated in <FIG> and <FIG>, one or more of the elements, processes and/or devices illustrated in <FIG> and/or 5A may be combined, divided, re-arranged, omitted, eliminated and/or implemented in any other way. Further, the example neural network input detector <NUM>, the example layer selector <NUM>, the example layer map generator <NUM>, the example iteration tracker <NUM>, the example constraint definer <NUM>, the example agent generator <NUM>, the example state definer <NUM>, the example action determiner <NUM>, the example reward determiner <NUM> and/or, more generally, the example neural network mapper <NUM> of <FIG>, the example mapping process <NUM> of <FIG>, the example mapping process <NUM> of <FIG>, and/or the example system <NUM> of <FIG> may be implemented by hardware, software, firmware and/or any combination of hardware, software and/or firmware. Thus, for example, any of the example neural network input detector <NUM>, the example layer selector <NUM>, the example layer map generator <NUM>, the example iteration tracker <NUM>, the example constraint definer <NUM>, the example agent generator <NUM>, the example state definer <NUM>, the example action determiner <NUM>, the example reward determiner <NUM> and/or, more generally, the example neural network mapper <NUM> of <FIG>, the example mapping process <NUM> of <FIG>, the example mapping process <NUM> of <FIG>, and/or the example system <NUM> of <FIG> could be implemented by one or more analog or digital circuit(s), logic circuits, programmable processor(s), programmable controller(s), graphics processing unit(s) (GPU(s)), digital signal processor(s) (DSP(s)), application specific integrated circuit(s) (ASIC(s)), programmable logic device(s) (PLD(s)) and/or field programmable logic device(s) (FPLD(s)). When reading any of the apparatus or system claims of this patent to cover a purely software and/or firmware implementation, at least one of the example neural network input detector <NUM>, the example layer selector <NUM>, the example layer map generator <NUM>, the example iteration tracker <NUM>, the example constraint definer <NUM>, the example agent generator <NUM>, the example state definer <NUM>, the example action determiner <NUM>, the example reward determiner <NUM> and/or, more generally, the example neural network mapper <NUM> of <FIG>, the example mapping process <NUM> of <FIG>, the example mapping process <NUM> of <FIG>, and/or the example system <NUM> of <FIG> is/are hereby expressly defined to include a non-transitory computer readable storage device or storage disk such as a memory, a digital versatile disk (DVD), a compact disk (CD), a Blu-ray disk, etc. including the software and/or firmware. Further still, the example neural network mapper <NUM> of <FIG> and <FIG> may include one or more elements, processes and/or devices in addition to, or instead of, those illustrated in <FIG>, and <FIG>, and/or may include more than one of any or all of the illustrated elements, processes and devices. As used herein, the phrase "in communication," including variations thereof, encompasses direct communication and/or indirect communication through one or more intermediary components, and does not require direct physical (e.g., wired) communication and/or constant communication, but rather additionally includes selective communication at periodic intervals, scheduled intervals, aperiodic intervals, and/or one-time events.

Flowcharts representative of example hardware logic, machine readable instructions, hardware implemented state machines, and/or any combination thereof for implementing the neural network mapper <NUM> of <FIG> and/or 5A are shown in <FIG> and <FIG>. The machine readable instructions may be one or more executable programs or portion(s) of an executable program for execution by a computer processor such as the processor <NUM> shown in the example processor platform <NUM> discussed below in connection with <FIG>. The program(s) may be embodied in software stored on a non-transitory computer readable storage medium such as a CD-ROM, a floppy disk, a hard drive, a DVD, a Blu-ray disk, or a memory associated with the processor <NUM>, but the entire program(s) and/or parts thereof could alternatively be executed by a device other than the processor <NUM> and/or embodied in firmware or dedicated hardware. Further, although the example program is described with reference to the flowcharts illustrated in <FIG> and <FIG>, many other methods of implementing the example neural network mapper <NUM> may alternatively be used. For example, the order of execution of the blocks may be changed, and/or some of the blocks described may be changed, eliminated, or combined. Additionally or alternatively, any or all of the blocks may be implemented by one or more hardware circuits (e.g., discrete and/or integrated analog and/or digital circuitry, an FPGA, an ASIC, a comparator, an operational-amplifier (op-amp), a logic circuit, etc.) structured to perform the corresponding operation without executing software or firmware.

The machine readable instructions described herein may be stored in one or more of a compressed format, an encrypted format, a fragmented format, a compiled format, an executable format, a packaged format, etc. Machine readable instructions as described herein may be stored as data (e.g., portions of instructions, code, representations of code, etc.) that may be utilized to create, manufacture, and/or produce machine executable instructions. For example, the machine readable instructions may be fragmented and stored on one or more storage devices and/or computing devices (e.g., servers). The machine readable instructions may require one or more of installation, modification, adaptation, updating, combining, supplementing, configuring, decryption, decompression, unpacking, distribution, reassignment, compilation, etc. in order to make them directly readable, interpretable, and/or executable by a computing device and/or other machine. For example, the machine readable instructions may be stored in multiple parts, which are individually compressed, encrypted, and stored on separate computing devices, wherein the parts when decrypted, decompressed, and combined form a set of executable instructions that implement a program such as that described herein.

In another example, the machine readable instructions may be stored in a state in which they may be read by a computer, but require addition of a library (e.g., a dynamic link library (DLL)), a software development kit (SDK), an application programming interface (API), etc. in order to execute the instructions on a particular computing device or other device. In another example, the machine readable instructions may need to be configured (e.g., settings stored, data input, network addresses recorded, etc.) before the machine readable instructions and/or the corresponding program(s) can be executed in whole or in part. Thus, the disclosed machine readable instructions and/or corresponding program(s) are intended to encompass such machine readable instructions and/or program(s) regardless of the particular format or state of the machine readable instructions and/or program(s) when stored or otherwise at rest or in transit.

As mentioned above, the example processes of <FIG> and <FIG> may be implemented using executable instructions (e.g., computer and/or machine readable instructions) stored on a non-transitory computer and/or machine readable medium such as a hard disk drive, a flash memory, a readonly memory, a compact disk, a digital versatile disk, a cache, a randomaccess memory and/or any other storage device or storage disk in which information is stored for any duration (e.g., for extended time periods, permanently, for brief instances, for temporarily buffering, and/or for caching of the information). As used herein, the term non-transitory computer readable medium is expressly defined to include any type of computer readable storage device and/or storage disk and to exclude propagating signals and to exclude transmission media.

The program <NUM> of <FIG> includes block <NUM>, where the example neural network input detector <NUM> determines whether a neural network input has been retrieved and/or otherwise received. In some examples, the neural network input detector <NUM> detects a request to process a neural network, and then retrieves a device agnostic model associated with the neural network (block <NUM>). As described above, the device agnostic model may be retrieved in a manner consistent with the example model <NUM> of <FIG>. Because the NN model typically has two or more layers, the example layer selector <NUM> selects a layer so that model analysis can occur on a layer-by-layer basis (block <NUM>). The example layer map generator <NUM> generates a layer resource mapping that is unique to the selected layer of the network (block <NUM>), as described in further detail below in connection with <FIG>.

<FIG> illustrates additional detail associated with generating a layer resource mapping of block <NUM> of <FIG>. In the illustrated example of <FIG>, the example iteration tracker <NUM> determines if a first iteration of reinforcement learning is occurring for the selected layer (block <NUM>). If so, then a seed configuration is needed, as well as a definition of constraints for the selected layer of interest and/or particular targeted performance objectives for the model. For instance, respective layers of a neural network may have different processing needs and/or memory storage needs, which should be accommodated and/or otherwise satisfied to allow the layer to complete its corresponding computing objectives. As such, the example constraint definer <NUM> defines resource constraints (block <NUM>). As described above, any number and/or type of resource constraints and/or objectives may be defined, such as a maximum throughput objective, a minimum latency objective, a minimum power consumed objective, etc. Because this is a first iteration, the example constraint definer <NUM> also identifies a heuristic and/or rule-based mapping configuration for the target hardware (block <NUM>).

The example agent generator <NUM> generates a reinforcement learning (RL) agent (block <NUM>) (e.g., see the example RL agent <NUM> of <FIG>). Additionally, the example state definer <NUM> generates a current state (St) representation that is based on the heuristics/rules and prior layer state information, if any (block <NUM>). The example action determiner <NUM> applies the mapping configuration to the available/candidate resources <NUM> (block <NUM>) so that an execution iteration of the selected layer can occur. Stated differently, because this is the first iteration of layer execution using a heuristically or rule-based configuration of hardware and/or memory, performance of the layer is not likely to meet and/or otherwise satisfy the optimum metrics sought. However, while poor resource configuration settings do not necessarily result in improved (e.g., optimized) performance metrics of the selected layer, because reinforcement learning is applied in view of the performance goals, the RL process can learn from bad decisions.

During subsequent iterations of the example program <NUM> of <FIG>, the example iteration tracker <NUM> determines that prior iterations have already occurred (block <NUM>). If so, then the example state definer <NUM> updates state metrics of the mapping (block <NUM>) and the example reward determiner <NUM> evaluates performance metrics of the mapping (block <NUM>). In other words, the example reward determiner <NUM> determines how well or how poorly the mapping performed in view of target performance characteristics. Based on the evaluation results and/or an indication of convergence of the RL framework, the example reward determiner <NUM> determines whether to continue with an alternate (additional) mapping configuration (block <NUM>). If not, such as when an indication of convergence suggests that further configuration attempts are unlikely to substantially improve performance characteristics, the example program of block <NUM> returns to block <NUM> of <FIG>. However, in the event additional mapping configurations are to be attempted with the RL framework (block <NUM>), then the example action determiner <NUM> updates the resource mapping to an alternate combination of hardware and/or memory (block <NUM>). In some examples, the alternate combination of which hardware and which memory combination to attempt is guided by an example reward function, such as the example reward function of <FIG>. The example action determiner <NUM> applies such mapping configuration to the example resources <NUM> (or hardware <NUM>, or hardware simulator <NUM>) and executes the configuration in an effort to acquire additional performance datapoints (block <NUM>). Control then returns to block <NUM>.

Returning to the illustrated example of <FIG>, in response to completion of analyzing one layer of the example network (block <NUM>), the example layer map generator <NUM> stores the mapping configuration for the previously analyzed layer that has a relatively highest score. For example, in the event of a desire to improve (e.g., optimize) multiple performance metrics for the model, the respective mapping configuration that exhibits a relatively highest aggregate score is selected as the hardware configuration to be used for that layer. The example layer map generator <NUM> stores the improved (e.g., optimized) configuration in the example workload data store <NUM> (block <NUM>).

The example layer selector <NUM> determines whether there are one or more additional layers of the model to analyze (block <NUM>). If so, then control returns to block <NUM> to initiate another iteration. On the other hand, when the example layer selector <NUM> determines that all layers of the model have been analyzed (and a corresponding improved (e.g., optimized) mapping for each respective layer has been determined and stored in the example workload data store <NUM>) (block <NUM>), then the example layer map generator <NUM> generates a mapping file for the model (block <NUM>). As described above, the mapping file may be generated in a manner consistent with the example mapping file <NUM> of <FIG>. The example neural network mapper <NUM> uses the generated mapping file to override compiler directives of a compiler (block <NUM>), such as the example compiler <NUM> of <FIG>.

<FIG> is a block diagram of an example processor platform <NUM> structured to execute the instructions of <FIG> and <FIG> to implement the neural network mapper <NUM> of <FIG> and/or 5A. The processor platform <NUM> can be, for example, a server, a personal computer, a workstation, a selflearning machine (e.g., a neural network), a mobile device (e.g., a cell phone, a smart phone, a tablet such as an iPad™), a personal digital assistant (PDA), an Internet appliance, a digital video recorder, a gaming console, a personal video recorder, a set top box, a headset or other wearable device, or any other type of computing device.

For example, the processor <NUM> can be implemented by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers from any desired family or manufacturer. The hardware processor may be a semiconductor based (e.g., silicon based) device. In this example, the processor implements the example neural network input detector <NUM>, the example layer selector <NUM>, the example layer map generator <NUM>, the example iteration tracker <NUM>, the example constraint definer <NUM>, the example agent generator <NUM>, the example state definer <NUM>, the example action determiner <NUM>, the example reward determiner <NUM> and/or, more generally, the example neural network mapper <NUM>.

The input device(s) can be implemented by, for example, an audio sensor, a microphone, a camera (still or video), a keyboard, a button, a mouse, a touchscreen, a trackpad, a trackball, isopoint and/or a voice recognition system.

The machine executable instructions <NUM> of <FIG> and <FIG> may be stored in the mass storage device <NUM>, in the volatile memory <NUM>, in the non-volatile memory <NUM>, and/or on a removable non-transitory computer readable storage medium such as a CD or DVD.

From the foregoing, it will be appreciated that example methods, apparatus and articles of manufacture have been disclosed that improve resource utilization of neural networks. In particular, examples disclosed herein overcome the inefficiency of standard compilers that attempt to assign resources to neural networks using heuristics or rule-based approaches. For instance, mere application of the greedy algorithm that is implemented by typical compilers causes substantial hardware utilization inefficiencies and poor network performance. Such poor performance is due to, in part, a lack of consideration of multiple performance characteristics of interest to be improved (e.g., optimized) when selecting particular hardware to be applied on a layer-by-layer basis of a neural network. Disclosed methods, apparatus and articles of manufacture improve the efficiency of executing an AI/DL/NN machine learning operation(s) on a computing device by facilitating resource selection in a manner that avoids operator discretion or inefficient and short-sighted application of the greedy algorithm. Improved resource mapping techniques disclosed herein facilitate resource selection with reinforcement learning in view of multiple performance characteristics measured by one or more cost/reward functions, such that subsequent resource combination permutations attempt to improve (e.g., maximize) the cost/reward function. Accordingly, disclosed methods, apparatus, systems and articles of manufacture are directed to one or more improvement(s) in the functioning of a computer.

Although certain example methods, apparatus and articles of manufacture have been disclosed herein, the scope of coverage of this patent is not limited thereto. On the contrary, this patent covers all methods, apparatus and articles of manufacture fairly falling within the scope of the claims of this patent.

Claim 1:
An apparatus to generate a resource map for a neural network, the apparatus comprising:
a constraint definer (<NUM>) to define performance characteristic targets for selected layers of the neural network, wherein performance characteristic targets comprise throughput metrics, response time metrics, and/or reduced power metrics;
an action determiner (<NUM>) to apply permutations of resources to each of the selected layers in a first sequence of iterations over the permutations of resources and in a second sequence of iterations over the selected layers;
a state definer (<NUM>) to update state metrics in response to applying a particular permutation of resources;
a reward determiner (<NUM>) to calculate a results metric based on (a) resource performance metrics and (b) the performance characteristic targets, to determine, for each of the selected layers and each applied permutation of resources, which resource assignments exhibit the best performance as compared to other resource assignments, and to generate a respective iteration decision for each of the selected layers and each applied permutation of resources, based on the results metric, wherein the respective iteration decision indicates that the first sequence of iterations is to be stopped if a threshold degree of convergence has been detected or a threshold number of iterations has been reached; and
a layer map generator (<NUM>) to generate a resource mapping file, the resource mapping file including respective resource assignments for corresponding layers of the neural network, the resource assignments selected based on the results metric.