Patent Description:
The high-level of performance of DNNs stems from their ability to extract high-level features from input data after using statistical learning over a large data set to obtain an effective representation of an input space. However, the superior performance of DNNs comes at the cost of high computational complexity. High performance general-purpose processors, such as graphics processing units ("GPUs"), are commonly utilized to provide the high level of computational performance required by many DNN applications.

While general-purpose processors, like GPUs, can provide a high level of computational performance for implementing DNNs, these types of processors are typically unsuitable for use in performing DNN operations over long durations in computing devices where low power consumption is critical. For example, general-purpose processors, such as GPUs, can be unsuitable for use in performing long-running DNN tasks in battery-powered portable devices, like smartphones or alternate/virtual reality ("AR/VR") devices, where the reduced power consumption is required to extend battery life.

Reduced power consumption while performing continuous DNN tasks, such as detection of human movement, can also be important in non- battery-powered devices, such as a power-over-Ethernet ("POE") security camera for example. In this specific example, POE switches can provide only a limited amount of power; reducing the power consumption of POE devices like security cameras permits the use of POE switches that provide less power.

Application-specific integrated circuits ("ASICs") have been developed that can provide performant DNN processing while at the same time reducing power consumption as compared to general-purpose processors. Despite advances in this area, however, there is a continued need to improve the performance and reduce the power consumption of ASICs that perform DNN processing, particularly for use in computing devices where the low power consumption is critical.

It is with respect to these and other technical challenges that the disclosure made herein is presented. <CIT> discloses hardware implementations of, and methods for processing, a convolution layer of a DNN that comprise a plurality of convolution engines wherein the input data and weights are provided to the convolution engines in an order that allows input data and weights read from memory to be used in at least two filter-window calculations performed either by the same convolution engine in successive cycles or by different convolution engines in the same cycle. For example, in some hardware implementations of a convolution layer the convolution engines are configured to process the same weights but different input data each cycle, but the input data for each convolution engine remains the same for at least two cycles so that the convolution engines use the same input data in at least two consecutive cycles.

An neural processing element with single instruction, multiple data ("SIMD") compute lanes is disclosed herein. The architecture of the neural processing element disclosed herein reduces the number of accumulator bits per multiplier in the neuron as compared to previous solutions. Reducing the number of accumulator bits in this manner can reduce the amount of power consumed by the neural processing element as also compared to previous implementations. Other technical benefits not specifically mentioned herein can also be realized through implementations of the disclosed subject matter.

In order to realize the technical benefits mentioned briefly above, a DNN processor is disclosed that includes neural processing elements. The neural processing elements include, among other things, hardware binary multipliers (which might be referred to herein as "multipliers"). The hardware binary multipliers are configured to multiply a binary operand with another binary operand to generate a binary output. In one particular configuration, the operands are signed <NUM>-bit binary numbers (i.e. <NUM>-bit binary numbers). In this configuration, the output of the hardware binary multipliers is an <NUM>-bit binary number. The operands can be powers of two plus one bit (i.e. (N<NUM>)+<NUM>)).

The neural processing elements also include a single hardware binary adder tree (which might be referred to herein as an "adder tree") for summing the binary outputs of the hardware binary multipliers. In one particular implementation, the hardware binary adder tree includes an even number of hardware binary adders (which might be referred to herein as "first hardware binary adders") for summing the binary outputs of the hardware binary multipliers. The hardware binary adders are configured to output <NUM>-bit binary numbers in one particular configuration.

The single hardware binary adder tree can also include an even number of additional hardware binary adders (which might be referred to herein as "second hardware binary adders") for summing outputs of the hardware binary adders described above. The outputs of these hardware binary adders are <NUM>-bit binary numbers in one configuration but can include a different number of bits in other configurations.

In some configurations, the single hardware binary adder tree includes another hardware binary adder (which might be referred to herein as a "third hardware binary adder") for summing outputs of the second hardware binary adders described above. In one particular configuration, the output of the third hardware binary adder is a <NUM>-bit binary number. Other numbers of binary adders having different bit widths can be utilized in other configurations.

In one configuration, the neural processing elements also include a storage element (e.g. a flip flop, SRAM, DRAM, etc.) for storing the binary output of the single hardware binary adder tree. The binary output of the single hardware binary adder tree is a <NUM>-bit binary number in some configurations. Other bit widths can be utilized in other configurations.

It should be appreciated that the above-described subject matter can be implemented as a computer-controlled apparatus, a computer-implemented method, a computing device, or as an article of manufacture such as a computer readable medium. These and various other features will be apparent from a reading of the following Detailed Description and a review of the associated drawings.

This Summary is provided to introduce a brief description of some aspects of the disclosed technologies in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended that this Summary be used to limit the scope of the claimed subject matter.

The following detailed description is directed to an neural processing element with SIMD compute lanes. As discussed briefly above, implementations of the disclosed technologies can conserve power as compared to previous neural processing element implementations. Other technical benefits not specifically mentioned herein can also be realized through implementations of the disclosed subject matter.

While the subject matter described herein is presented in the general context of hardware neural processing elements implemented in conjunction with a hardware DNN processor, those skilled in the art will recognize that other implementations can be performed in combination with other types of computing systems and modules. Those skilled in the art will also appreciate that the subject matter described herein can be practiced with other computer system configurations, including hand-held devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, computing or processing systems embedded in devices (such as wearable computing devices, automobiles, home automation etc.), minicomputers, mainframe computers, and the like.

In the following detailed description, references are made to the accompanying drawings that form a part hereof, and which are shown by way of illustration specific configurations or examples. Referring now to the drawings, in which like numerals represent like elements throughout the several FIGS. , aspects of an neural processing element with SIMD compute lanes will be described.

<FIG> is a computing architecture diagram that shows aspects of the configuration and operation of a processing system that implements the technologies disclosed herein, according to one embodiment. The processing system disclosed herein is configured in some embodiments to solve classification problems (and related problems) such as, but not limited to, eye tracking, hand tracking, object detection, semantic labeling, and feature extraction.

In order to provide this functionality, a DNN processor <NUM> is provided that can implement a recall-only neural network and programmatically support a wide variety of network structures. Training for the network implemented by the DNN processor <NUM> can be performed offline in a server farm, data center, or another suitable computing environment. The result of training a DNN is a set of parameters that can be known as "weights" or "kernels. " These parameters represent a transform function that can be applied to an input with the result being a classification or semantically labeled output.

The DNN processor <NUM> disclosed herein can be considered a superscalar processor. The DNN processor <NUM> can dispatch one or more instructions to multiple execution units, called neural processing elements 105F. The execution units can be "simultaneous dispatch simultaneous complete," where each execution unit is synchronized with each of the other execution units. The DNN processor <NUM> can be classified as a single instruction stream, multiple data stream ("SIMD") architecture.

A neural processing element 105F is the base unit in artificial neural networks that is used to model a biological neuron in the brain. The model of a neural processing element 105F can include the inner product of an input vector with a weight vector added to a bias, with an activation function applied.

Each neural processing element 105F in the DNN processor <NUM> is capable of performing weighted sum, max pooling, bypass, and potentially other types of operations. The neural processing elements 105F process input and weight data every clock cycle. Each neural processing element 105F is synchronized to all other neural processing elements 105F in terms of progress within a kernel to minimize the flow of kernel data within the DNN processor <NUM>.

Each neural processing element 105F can contain a multiplier, an adder, a comparator, and a number of accumulators (not shown in <FIG>). By having multiple accumulators, the neural processing elements 105F are able to maintain context for multiple different active kernels at a time. Each accumulator is capable of being loaded from a read of the SRAM <NUM> (described below). The accumulators can sum themselves with the contents of other accumulators from other neural processing elements 105F.

The DNN processor <NUM> accepts planar data as input, such as image data. Input to the DNN processor <NUM> is not, however, limited to image data. Rather, the DNN processor <NUM> can operate on any input data presented to the DNN processor <NUM> in a uniform planar format. In one particular embodiment, the DNN processor <NUM> can accept as input multi-planar one-byte or two-byte data frames.

Each input frame can be convolved with an NxKxHxW set of kernels, where N is the number of kernels, K is the number of channels per kernel, H is the height, and W is the width. Convolution is performed on overlapping intervals across the input data where the interval is defined by strides in the X and Y directions. These functions are performed by the neural processing elements 105F and managed by the DNN processor <NUM> and software-visible control registers. Other types of operations might also be performed including, but not limited to, fully connected, batch normalization, and other types of operations.

The DNN processor <NUM> supports multiple data types: weights; input data/feature maps; activation data; biases; and scalars. Input data/feature maps and activation data are, in most cases, two names for the same data with the distinction that when referring to an output of a layer the term activation data is used. When referring to the input of a layer the term input data/feature map is used.

The neural processing elements 105F in the DNN processor <NUM> compute a weighted sum of their inputs and pass the weighted sum through an "activation function" or "transfer function. " The transfer function commonly has a sigmoid shape but might also take on the form of a piecewise linear function, step function, or another type of function. The activation function allows the neural processing elements 105F to train to a larger set of inputs and desired outputs where classification boundaries are non-linear.

The DNN processor <NUM> operates on a list of layer descriptors which correspond to the layers of a neural network. The list of layer descriptors can be treated by the DNN processor <NUM> as instructions. These descriptors can be pre-fetched from memory into the DNN processor <NUM> and executed in order. The descriptor list acts as a set of instructions to the DNN processor <NUM>. In some configurations, two types of instructions are utilized: layer descriptors; and program instructions that get executed on a sequence controller. Software tools and/or compilers can be executed on devices external to the DNN processor <NUM> to create the descriptor lists that are executed on the DNN processor <NUM>.

Generally, there can be two main classes of descriptors: memory-to-memory move ("M2M") descriptors; and operation descriptors. M2M descriptors can be used to move data to/from the main memory to/from a local buffer (i.e. the buffer <NUM> described below) for consumption by the operation descriptors. M2M descriptors follow a different execution pipeline than the operation descriptors. The target pipeline for M2M descriptors can be the internal DMA engine 105B or the configuration registers <NUM>, whereas the target pipeline for the operation descriptors can be the neural processing elements 105F.

Operational descriptors specify a specific operation that the neural processing elements 105F should perform on a data structure located in local static random-access memory ("SRAM") memory. The operational descriptors are processed in order and are capable of many different layer operations, at least some of which are described herein.

As illustrated in <FIG>, the DNN processor <NUM> has a memory subsystem with a unique L1 and L2 buffer structure. The L1 and L2 buffers shown in <FIG> are designed specifically for neural network processing. By way of example, the L2 buffer <NUM> can maintain a selected storage capacity with a high speed private interface operating at a selected frequency. The L1 buffer <NUM> can maintain a selected storage capacity that can be split between kernel and activation data. The L1 buffer <NUM> might be referred to herein as the "buffer <NUM>," and the L2 buffer <NUM> might be referred to herein as the SRAM <NUM>.

Computational data (i.e. inputs data, weights and activation data) is stored in the SRAM <NUM> row-major in some embodiments. The computational data can be organized as two buffers, where one buffer contains input data, which might be referred to herein as the "input buffer," and the other buffer, which might be referred to herein as the "weight buffer," contains kernel weights. The buffers are filled from the SRAM <NUM> by the load/store unit 105C. Data is accumulated in each buffer until it has reached its predetermined capacity. The buffer data is then copied to a shadow buffer in some embodiments and presented to the neural processing elements 105F.

The DNN processor <NUM> can also comprise a number of other components including, but not limited to, a register interface <NUM>, a prefetch unit 105A, a store/gather unit 105E, a layer controller 105D, and a register interface <NUM>. The DNN processor <NUM> can include additional or alternate components in some embodiments.

The DNN processor <NUM> operates in conjunction with other external computing components in some configurations. For example, the DNN processor <NUM> is connected to a host application processor system on chip ("the host SoC") <NUM> in some embodiments. The DNN processor <NUM> can be connected to the host SoC <NUM> through a PCIe interface, for example. Appropriate PCIe components, such as the PCIe endpoint <NUM> can be utilized to enable these connections.

The Host SoC <NUM> serves as the application processor for the DNN processor <NUM>. The main operating system, application, and auxiliary sensor processing are performed by the host SoC <NUM>. The host SoC <NUM> can also be connected to an input data source <NUM>, such as an external camera, that provides input data, such as image data, to the DNN processor <NUM>. Additional central processing units ("CPUs" or "processors"), such as TENSILICA nodes <NUM>, can also be utilized to coordinate the operation of the DNN processor <NUM>, aspects of which will be described below.

DDR DRAM <NUM> can also be connected to the host SoC <NUM> that can be used as the main system memory. This memory is accessible from the host SoC <NUM> across the high bandwidth fabric <NUM> (e.g. PCIe bus) by way of a memory controller <NUM>. The high bandwidth fabric <NUM> provides bidirectional direct memory access ("DMA") small messaging transactions and larger DMA transactions. A bridge <NUM> and low bandwidth fabric <NUM> can connect the DNN processor <NUM> to the host SoC <NUM> for sub-module configuration and other functions.

The DNN processor <NUM> can include a DMA engine 105B that is configured to move data to and from main memory <NUM>. The DMA engine 105B has two channels in some embodiments. One channel is dedicated to fetching operation descriptors while the other channel is dedicated to M2M operations. A DMA descriptor can be embedded in the M2M descriptor. Descriptors in this context are DMA descriptors that are used to move the contents of memory, not to be confused with the operation descriptors described above.

To offload the local SRAM memory <NUM>, and to provide more space for input data and weight data, the activation output can optionally be streamed directly to DDR memory <NUM>. When streaming data to DDR memory <NUM>, the DNN processor <NUM> will accumulate enough data for a burst transaction on the high bandwidth fabric <NUM> and will buffer enough transactions to minimize backpressure on the neural processing elements 105F. Additional details regarding the operation of the DNN processor <NUM> will be provided below. In particular, details regarding the configuration and operation of an neural processing element 105F with SIMD compute lanes will be provided below with regard to <FIG> and <FIG>.

<FIG> is a computing architecture diagram showing aspects of the configuration and operation of an neural processing element 105F having SIMD compute lanes. As shown in <FIG>, the neural processing elements 105F of the DNN processor <NUM> can include, among other things, hardware binary multipliers <NUM> (which might be referred to herein as "multipliers") arranged into compute lanes (e.g. lane <NUM> to lane <NUM> in the illustrated example). The number of compute lanes present is computed as a power of two.

The hardware binary multipliers <NUM> are configured to multiply a binary operand <NUM> with another binary operand <NUM> to generate a binary output. The operands <NUM> and <NUM> might, for example, be inputs data, weights, activation data, or another type of data. In one particular configuration, the operands <NUM> and <NUM> are signed <NUM>-bit binary numbers (i.e. <NUM>-bit binary numbers). In this configuration, the output of the hardware binary multipliers <NUM> is an <NUM>-bit binary number. The operands can be powers of two plus one bit (i.e. (N<NUM>)+<NUM>)) in other configurations.

In one particular configuration, the neural processing element 105F includes eight hardware binary multipliers 202A-<NUM>. In this configuration, the hardware binary multiplier 202A is configured to multiply the operands 204A and 206A, the hardware binary multiplier 202B is configured to multiply the operands 204B and 206B, the hardware binary multiplier 202C is configured to multiply the operands 204C and 206C, the hardware binary multiplier 202D is configured to multiply the operands 204D and 206D, the hardware binary multiplier 202E is configured to multiply the operands 204E and 206E, the hardware binary multiplier 202F is configured to multiply the operands 204F and 206F, and the hardware binary multiplier <NUM> is configured to multiply the operands <NUM> and <NUM>. Other numbers of multipliers <NUM> can be utilized in other configurations.

As shown in <FIG>, the neural processing elements 105F <NUM> also include a single hardware binary adder tree <NUM> (which might be referred to herein as an "adder tree") for summing the binary outputs of the hardware binary multipliers <NUM>. In one particular implementation, the adder tree <NUM> includes an even number (four in the illustrated example) of hardware binary adders 208A-208D (which might be referred to herein as "first hardware binary adders") for summing the binary outputs of the hardware binary multipliers <NUM>. The hardware binary adders 208A-208D are configured to output <NUM>-bit binary numbers in one particular configuration. In this regard, it is to be appreciated that the output might be a different number of bits in other configurations and that the number of levels in the adder tree <NUM> is determined by the number of compute lanes.

In the illustrated configuration, the adder tree <NUM> includes four first hardware binary adders 208A-208D. In particular, the hardware adder tree <NUM> includes a hardware binary adder 208A that adds the binary outputs of the hardware binary multiplier 202A and the hardware binary multiplier 202B, a hardware binary adder 208B that adds the binary outputs of the hardware binary multiplier 202C and the hardware binary multiplier 202D, a hardware binary adder 208C that adds the binary outputs of the hardware binary multiplier 202E and the hardware binary multiplier 202F, and a hardware binary adder 208D that adds the binary outputs of the hardware binary multiplier <NUM> and the hardware binary multiplier <NUM>.

The adder tree <NUM> can also include an even number (two in the illustrated example) of additional hardware binary adders 208E and 208F (which might be referred to herein as "second hardware binary adders") for summing outputs of the hardware binary adders 208A-208D, described above. The outputs of these hardware binary adders 208E and 208F are <NUM>-bit binary numbers in one configuration.

In some configurations, the adder tree <NUM> includes a single hardware binary adder <NUM> (which might be referred to herein as a "third hardware binary adder") for summing outputs of the second hardware binary adders 208E and 20F described above. In one particular configuration, the output of the third hardware binary adder <NUM> is a <NUM>-bit binary number. Other numbers of binary adders <NUM> having different bit widths can be utilized in other configurations. In this regard, it is to be appreciated that the number of binary adders <NUM> in the adder tree <NUM> will vary based upon the number of multipliers <NUM> in the neural processing element 105F.

In one configuration, the neural processing elements 105F <NUM> also include a storage element <NUM> (e.g. flip flop, SRAM, DRAM, etc.) for storing a binary output of the adder tree <NUM> (i.e. the output of the adder <NUM> in the illustrated configuration). The binary output of the adder tree <NUM> is a <NUM>-bit binary number in some configurations. Other bit widths can be utilized in other configurations.

<FIG> is a flow diagram showing a routine <NUM> that illustrates aspects of the operation of the neural processing element 105F configured with SIMD compute lanes described above with reference to <FIG> and <FIG>, according to one embodiment disclosed herein. It should be appreciated that the logical operations described herein with regard to <FIG>, and the other FIGS. , can be implemented (<NUM>) as a sequence of computer implemented acts or program modules running on a computing device and/or (<NUM>) as interconnected machine logic circuits or circuit modules within a computing device.

The particular implementation of the technologies disclosed herein is a matter of choice dependent on the performance and other requirements of the computing device. Accordingly, the logical operations described herein are referred to variously as states, operations, structural devices, acts, or modules. These states, operations, structural devices, acts and modules can be implemented in hardware, software, firmware, in special-purpose digital logic, and any combination thereof. It should be appreciated that more or fewer operations can be performed than shown in the FIGS. and described herein. These operations can also be performed in a different order than those described herein.

The routine <NUM> begins at operation <NUM>, where the multipliers <NUM> of an neural processing element 105F multiply the operands <NUM> and <NUM> in the manner described above with regard to <FIG>. The <NUM> then proceeds from operation <NUM> to operation <NUM>, where the adders <NUM> in the adder tree <NUM> sum the outputs of the multipliers <NUM> in the manner described above.

From operation <NUM>, the routine <NUM> proceeds to operation <NUM>, where the adder tree <NUM> stores the result of the summation in the storage element <NUM>. The routine <NUM> then proceeds from operation <NUM> back to operation <NUM>, where the process described above can be repeated for additional operands <NUM> and <NUM>.

<FIG> is a computer architecture diagram showing an illustrative computer hardware and software architecture for a computing device that can act as an application host for the DNN processor <NUM> presented herein. In particular, the architecture illustrated in <FIG> can be utilized to implement a server computer, mobile phone, an e-reader, a smartphone, a desktop computer, an AR/VR device, a tablet computer, a laptop computer, or another type of computing device suitable for use with the DNN processor <NUM>.

The computer <NUM> illustrated in <FIG> includes a central processing unit <NUM> ("CPU"), a system memory <NUM>, including a random-access memory <NUM> ("RAM") and a read-only memory ("ROM") <NUM>, and a system bus <NUM> that couples the memory <NUM> to the CPU <NUM>. A basic input/output system ("BIOS" or "firmware") containing the basic routines that help to transfer information between elements within the computer <NUM>, such as during startup, can be stored in the ROM <NUM>. The computer <NUM> further includes a mass storage device <NUM> for storing an operating system <NUM>, application programs, and other types of programs. The mass storage device <NUM> can also be configured to store other types of programs and data.

The mass storage device <NUM> is connected to the CPU <NUM> through a mass storage controller (not shown) connected to the bus <NUM>. The mass storage device <NUM> and its associated computer readable media provide non-volatile storage for the computer <NUM>. Although the description of computer readable media contained herein refers to a mass storage device, such as a hard disk, CD-ROM drive, DVD-ROM drive, or USB storage key, it should be appreciated by those skilled in the art that computer readable media can be any available computer storage media or communication media that can be accessed by the computer <NUM>.

Communication media includes computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any delivery media. The term "modulated data signal" means a signal that has one or more of its characteristics changed or set in a manner so as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency, infrared and other wireless media. Combinations of the any of the above should also be included within the scope of computer readable media.

By way of example, and not limitation, computer storage media can include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. For example, computer storage media includes, but is not limited to, RAM, ROM, EPROM, EEPROM, flash memory or other solid-state memory technology, CD-ROM, digital versatile disks ("DVD"), HD-DVD, BLU-RAY, or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and which can be accessed by the computer <NUM>. For purposes of the claims, the phrase "computer storage medium," and variations thereof, does not include waves or signals per se or communication media.

According to various configurations, the computer <NUM> can operate in a networked environment using logical connections to remote computers through a network such as the network <NUM>. The computer <NUM> can connect to the network <NUM> through a network interface unit <NUM> connected to the bus <NUM>. It should be appreciated that the network interface unit <NUM> can also be utilized to connect to other types of networks and remote computer systems. The computer <NUM> can also include an input/output controller <NUM> for receiving and processing input from a number of other devices, including a keyboard, mouse, touch input, an electronic stylus (not shown in <FIG>), or a physical sensor such as a video camera. Similarly, the input/output controller <NUM> can provide output to a display screen or other type of output device (also not shown in <FIG>).

It should be appreciated that the software components described herein, when loaded into the CPU <NUM> and executed, can transform the CPU <NUM> and the overall computer <NUM> from a general-purpose computing device into a special-purpose computing device customized to facilitate the functionality presented herein. The CPU <NUM> can be constructed from any number of transistors or other discrete circuit elements, which can individually or collectively assume any number of states. More specifically, the CPU <NUM> can operate as a finite-state machine, in response to executable instructions contained within the software modules disclosed herein. These computer-executable instructions can transform the CPU <NUM> by specifying how the CPU <NUM> transitions between states, thereby transforming the transistors or other discrete hardware elements constituting the CPU <NUM>.

Encoding the software modules presented herein can also transform the physical structure of the computer readable media presented herein. The specific transformation of physical structure depends on various factors, in different implementations of this description. Examples of such factors include, but are not limited to, the technology used to implement the computer readable media, whether the computer readable media is characterized as primary or secondary storage, and the like. For example, if the computer readable media is implemented as semiconductor-based memory, the software disclosed herein can be encoded on the computer readable media by transforming the physical state of the semiconductor memory. For instance, the software can transform the state of transistors, capacitors, or other discrete circuit elements constituting the semiconductor memory. The software can also transform the physical state of such components in order to store data thereupon.

As another example, the computer readable media disclosed herein can be implemented using magnetic or optical technology. In such implementations, the software presented herein can transform the physical state of magnetic or optical media, when the software is encoded therein. These transformations can include altering the magnetic characteristics of particular locations within given magnetic media. These transformations can also include altering the physical features or characteristics of particular locations within given optical media, to change the optical characteristics of those locations. Other transformations of physical media are possible without departing from the scope and spirit of the present description, with the foregoing examples provided only to facilitate this discussion.

In light of the above, it should be appreciated that many types of physical transformations take place in the computer <NUM> in order to store and execute the software components presented herein. It also should be appreciated that the architecture shown in <FIG> for the computer <NUM>, or a similar architecture, can be utilized to implement other types of computing devices, including hand-held computers, video game devices, embedded computer systems, mobile devices such as smartphones, tablets, and AR/VR devices, and other types of computing devices known to those skilled in the art. It is also contemplated that the computer <NUM> might not include all of the components shown in <FIG>, can include other components that are not explicitly shown in <FIG>, or can utilize an architecture completely different than that shown in <FIG>.

<FIG> is a network diagram illustrating a distributed network computing environment <NUM> in which aspects of the disclosed technologies can be implemented, according to various embodiments presented herein. As shown in <FIG>, one or more server computers 500A can be interconnected via a communications network <NUM> (which may be either of, or a combination of, a fixed-wire or wireless LAN, WAN, intranet, extranet, peer-to-peer network, virtual private network, the Internet, Bluetooth communications network, proprietary low voltage communications network, or other communications network) with a number of client computing devices such as, but not limited to, a tablet computer 500B, a gaming console 500C, a smart watch 500D, a telephone 500E, such as a smartphone, a personal computer 500F, and an AR/VR device <NUM>.

In a network environment in which the communications network <NUM> is the Internet, for example, the server computer 500A can be a dedicated server computer operable to process and communicate data to and from the client computing devices 500B-<NUM> via any of a number of known protocols, such as, hypertext transfer protocol ("HTTP"), file transfer protocol ("FTP"), or simple object access protocol ("SOAP"). Additionally, the networked computing environment <NUM> can utilize various data security protocols such as secured socket layer ("SSL") or pretty good privacy ("PGP"). Each of the client computing devices 500B-<NUM> can be equipped with an operating system operable to support one or more computing applications or terminal sessions such as a web browser (not shown in <FIG>), or other graphical user interface (not shown in <FIG>), or a mobile desktop environment (not shown in <FIG>) to gain access to the server computer 500A.

The server computer 500A can be communicatively coupled to other computing environments (not shown in <FIG>) and receive data regarding a participating user's interactions/resource network. In an illustrative operation, a user (not shown in <FIG>) may interact with a computing application running on a client computing device 500B-<NUM> to obtain desired data and/or perform other computing applications.

The data and/or computing applications may be stored on the server 500A, or servers 500A, and communicated to cooperating users through the client computing devices 500B-<NUM> over an exemplary communications network <NUM>. A participating user (not shown in <FIG>) may request access to specific data and applications housed in whole or in part on the server computer 400A. These data may be communicated between the client computing devices 500B-<NUM> and the server computer 500A for processing and storage.

The server computer 500A can host computing applications, processes and applets for the generation, authentication, encryption, and communication of data and applications, and may cooperate with other server computing environments (not shown in <FIG>), third party service providers (not shown in <FIG>), network attached storage ("NAS") and storage area networks ("SAN") to realize application/data transactions.

It should be appreciated that the computing architecture shown in <FIG> and the distributed network computing environment shown in <FIG> have been simplified for ease of discussion. It should also be appreciated that the computing architecture and the distributed computing network can include and utilize many more computing components, devices, software programs, networking devices, and other components not specifically described herein.

Based on the foregoing, it should be appreciated that an neural processing element having SIMD compute lanes has been disclosed herein. Although the subject matter presented herein has been described in language specific to computer structural features, methodological and transformative acts, specific computing machinery, and computer readable media, it is to be understood that the subject matter set forth in the appended claims is not necessarily limited to the specific features, acts, or media described herein. Rather, the specific features, acts and mediums are disclosed as example forms of implementing the claimed subject matter.

Claim 1:
A deep neural network (DNN) processor (<NUM>), comprising:
a plurality of neuron processing elements implemented as ASICs configured to provide SIMD compute lanes, wherein each of the neuron processing elements are synchronized with each of the other neuron processing elements based on flow of kernel data within the DNN processor, the neuron processing elements comprising:
an even number of hardware binary multipliers (<NUM>), each of the hardware binary multipliers (<NUM>) configured to multiply a first signed <NUM>-bit operand, that is, a <NUM>-bit number, (<NUM>) with a second signed <NUM>-bit operand, that is, a <NUM>-bit number, (<NUM>) to generate a <NUM>-bit binary output, and
a single hardware binary adder tree (<NUM>) for summing binary outputs of the hardware binary multipliers (<NUM>) as follows:
process the <NUM>-bit binary outputs to generate <NUM>-bit values;
process the <NUM>-bit values to generate <NUM>-bit values;
process the <NUM>-bit values to generate a <NUM>-bit value; and
output the <NUM>-bit value.