Computational memory with cooperation among rows of processing elements and memory thereof

A computing device includes an array of processing elements mutually connected to perform single instruction multiple data (SIMD) operations, memory cells connected to each processing element to store data related to the SIMD operations, and a cache connected to each processing element to cache data related to the SIMD operations. Caches of adjacent processing elements are connected. The same or another computing device includes rows of mutually connected processing elements to share data. The computing device further includes a row arithmetic logic unit (ALU) at each row of processing elements. The row ALU of a respective row is configured to perform an operation with processing elements of the respective row.

BACKGROUND

Deep learning has proven to be a powerful technique for performing functions that have long resisted other artificial intelligence approaches. For example, deep learning may be applied to recognition of objects in cluttered images, speech understanding and translation, medical diagnosis, gaming, and robotics. Deep learning techniques typically apply many layers (hence “deep”) of neural networks that are trained (hence “learning”) on the tasks of interest. Once trained, a neural network may perform “inference”, that is, inferring from new input data an output consistent with what it has learned.

Neural networks, which may also be called neural nets, perform computations analogous to the operations of biological neurons, typically computing weighted sums (or dot products) and modifying the results with a memoryless nonlinearity. However, it is often the case that more general functionality, such as memory, multiplicative nonlinearities, and “pooling”, are also required.

In many types of computer architecture, power consumption due to physically moving data between memory and processing elements is non-trivial and is frequently the dominant use of power. This power consumption is typically due to the energy required to charge and discharge the capacitance of wiring, which is roughly proportional to the length of the wiring and hence to distance between memory and processing elements. As such, processing a large number of computations in such architectures, as generally required for deep learning and neural networks, often requires a relatively large amount of power. In architectures that are better suited to handle deep learning and neural networks, other inefficiencies may arise, such as increased complexity, increased processing time, and larger chip area requirements.

SUMMARY

According to one aspect of this disclosure, a computing device includes an array of processing elements mutually connected to perform single instruction multiple data (SIMD) operations, memory cells connected to each processing element to store data related to the SIMD operations, and a cache connected to each processing element to cache data related to the SIMD operations. A first cache of a first processing element is connected to a second cache of a second processing element that is adjacent the first processing element in the array of processing elements.

According to another aspect of this disclosure, a computing device includes a plurality of rows of processing elements to perform SIMD operations. The processing elements of each row are mutually connected to share data. The computing device further includes a row arithmetic logic unit (ALU) at each row of the plurality of rows of processing elements. The row ALU of a respective row is configured to perform an operation with processing elements of the respective row.

DETAILED DESCRIPTION

The techniques described herein aim to improve computational memory to handle large numbers of dot-product and neural-network computations with flexible low-precision arithmetic, provide power-efficient communications, and provide local storage and decoding of instructions and coefficients. The parallel processing described herein is suitable for neural networks, particularly where power consumption is a concern, such as in battery-powered devices, portable computers, smartphones, wearable computers, smart watches, and the like.

FIG. 1shows a computing device100. The computing device100includes a plurality of banks102of processing elements. The banks102may be operated in a cooperative manner to implement a parallel processing scheme, such as a single instruction, multiple data (SIMD) scheme.

The banks102may be arranged in a regular rectangular grid-like pattern, as illustrated. For sake of explanation, relative directions mentioned herein will be referred to as up, down, vertical, left, right, horizontal, and so on. However, it is understood that such directions are approximations, are not based on any particular reference direction, and are not to be considered limiting.

Any practical number of banks102may be used. Limitations in semiconductor fabrication techniques may govern. In some examples,512banks102are arranged in a 32-by-16 grid.

A bank102may include a plurality of rows104of processing elements (PEs)108and a controller106. A bank102may include any practical number of PE rows104. For example, eight rows104may be provided for each controller106. In some examples, all banks102may be provided with the same or similar arrangement of rows. In other examples, substantially all banks102are substantially identical. In still other examples, a bank102may be assigned a special purpose in the computing device and may have a different architecture, which may omit PE rows104and/or a controller106.

Any practical number of PEs108may be provided to a row104. For example, 256 PEs may be provided to each row104. Continuing the numerical example above, 256 PEs provided to each of eight rows104of 512 banks102means the computing device100includes about 1.05 million PEs108, less any losses due to imperfect semiconductor manufacturing yield.

A PE108may be configured to operate at any practical bit size, such as one, two, four, or eight bits. PEs may be operated in pairs to accommodate operations requiring wider bit sizes.

Instructions and/or data may be communicated to/from the banks102via an input/output (I/O) bus110. The I/O bus110may include a plurality of segments.

A bank102may be connected to the I/O bus110by a vertical bus112. Additionally or alternatively, a vertical bus112may allow communication among banks102in a vertical direction. Such communication may be restricted to immediately vertically adjacent banks102or may extend to further banks102.

A bank102may be connected to a horizontally neighboring bank102by a horizontal bus114to allow communication among banks102in a horizontal direction. Such communication may be restricted to immediately horizontally adjacent banks102or may extend to further banks102.

Communications through any or all of the busses110,112,114may include direct memory access (DMA) to memory of the rows104of the PEs108. Additionally or alternatively, such communications may include memory access performed through the processing functionality of the PEs108.

The computing device100may include a main processor (not shown) to communicate instructions and/or data with the banks102via the I/O bus110, manage operations of the banks102, and/or provide an I/O interface for a user, network, or other device. The I/O bus110may include a Peripheral Component Interconnect Express (PCIe) interface or similar.

FIG. 2shows an example row104including an array of processing elements108, which may be physically arranged in a linear pattern (e.g., a physical row). Each PE108includes an arithmetic logic unit (ALU) to perform an operation, such as addition, multiplication, and so on.

The PEs108are mutually connected to share or communicate data. For example, interconnections200may be provided among the array of PEs108to provide direct communication among neighboring PEs108.

A PE108(e.g., indicated at “n”) is connected to a first neighbor PE108(i.e., n+1) that is immediately adjacent the PE108. Likewise, the PE108(n) is further connected to a second neighbor PE108(n+2) that is immediately adjacent the first neighbor PE108(n+1). A plurality of PEs108may be connected to neighboring processing elements in the same relative manner, where n merely indicates an example PE108for explanatory purposes. That is, the first neighbor PE108(n+1) may be connected to its respective first and second neighbors (n+2 and n+3).

A given PE108(e.g., n+5) may also be connected to an opposite first neighbor PE108(n+4) that is immediately adjacent the PE108(n+5) on a side opposite the first neighbor PE108(n+6). Similarly, the PE108(n+5) may further be connected to an opposite second neighbor PE108(n+3) that is immediately adjacent the opposite first neighbor PE108(n+4).

Further, a PE108may be connected to a fourth neighbor PE108that is immediately adjacent a third neighbor PE108that is immediately adjacent the second neighbor PE108. For example, the PE108designated at n may be connected to the PE designated at n+4. A connection of the PE108(n) to its third neighbor PE108(n+3) may be omitted. The fourth-neighbor connection may also be provided in the opposite direction, so that the PE108(n) connects to its fourth neighbor PE108at n−4 (not shown).

Still further, a PE108may be connected to a sixth neighbor PE108that is immediately adjacent a fifth neighbor PE108that is immediately adjacent the fourth neighbor PE108. For example, the PE108designated at n may be connected to the PE designated at n+6. A connection of the PE108(n) to its fifth neighbor PE108(n+5) may be omitted. The sixth-neighbor connection may also be provided in the opposite direction, so that the PE108(n) connects to its sixth neighbor PE108at n−6 (not shown).

Again, a plurality of PEs108may be connected to neighboring processing elements in the above relative manner. The designation of a PE108as n may be considered arbitrary for non-endmost PEs108. PEs108at the ends of the array may omit certain connections by virtue of the array terminating. In the example of each PE108being connected to its first, second, fourth, and sixth neighbor PEs108in both directions, the six endmost PEs108have differing connections.

With reference toFIG. 3, endmost PEs108at one end of a row104may have connections300to a controller106. Further, endmost PEs108at the opposite end of the row104may have a reduced number of connections302. Additionally or alternatively, end-most PEs108of one bank102may connect in the same relative manner through the controller106and to PEs108of an adjacent bank102. That is, the controller106may be connected between two rows104of PEs108in adjacent banks102, where the two rows104of PEs108are connected in the same manner as shown inFIG. 2

With reference toFIG. 4, a row104of PEs108may include memory400to store data for the row104. A PE108may have a dedicated space in the memory400. For example, each PE108may be connected to a different range of memory cells402. Any practical number of memory cells402may be used. In one example,144memory cells402are provided to each PE108. Note that inFIG. 4the interconnections200among the PEs108and with the controller106are shown schematically for sake of explanation.

The controller106may control the array of PEs108to perform a SIMD operation with data in the memory400. For example, the controller106may trigger the PEs108to simultaneously add two numbers stored in respective cells402.

The controller106may communicate data to and from the memory400though the PEs108. For example, the controller106may load data into the memory400by directly loading data into connected PEs108and controlling PEs108to shift the data to PEs108further in the array. PEs108may load such data into their respective memory cells402. For example, data destined for rightmost PEs108may first be loaded into leftmost PEs and then communicated rightwards by interconnections200before being stored in rightmost memory cells402. Other methods of I/O with the memory, such as direct memory access by the controller106, are also contemplated. The memory cells402of different PEs108may have the same addresses, so that address decoding may be avoided to the extent possible.

Data stored in memory cells402may be any suitable data, such as operands, operators, coefficients, vector components, mask data, selection data, and similar. Mask data may be used to select portions of a vector. Selection data may be used to make/break connections among neighboring PEs108.

Further, the controller106may perform a rearrangement of data within the array of PEs108by controlling communication of data through the interconnections200among the array of PEs108. A rearrangement of data may include a rotation or cycling that reduces or minimizes a number of memory accesses while increasing or maximizing operational throughput. Other examples of rearrangements of data include reversing, interleaving, and duplicating.

In other examples, a set of interconnections200may be provided to connect PEs108in up-down (column-based) connections, so that information may be shared directly between PEs108that are in adjacent rows. In this description, interconnections200and related components that are discussed with regard to left-right (row-based) connections among PEs apply in principle to up-down (column-based) connections among PEs.

FIG. 5shows an array of PEs108and related memory cells402. Each PE108may include local registers500,502to hold data undergoing an operation. Memory cells402may also hold data contributing to the operation. For example, the PEs108may carry out a matrix multiplication, as shown inFIG. 6.

A matrix multiplication may be a generalized matrix-vector multiply (GEMV). A matrix multiplication may use a coefficient matrix and an input vector to obtain a resultant vector. In this example, the coefficient matrix is a four-by-four matrix and the vectors are of length four. In other examples, matrices and vectors of any practical size may be used. In other examples, a matrix multiplication may be a generalized matrix-matrix multiply (GEMM).

As matrix multiplication involves sums of products, the PEs108may additively accumulate resultant vector components d0to d3in respective registers500, while input vector components a0to a3are multiplied by respective coefficients c00to c33. That is, one PE108may accumulate a resultant vector component d0, a neighbor PE108may accumulate another resultant vector component d1, and so on. Resultant vector components d0to d3may be considered dot products. Generally, a GEMV may be considered a collection of dot products of a vector with a set of vectors represented by the rows of a matrix.

To facilitate matrix multiplication, the contents of registers500and/or registers502may be rearranged among the PEs108. A rearrangement of resultant vector components d0to d3and/or input vector components a0to a3may use the direct interconnections among neighbor PEs108, as discussed above. In this example, resultant vector components d0to d3remain fixed and input vector components a0to a3are moved. Further, coefficients c00to c33may be loaded into memory cells to optimize memory accesses.

In the example illustrated inFIG. 5, the input vector components a0to a3are loaded into a sequence of PEs108that are to accumulate resultant vector components do to d3in the same sequence. The relevant coefficients c00, c11, c22, c33are accessed and multiplied by the respective input vector components a0to a3. That is, a0and c00are multiplied and then accumulated as d0, a1and c11are multiplied and then accumulated as d1, and so on.

The input vector components a0to a3are then rearranged, as shown in the PE state sequence ofFIG. 7A, so that a remaining contribution of each input vector components a0to a3to a respective resultant vector components d0to d3may be accumulated. In this example, input vector components a0to a2are moved one PE108to the right and input vector components a3is moved three PEs108to the left. With reference to the first and second neighbor connections shown inFIG. 2, this rearrangement of input vector components a0to a3may be accomplished by swapping a0with a1and simultaneously swapping a2with a3, using first neighbor connections, and then by swapping a1with a3using second neighbor connections. The result is that a next arrangement of input vector components a3, a0, a1, a2at the PEs108is achieved, where each input vector component is located at a PE108that it has not yet occupied during the present matrix multiplication.

Appropriate coefficients c03, c10, c21, c32in memory cells402are then accessed and multiplied by the respective input vector components a3, a0, a1, a2. That is, a3and c03are multiplied and then accumulated as d0, a0and c10are multiplied and then accumulated as d1, and so on.

The input vector components a0to a3are then rearranged twice more, with multiplying accumulation being performed with the input vector components and appropriate coefficients at each new arrangement. At the conclusion of four sets of multiplying accumulation and three intervening rearrangements, the accumulated resultant vector components d0to d3represent the final result of the matrix multiplication.

Rearrangement of the input vector components a0to a3allows each input vector component to be used to the extent needed when it is located at a particular PE108. This is different from traditional matrix multiplication where each resultant vector component is computed to finality prior to moving to the next. The present technique simultaneously accumulates all resultant vector components using sequenced arrangements of input vector components.

Further, such rearrangements of data at the PEs108using the PE neighbor interconnections (FIG. 2) may be optimized to reduce or minimize processing cost. The example given above of two simultaneous first neighbor swaps followed by a second neighbor swap is merely one example. Additional examples are contemplated for matrices and vectors of various dimensions.

Further, the arrangements of coefficients c00to c33in the memory cells402may be predetermined, so that each PE108may access the next coefficient needed without requiring coefficients to be moved among memory cells402. The coefficients c00to c33may be arranged in the memory cells402in a diagonalized manner, such that a first row of coefficients is used for a first arrangement of input vector components, a second row of coefficients is used for a second arrangement of input vector components, and so on. Hence, the respective memory addresses referenced by the PEs108after a rearrangement of input vector components may be incremented or decremented identically. For example, with a first arrangement of input vector components, each PE108may reference its respective memory cell at address 0 for the appropriate coefficient. Likewise, with a second arrangement of input vector components, each PE108may reference its respective memory cell at address 1 for the appropriate coefficient, and so on.

FIG. 7Bshows another example sequence. Four states of a set of PEs108are shown with four sets of selected coefficients. Input vector components a0to a3are rotated so that each component a0to a3is used exactly once to contribute to the accumulation at each resultant vector component d0to d3. The coefficients c00to c33are arranged so that the appropriate coefficient c00to c33is selected for each combination of input vector component a0to a3and resultant vector component d0to d3. In this example, the input vector components a0to a3are subject to the same rearrangement three times to complete a full rotation. Specifically, the input vector component of an nthPE108is moved right to the second neighbor PE108(i.e., n+2), the input vector component of the PE108n+1 is moved left (opposite) to its first neighbor PE108(i.e., n) in that direction, the input vector component of the PE108n+2 is moved right to the first neighbor PE108(i.e., n+3), and the input vector component of the PE108n+3 is moved left to the second neighbor PE108(i.e., n+1).

FIG. 7Cshows a generalized solution, which is implicit from the examples discussed herein, to movement of input vector components among a set of PEs108. As shown by the row-like arrangement700of input vector components a0to ai,which may be held by a row104of PEs108, rotating information may require many short paths702, between adjacent components a0to ai, and a long path704between end-most components ai, and a0. The short paths are not a concern. However, the long path704may increase latency and consume additional electrical power because charging and charging a conductive trace takes time and is not lossless. The longer the trace, the greater the time/loss. The efficiency of a row104of PEs108is limited by its long path704, in that power is lost and other PEs108may need to wait while data is communicated over the long path704.

As shown at710, a circular arrangement of PEs108may avoid a long path704. All paths712may be segments of a circle and may be made the same length. A circular arrangement710of PEs108may be considered an ideal case. However, a circular arrangement710is impractical for manufacturing purposes.

Accordingly, the circular arrangement720may be rotated slightly and flattened (or squashed), while preserving the connections afforded by circular segment paths712and the relative horizontal (X) positions of the PEs, to provide for an efficient arrangement720, in which paths722,724connect adjacent PEs or skip one intermediate PE. As such, PEs108may be connected by a set of first-neighbor paths722(e.g., two end-arriving paths) and a set of second neighbor paths724(e.g., four intermediate and two end-leaving paths) that are analogous to circular segment paths712of a circular arrangement710. The paths722,724have much lower variance than the short and long paths702,704, so power may be saved and latency reduced. Hence, the arrangement720represents a readily manufacturable implementation of an ideal circular arrangement of PEs108.

FIG. 8shows a method900that generalizes the above example. The method900may be performed with the computing device100or a similar device. The method may be implemented by instructions executable by the device100or a controller106thereof. The instructions may be stored in a non-transitory computer-readable medium integral to the device100or controller106.

At block902, operands (e.g., matrix coefficients) are loaded into PE memory cells. The arrangement of operands may be predetermined with the constraint that moving operands is to be avoided where practical. An operand may be duplicated at several cells to avoid moving an operand between such cells.

At block904, operands (e.g., input vector components) are loaded into PE registers. The operands to be loaded into PE registers may be distinguished from the operands to be loaded into PE memory cells, in that there may be fewer PE registers than PE memory cells. Hence, in the example of a matrix multiplication, it may be more efficient to load the smaller matrix/vector to the into PE registers and load the larger matrix into the PE memory cells. In other applications, other preferences may apply.

At block906, a set of memory cells may be selected for use in an operation. The set may be a row of memory cells. For example, a subset of coefficients of a matrix to be multiplied may be selected, one coefficient per PE.

At block908, the same operation is performed by the PEs on the contents of the selected memory cells and respective PE registers. The operation may be performed substantially simultaneously with all relevant PEs. All relevant PEs may be all PEs of a device or a subset of PEs assigned to perform the operation. An example operation is a multiplication (e.g., multiplying PE register content with memory cell content) and accumulation (e.g., accumulating the resulting product with a running total from a previous operation).

Then, if a subsequent operation is to be performed, via block910, operands in the PE registers may be rearranged, at block912, to obtain a next arrangement. A next set of memory cells is then selected at block906, and a next operation is performed at block908. For example, a sequence of memory cells may be selected during each cycle and operands in the PE registers may be rearranged to correspond to the sequence of memory cells, so as to perform a matrix multiplication. In other examples, other operations may be performed.

Hence, a sequence or cycle or operations may be performed on the content of selected memory cells using the content of PE registers that may be rearranged as needed. The method900ends after the last operation, via block910.

The method900may be varied. In various examples, selection of the memory cells need not be made by selection of a contiguous row. Arranging data in the memory cells according to rows may simplify the selection process. For example, a single PE-relative memory address may be referenced (e.g., all PEs refer to their local memory cell with the same given address). That said, it is not strictly necessary to arrange the data in rows. In addition or alternatively, a new set of memory cells need not be selected for each operation. The same set may be used in two or more consecutive cycles. Further, overlapping sets may be used, in that a memory cell used in a former operation may be deselected and a previously unselected memory cell may be selected for a next operation, while another memory cell may remain selected for both operations. In addition or alternatively, the operands in the PE registers need not be rearranged each cycle. Operands may remain in the same arrangement for two or more consecutive cycles. Further, operand rearrangement does not require each operand to change location, in that a given operand may be moved while another operand may remain in place.

FIG. 9shows an example PE108schematically. The PE108includes an ALU1000, registers1002, a memory interface1004, and neighbor PE interconnect control1006.

The ALU1000performs the operational function of the PE. The ALU1000may include an adder, multiplier, accumulator, or similar. In various examples, the ALU1000is a multiplying accumulator. The ALU1000may be connected to the memory interface1004, directly or indirectly, through the registers1002to share information with the memory cells402. In this example, the ALU1000is connected to the memory interface1004though the registers1002and a bus interface1008.

The registers1002are connected to the ALU1000and store data used by the PE108. The registers1002may store operands, results, or other data related to operation of the ALU1000, where such data may be obtained from or provided to the memory cells402or other PEs108via the neighbor PE interconnect control1006.

The memory interface1004is connected to the memory cells402and allows for reading/writing at the memory cells402to communicate data with the registers1002, ALU1000, and/or other components of the PE108.

The neighbor PE interconnect control1006connects to the registers1002and controls communication of data between the registers1002and like registers of neighboring PEs108, for example via interconnections200(FIG. 2), and/or between a controller (see106inFIG. 3). The neighbor PE interconnect control1006may include a logic/switch array to selectively communicate the registers1002to the registers1002of neighboring PEs108, such as first, second, fourth, or sixth neighbor PEs. The neighbor PE interconnect control1006may designate a single neighbor PE108from which to obtain data. That is, the interconnections200may be restricted so that a PE108only at most listens to one selected neighbor PE108. The neighbor PE interconnect control1006may connect PEs108that neighbor each other in the same row. Additionally or alternatively, a neighbor PE interconnect control1006may be provided to connect PEs108that neighbor each other in the same column.

The PE may further include a bus interface1008to connect the PE108to a bus1010, such as a direct memory access bus. The bus interface1008may be positioned between the memory interface1004and registers1002and may selectively communicate data between the memory interface1004and either a component outside the PE108connected to the bus1010(e.g., a main processor via direct memory access) or the registers1002. The bus interface1008may control whether the memory402is connected to the registers1002or the bus1010.

The PE may further include a shifter circuit1012connected to the ALU1000and a wide-add bus1014to perform shifts to facilitate performing operations in conjunction with one or more neighbor PEs108.

FIG. 10shows an example of the neighbor PE interconnect control1006. The neighbor PE interconnect control1006includes a multiplexer1100or similar switch/logic array and a listen register1102.

The multiplexer1100selectively communicates one interconnection200to a neighbor PE108to a register1002used for operations of the PE108to which the neighbor PE interconnect control1006belongs. Hence, a PE108listens to one neighbor PE108.

The listen register1102controls the output of the multiplexer1100, that is, the listen register1102selects a neighbor PE108as source of input to the PE108. The listen register1102may be set by an external component, such as a controller106(FIG. 3), or by the PE108itself.

FIG. 11shows a diagram of an example PE1500and its associated memory1502. The memory1502may be arranged into blocks, so that the PE1500may access one block at the same time that an external process, such as direct memory access, accesses another block. Such simultaneous access may allow for faster overall performance of a row, bank, or other device containing the PE, as the PE and external process can perform operations with different blocks of memory at the same time and there will be fewer occasions of the PE or external process having to wait for the other to complete its memory access. In general, PE access to memory is faster than outside access, so it is expected that the PE1500will be able to perform N memory operations to one block per one outside operation to the other block.

The memory1502includes two blocks1504,1506, each containing an array of memory cells1508. Each block1504,1506may also include a local I/O circuit1510to handle reads/writes to the cells of the block1504,1506. In other examples, more than two blocks may be used.

The memory1502further includes a global I/O circuit1512to coordinate access by the PE and external process to the blocks1504,1506.

The PE1500may include memory access circuits1520-1526, such as a most-significant nibble (MSN) read circuit1520, a least-significant nibble (LSN) read circuit1522, an MSN write circuit1524, and an LSN write circuit1526. The memory access circuits1520-1526are connected to the global I/O circuit1512of the memory1502.

The memory address schema of the blocks1504,1506of memory1502may be configured to reduce latency. In this example, block1504contains cells1508with even addresses and the block1506contains cells1508with odd addresses. As such, when the PE1500is to write to a series of addresses, the global I/O circuit1512connects the PE1500in an alternating fashion to the blocks1504,1506. That is, the PE1500switches between accessing the blocks1504,1506for a sequence of memory addresses. This reduces the chance that the PE1500will have to wait for a typically slower external memory access. Timing between block access can overlap. For example, one block can still be finishing latching data into an external buffer while the other block is concurrently providing data to the PE1500.

FIG. 12shows an example two-dimensional array1600of processing banks102connected to an interface1602via I/O busses1604. The array1600may be grid-like with rows and columns of banks102. Rows need not have the same number of banks102, and columns need not have the same number of banks102.

The interface1602may connect the I/O busses1604to a main processor, such as a CPU of a device that contains the array1600. The interface1602may be a PCIe interface.

The interface1602and buses1604may be configured to communicate data messages1606with the banks102. The interface1602may pump messages through the busses1604with messages becoming accessible to banks102via bus connections1608. A bank102may read/write data from/to a message1606via a bus connection1608.

Each bus1604includes two legs1610,1612. Each leg1610,1612may run between two adjacent columns of banks102. Depending on its column, a given bank102may have bus connections1608to both legs1610,1612of the same bus1604or may have bus connections1608to opposite legs1610,1612of adjacent busses. In this example, even columns (e.g., 0th, 2nd, 4th) are connected to the legs1610,1612of the same bus1604and odd columns (e.g., 1st, 3rd) are connected to different legs1610,1612of adjacent busses1604.

In each bus1604, one end of each leg1610,1612is connected to the interface1602, and the opposite end of each leg1610,1612is connected to a reversing segment1620. Further, concerning the direction of movement of messages on the bus1604, one leg1610may be designated as outgoing from the interface1602and the other leg1612may be designated as incoming to the interface1602. As such, a message1606put onto the bus1604by the interface1602may be pumped along the leg1610, through the reversing segment1620, and back towards the interface1602along the other leg1612.

The reversing segment1620reverses an ordering of content for each message1606, such that the orientation of the content of each message1606remains the same relative to the PEs of the banks102, regardless of which side of the bank102the message1606is on. This is shown schematically as message packets “A,” “B,” and “C,” which are discrete elements of content of a message1606. As can be seen, the orientation of the packets of the message1606whether on the leg1610or the leg1612is the same due to the reversing segment1620. Without the reversing segment, i.e., with a simple loop bus, the orientation of the message1606on the return leg1612would be opposite.

FIG. 13shows another example of a PE2100that may be used with any of the example banks of processing elements discussed herein.

The PE2100includes an ALU2102, an array of resultant registers2104, a resultant selector2105, a hold register2106, a zero disable2110, a switch2112, an input vector register2114, an input zero detector2116, neighbor PE interconnect control2118, and a listen register2120.

The ALU2102implements an operation on data in the input vector register2114and data in memory cells2130associated with the PE2100. Examples of such operations include multiplying accumulation as discussed elsewhere herein. This may include, for example, multiplying matrix coefficients, which may be stored in memory cells2130, by an activation vector, which may be stored in input vector register2114. During such operation, the array of resultant registers2104may accumulate resultant vector components. The ALU2102may include one or more levels of multiplexor and/or a multiplier2108.

Accumulation of results in resultant registers2104may be performed. That is, at a given time, the input vector register2114may be multiplied with selected coefficients from memory cells2130and the products may be accumulated at the resultant registers2104(e.g., a product is added to a value already in a resultant register). As such, for a particular value in the input vector register2114, an appropriate value may be selected from memory cells2130for multiplication and an appropriate resultant register2104may perform the accumulation. This may implement any of the input vector cycling/shuffling described herein, such as discussed with respect toFIGS. 7A-7C.

The resultant selector2105selects a resultant register2104to write to the memory cells2130.

The neighbor PE interconnect control2118may communicate values between the input vector register2114of the PE2100and an input vector register2114of a neighboring PE2100. As such, the neighbor PE interconnect control2118is connected to a like element in neighboring PEs2100via interconnections2132. For example, the neighbor PE interconnect control2118may be connected to first neighbor PEs2100on each side of the PE2100, second neighbor PEs2100on each side of the PE2100, fourth neighbor PEs2100on each side of the PE2100, and/or sixth neighbor PEs2100on each side of the PE2100. When no such neighbor PE exists in the bank to which the PE2100is provided, the neighbor PE interconnect control2118may be connected to respective PEs2100of a neighboring bank and/or to a controller. The neighbor PE interconnect control2118may be configured to rotate or shuffle input vector values as discussed elsewhere herein. The neighbor PE interconnect control2118may connect neighboring PEs2100in a row (left-right) of PEs2100. Additionally or alternatively, a neighbor PE interconnect control2118may connect neighboring PEs2100in a column (up-down) of PEs2100.

The neighbor PE interconnect control2118may include a logic/switch array to selectively communicate values among PEs2100, such as the logic/switch arrays discussed elsewhere herein.

The neighbor PE interconnect control2118may designate a single neighbor PE2100from which to obtain data. That is, interconnections2132with neighbor PEs2100may be restricted so that a PE2100only at most listens to one selected neighbor PE2100. The listen register2120controls from which, if any, PE2100that the neighbor PE interconnect control2118obtains data. That is, the listen register2120selects a neighbor PE2100as the source of input to the PE2100.

The hold register2106may be set to disable computation by the ALU2102. That is, data may be selected from memory2130and moved into/out of input vector register2114while the hold register2106at the same time ensures that the computation is not performed by the PE2100, but may be performed by other PEs in the same row/column.

The zero disable2110controls the inputs to the multiplier2108to be unchanged when detecting that one or both intended inputs to the multiplier2108are zero. That is, should the intended inputs include a zero value for multiplication and accumulation, the zero disable2110holds the present input values as unchanged instead of providing the actual inputs that include the zero value. Multiplication by zero produces a zero product which does not need to be accumulated. As such, the zero disable2110saves energy, as the ALU2102uses significantly more energy when an input changes as opposed to when the inputs do not change.

The switch2112allows a selected resultant register2104or the input vector register2114, via the input zero detector2116, to be written to memory cells2130. The switch2112allows data from the memory cells2130to be written to the listen register2120. The switch2112allows one bit of data to be written to the hold register2106. The switch2112allows data to be written to the input vector register2114through the input zero detector2116. If switch2112is open, then the memory cells2130are connected to the multiplier2108, without being loaded down by inputs of the input vector register2114and input zero detector2116or the resultant selector2105.

The input zero detector2116detects whether the input vector register2114contains a value that is zero. Likewise, the memory cells2130may include OR logic2134to determine whether the selected value in the memory cells2130is zero. The OR logic2134provides an indication of a zero value. As such, when either (or both) of the input vector register2114and the selected value in the memory cells2130is zero, the zero disable2110controls both inputs from the input vector register2114and the selected value in the memory cells2130to appear to the ALU2102to be unchanged, thereby refraining from performing a needless multiplication and accumulation and saving power at the ALU2102.

With reference toFIG. 14, the memory cells2130associated with a PE2100may include blocks2200,2202of memory cells and related caches2204,2206. In this example, a main memory block2200is associated with a cache2204and secondary memory blocks2202are each associated with a cache2206. The caches2204,2206communicate with the PE2100rather than the PE communicating with the memory blocks2200,2202directly.

A cache2204,2206may read/write to its memory block2200,2202an amount of data that is larger than the amount communicated with the PE2100. For example, a cache2204,2206may read/write to its memory block2200,2202in 16-bit units and may communicate with the PE2100in 4-bit units. As such, timing of read/write operations at the memory blocks2200,2202may be relaxed. Thus, it is contemplated that the processing speed of the PE2100will govern operations of the PE2100with the memory cells2130. Clocking of the memory cells2130can be increased to meet the needs of the PE2100.

An isolation switch2208may be provided to isolate secondary memory blocks2202and their caches2206from the PE2100. As such, the PE2100may be selectably connected to a smaller set of memory cells or a larger set of memory cells. When the isolation switch2208is closed, the PE2100may access the main memory block2200and the secondary memory blocks2202, through the caches2204,2206, as a contiguous range of addresses. When the isolation switch2208is opened, the PE2100may only access the main memory block2200, though its cache2204, with the respective reduced range of addresses. Opening the isolation switch2208to reduce the amount of available memory to the PE2100may save energy. The isolation switch2208may be implemented by a switchable bus that connects the secondary caches2206to the PE2100.

In this example, each cache2204,2206includes OR logic to inform the PE2100as to whether the memory value selected by the PE2100is zero. As such, the above-discussed technique of refraining from changing ALU input values may be used to save power.

Further, in this example, the memory cells2130include working or scratch registers2210connected to the PE2100to provide temporary space for intermediate or larger-bit computations.

A memory-sharing switch2214,2218may be provided to connect memory blocks2200,2202of a PE2100to memory blocks2200,2202of a neighboring PE2100. The memory-sharing switch2214,2218may be implemented as a switchable bus that connects the caches2204,2206to respective caches2204,2206of a set of memory cells2130associated with a neighboring PE2100.

As shown inFIG. 15, a left-right (row-based) memory-sharing switch2214may connect PE memory2130to a left/right neighbor PE memory2130. Similarly, an up-down (column-based) memory-sharing switch2218may connect PE memory2130to an up/down neighbor PE memory2130. Any number and combination of such memory-sharing switches2214,2218may be provided to combine memory cells2130associated with individual PEs2100into a combined pool of memory cells2130associated with a group of PEs2100.

Memory-sharing switches2214,2218may be provided between groups of memory cells2130so that a maximum pool of memory cells2130is determined by hardware. Alternatively, memory-sharing switches2214,2218may be provided between all memory cells2130, and firmware or software may govern a maximum pool size, if any.

PEs2100can share memory in groups of two or four, in this example. If a PE fails, it can be skipped. An entire column of PEs can be labelled as bad if a particular PE in the column is bad, so as to avoid having to move data laterally around the bad PE. InFIG. 15, for example, if the top center PE2100had failed, it and both the top and bottom center PEs can be labeled “bad” and be skipped. However, if the rightmost PEs were supposed to be grouped with the center PEs to share memory, this memory can no longer be shared due to the bad PEs. The rightmost PEs can also skip the bad column of PEs and share with the leftmost PEs. This can be achieved by each PE having a DUD register that can be written from SRAM. An application can then initially detect the bad or failed PEs, and set the DUD bits for the entire column of PEs of the broken PE. When SRAM is then to be shared in a group that contains a DUD column, the controller can read the DUD bit and skip over the PE in that column. Hence, if the center column of PEs had their DUD bit set, the rightmost PEs could still share SRAM with the leftmost PEs.

In other examples, if size and power restrictions are a concern, a maximum size of shared PEs may be enforced such as a two-by-two arrangement of four PEs. Groups of four PEs may have hardwired interconnections. In this example, the DUD bit disables an entire block of four PEs.

In still other examples, the hardware may be further simplified and memory sharing may not be omitted. The DUD bit may provide a way of turning off a PE to save power.

With reference back toFIG. 14, caches2204,2206may provide I/O capabilities, such as addressing and address mapping for a pool of memory cells2130as enabled by memory-sharing switches2214,2218. Further, caches2204,2206may be configured to perform copy, move, or other operations as facilitated by memory-sharing switches2214,2218. That is, caches2204,2206may be I/O enabled so as to communicate information with the respective PE2100and further to communicate information with any other caches2204,2206connected by memory-sharing switches2214,2218. As such, data in memory cells2130may be copied, moved, or undergo other operation independent of operation of a PE2100.

Further, the isolation switch2208may allow a PE2100to access its main memory block2200while memory-to-memory operations, enabled by secondary caches2206and memory-sharing switches2214,2218, are performed to share information in secondary memory blocks2202with neighboring memory. This may allow for greater operational flexibility and for reading/writing data to memory2130while allowing PEs2100to continue with their assigned computations.

FIG. 16shows a zero disable circuit2110that may be used in PEs discussed elsewhere herein, such as the PE2100ofFIG. 13. The zero disable circuit2110reduces power consumption used by a multiplier and/or accumulator of the PE. Such power may be saved by disabling the multiplier and/or the accumulator when an input value “a” of an adjacent PE is zero and/or when a coefficient “c” at the present PE is zero. Power savings may be substantial when the PEs are part of a device that processes a neural network, as it is often the case that a significant number (e.g., 50%) of values in such processing are zeros. Further, neural network can be trained to have coefficients “c” that tend to be zero, such that specific training can enhance power savings.

In addition to input values “a” taken from an adjacent PE at2132A and coefficients “c” taken from memory, the zero disable circuit2110also takes respective indicators of whether or not such values “a” and “c” are zero, as indicated by “a=0” and “c=0”, respectively.

Various components of the zero disable circuit2110operate according to a clock (“clk”).

The zero disable circuit2110includes a multiplier2402to multiply input value “a” and coefficient “c”. The multiplier2402includes cascading logic, operates asynchronously, and therefore does not operate according to the clock. The multiplier2402is triggered only if either or both of its inputs, i.e., the input value “a” and the coefficient “c”, change. Hence, if either or both of the inputs “a” and “c” is zero, then the inputs to the multiplier2402are held unchanged, so as to prevent the multiplier2402from computing a zero result that unnecessarily consumes power.

The zero disable circuit2110includes a transparent latch2404. Input to the transparent latch2404includes coefficients “c” from memory associated with the PE (e.g., memory cells2130ofFIG. 13). The transparent latch2404acts as a pass-through when its select line stays high, requiring minimal power. When the select line of the transparent latch2404is lowered, it latches the current value into a fixed state. This uses power, but occurs less frequently than a latch that is set at each clock cycle. The transparent latch2404is set if coefficient “c” or input value “a” is zero, and the multiplier2402consequently does not receive a change in the coefficient “c” value. Rather, the multiplier2402still receives the previous value of the coefficient “c”.

The input value “a” is handled similarly. The zero disable circuit2110includes a register2114to store an input value “a”, as may be received with data from an adjacent PE at2132A. The zero disable circuit2110receives a value for “a” from an adjacent PE and does not latch this value of “a” if either “a” or “c” is zero. If the coefficient “c” is zero and the input value “a” is not zero (a!=0), then the input value “a” is stored, as the input value “a” may still need to be passed to the next adjacent PE at2132B. The zero disable circuit2110includes a shadow register2408to transfer the input value “a” to the next PE at2132B. The shadow register2408has a path parallel to the main register2114that stores the input value “a”. The shadow register2408is used to latch in the input value “a” if either “a” or “c” is zero.

The zero disable circuit2110further includes a demultiplexer2414at the outputs of the main register2114and shadow register2408to select which value to pass to the next PE at2132B.

The shadow register2408is useful when the input value “a” is not zero (a!=0) and the coefficient “c” is zero. However, in this example, signal timing is simplified by also using the shadow register2408for the case where the input value “a” is zero and the coefficient “c” is zero.

When neither the input value “a” nor the coefficient “c” is zero, then the coefficient “c” flows from memory, through the transparent latch2404, and to the multiplier2402. Further, the input value “a” flows from the previous adjacent PE at2132A through the main registers2114, to the multiplier2402, and also to the next adjacent PE at2132B.

When either “a” or “c” are zero, the previous coefficient “c” from memory is held in the transparent latch2404, and the previous input value “a” is held in the main register2114. As such, the inputs to the multiplier2402do not change, and the multiplier2402is therefore not triggered, saving power.

The “a=0” signal from the previous adjacent PE at2132A is held for one clock cycle, and then passed on to the next adjacent PE at2132B.

When the input value “a” is not zero and the coefficient “c” is zero, then the contents of the shadow register2408is selected to be passed on to the next PE at2132B. It is selected by a signal, delayed by one clock cycle, by a delay flipflop2420, which holds the signal for the duration of one clock.

Further, in this example if the input value “a” is zero and the coefficient “c” is also zero, the input value “a” value of zero is latched to the shadow register2408, although it is never used. This simplifies signal timing issues and may further save power. Such a zero value is not passed on to the next PE at2132B, since the demultiplexer signal that selects between the main register2114and the shadow register2408is only triggered for the case where “c=0 and a!=0”. However, the “a=0” signal is passed on. This may save some power, since the previous value of the main register2114is passed on to the next PE at2132B (along with the “a=0” signal that tells the next PE to ignore the actual input value “a”), and since the value has not changed, the signals in the conductors connecting the PEs do not have to change, which would otherwise cost power.

There are various examples where refraining from triggering an ALU can reduce computations and save power. In neural networks, layers of convolutions are often used. After each convolution, a scale factor (e.g., multiplying all the results by a common factor) is often applied to normalize data or to shift the data into a useable range. When using integer math, the scale factor may be performed as two steps: multiplication by a factor, and then a bit-shift. A bit-shift shifts the bits of a result rightwards, discarding the lower least significant bits.

Multiplication and shift values are often known in advance, as with the coefficients. If the multiplier value and shift value are such that some or many of the lower bits will be discarded, this means that some of the least significant terms are never used. Hence, two approaches may be used to save power: (1) a part of the output may be skipped (e.g., the lowest “little ‘a’ x little ‘c’” term) and/or (2) refrain from calculating the lowest term (e.g., “little ‘a’ x little ‘c’”) of the convolution at all. The first may save some cycles. The second may save many cycles. In an example that uses 4-bit PEs to perform 8-bit computations, up to ¼ of the convolution computation may be saved.

With regard to all examples herein, a power-saving floating point representation may be used. Power may be saved when input values “a” and/or coefficients “c” are zero. Such values “a” and “c” may be represented by 4-bit nibbles. For 8-bit (Aa) by 8-bit (Cc) multiplications, multiplications may be performed in four stages, A*C, A*c, a*C, a*c. It is contemplated that Aa and Cc are distributed in some kind of distribution (such as a Gaussian distribution), where values near zero are most common, and the values farther away from zero are less common. Accordingly, values may be quantized, such that if a value is greater or equal to +/−16, it is rounded to the nearest multiple of 16. In this way, a lower nibble for such numbers will always be zero and power will be saved when processing this nibble. In this way, all small values, less than 16, will have their MSNs=0. All large values, greater or equal to 16 will have their LSNs=0. When multiplication is performed on the basis of nibbles, this kind of rounding can be used to force a significant number of nibbles to zero, thereby saving power at the cost of some accuracy. If rounding on the basis of +/−16 causes too much loss in accuracy, then quantization at +/−32 or other value may be used.

FIG. 17shows a cache arrangement2500to facilitate communications among memory allocated to different PEs. The cache arrangement2500includes a plurality of caches2502, each associated with a different set of memory blocks (not shown). The selection of which cache2502and/or memory cells to read/write from/to may be made by activation of row and column lines.

For example, with reference toFIG. 14, each cache2502may be used as a cache2204,2206associated with any suitable number of blocks2200,2202of memory cells. Communications provided for by switches2208,2218in the example ofFIG. 15may be replaced or augmented by the components discussed below.

The caches2502are in mutual communication via a write bus2504and a read bus2506. A write multiplexer2508puts signals onto the write bus2504and a read multiplexer2510takes signals from the read bus2506.

The write multiplexer2508selectively takes input from the read bus2506, a read bus of an adjacent cache arrangement of an adjacent PE above the present PE, at2512, and a read bus of an adjacent cache arrangement of an adjacent PE below the present PE, at2514. As such, the write multiplexer2508may be controlled to write to a cache2502from another cache2502of the same arrangement2500or a cache of a PE in an adjacent row.

The write multiplexer2508also selectively takes data from the PE registers, at2516, so that accumulated results “d” and/or input values “a” may be written to memory.

The write multiplexer2508may be controlled, at selection input2518, by a controller associated with the row or bank of PEs.

The read bus2506takes input from the caches2502and provides same to the write multiplexer2508, a write bus of the adjacent cache arrangement of the adjacent PE above the present PE, at2512, and a write bus of the adjacent cache arrangement of the adjacent PE below the present PE, at2514.

The read multiplexer2510may provide input to the PE registers, at2520, so that the PE may read coefficients “c” and/or write input values “a” from memory.

The cache arrangement2500allows for cache-to-cache communications between memory blocks associated with different PEs as well as blocks associated with the same PE. The cache arrangements2500of a top and/or bottom row of PEs in a bank102(FIG. 1) of PEs may be connected to respective cache arrangements2500of a bottom and/or top row of an adjacent bank102of PEs, so as to facilitate communications among different banks102of PEs. Additionally or alternatively, the cache arrangements2500of a top and/or bottom row of PEs in a bank102may be connected to an I/O bus, such as bus1604(FIG. 12), so as to facilitate communications among different banks of PEs.

The cache arrangement2500may additionally provide redundancy in case of bad memory cells. Each cache2502may serve a number of rows (e.g., 32) of memory cells. A given memory block may be assigned to compensate for bad memory cells in another block. This memory block may have a number of rows (e.g., 2) reserved to replace rows containing bad cells.

A table of virtual registers may be maintained for each cache2502. The table may map logical addresses to caches2502.

FIG. 18is a block diagram of a computing device1800including an array of PEs1802with memory cells1804and cache memory1806and connections there-between. The computing device1800may include any number of PEs1802, each with its own connected blocks of memory cells1804and cache memory1806. Examples of PEs with memory cells and cache have been described with regard toFIGS. 14, 15, and 17, which may be referenced for further description. The techniques described below may be used in the other examples discussed herein, such as those ofFIGS. 14, 15, and 17, and vice versa. The PEs1802may implement a row104of PEs in a bank102, where a plurality of banks102form a computing device, such as discussed elsewhere herein (seeFIG. 1for example).

The PEs1802are mutually connected to share data, such as described elsewhere herein, so as to perform SIMD operations, such as multiplying accumulations. Blocks of memory cells1804are connected to each PE1802to store data related to the SIMD operations, such as coefficients, input/activation values, and accumulated results, performed by the PEs1802. A cache1806is connected to each PE1802to cache data of a respective block of memory cells1804. A PE1802may be connected to its memory cells1804through its cache1806. In this example, each PE1802includes a plurality of caches1806each associated with a different block of memory cells1804. Any number of caches1806and blocks of memory cells1804may be used for a respective PE1802.

A cache (first cache)1806of a given PE (first PE)1802is connected to an adjacent cache (second or third cache)1806of a PE1802that is adjacent the given PE1802. Adjacency may include any one or combination of immediately adjacent, second adjacent, third adjacent, and so on. Such connections may have special cases at or near the ends of the array of PEs. An end of the array at or near a controller1810may have connections with the controller1810.

The connections of caches1806of adjacent PEs1802allow sharing of recently or frequently used data among adjacent PEs1802. While the PEs1802may be mutually connected to share data, such as data stored in PE registers, and described elsewhere herein, it may be useful to provide direct memory communication via the caches1806.

The computing device1800may further include a multiplexer1812connecting each PE1802to its caches1806. The multiplexer1812may implement cache read and/or write functionality between the PE1802and its caches1806and between the caches1806of adjacent PEs1802. The multiplexer1812may include a write multiplexer and/or a read multiplexer, as discussed above with regard toFIG. 17.

Regarding writing, the multiplexer1812(or write multiplexer2508ofFIG. 17) may include an output connected to the cache1806of the PE1802to which the multiplexer1812belongs. The multiplexer1812(or write multiplexer2508) may further include selectable inputs connected to a register1814of the same PE1802and to a cache1806of an adjacent PE1802.

Regarding reading, the multiplexer1812(or read multiplexer2510ofFIG. 17) may include an output connected to a register1814of the PE1802to which the multiplexer1812belongs. The multiplexer1812(or read multiplexer2510) may further include selectable inputs connected to the cache1806and to a cache1806of an adjacent PE1802.

As such the multiplexer1812of a respective PE1802may read from its (first) cache1806and (second, third) caches1806of adjacent PEs1802and provide such data to its PE's registers1814. The multiplexer1812may further write to its (first) cache1806from its PE's registers1814and from (second, third) caches1806of adjacent PEs1802.

The multiplexer1812selection input that determines cache read source and/or write destination may be controlled by a controller1810, which may be a SIMD controller of the row or bank of PEs1802.

The controller1810may control the PEs1802to perform a multiplying accumulation that uses coefficients, input/activation values, and accumulated results. Accordingly, the controller1810may be configured to control a multiplexer1812to write to its (first) cache1806accumulated results and/or input values of the multiplying accumulation. The controller1810may further be configured to control the multiplexer1812to read from its (first) cache coefficients and/or input values of the multiplying accumulation. Input values and/or coefficients, for example, may be shared among adjacent PEs1802via the cache connections provided by the multiplexers1812of the PEs1802and as controlled by the controller1810.

FIG. 19shows a bank2600of PEs108arranged in rows104and connected to a controller2602that may be used to control SIMD operations of the PEs108. The controller2602may be connected to end-most PEs108of each row104, for example, two adjacent end-most PEs108.

The controller2602may be configured to control the PEs108as discussed elsewhere herein.

An ALU2604may be provided to each row104of PEs108. The ALU may be connected to end-most PEs108, for example, two adjacent end-most PEs108. Local registers2606may be provided to each ALU2604.

Each ALU2604may be configured to perform an operation on the respective row104of PEs108. Example operations include move, add, argmax, and maximum/minimum determination. The intermediate and final results of such operation may be stored in the associated registers2606. The ALU2604may be purposefully limited in its operational capacity, so as to reduce complexity, and addition and argmax operations are contemplated to be quite important for neural networks.

Operations on a row104of PEs108, as performed by the ALU2604, may be facilitated by copying data between neighboring PEs108, as discussed elsewhere herein. For example, operations may be performed on data as the data is shifted towards and into the end-most PEs108at the ALU2604.

The controller2602may control the ALUs2604in a SIMD fashion, so that each ALU2604performs the same operation at the same time.

The local registers2606may also be used as staging for reading and writing from/to the respective row104of PEs108.

The controller2602may further include an ALU2608to perform an operation on the results obtained by the row-based ALUs2604. Example operations include move, add, argmax, and maximum/minimum determination. The bank-based ALU2608may include registers2610to store intermediate and final results. As such, results obtained by individual PEs108may be distilled to row-based results, by ALUs2604, that may be further distilled to a bank-based result, by the ALU2608.

As should be apparent from the above discussion, the techniques discussed herein are suitable for low-power neural-network computations and applications. Further, the techniques are capable of handling a large number of computations with flexibility and configurability.

It should be recognized that features and aspects of the various examples provided above can be combined into further examples that also fall within the scope of the present disclosure. In addition, the figures are not to scale and may have size and shape exaggerated for illustrative purposes.