ACCELERATOR CIRCUIT, SEMICONDUCTOR DEVICE, AND METHOD FOR ACCELERATING CONVOLUTION CALCULATION IN CONVOLUTIONAL NEURAL NETWORK

The present disclosure provides an accelerator circuit, a semiconductor device, and a method for accelerating convolution in a convolutional neural network. The accelerator circuit includes a plurality of sub processing-element (PE) arrays, and each of the plurality of sub PE arrays includes a plurality of processing elements. The processing elements in each of the plurality of sub PE arrays implement a standard convolutional layer during a first configuration applied to the accelerator circuit, and implement a depth-wise convolutional layer during a second configuration applied to the accelerator circuit.

BACKGROUND

The present disclosure relates to a machine-learning accelerators, and, in particular, to an accelerator circuit, a semiconductor device, and a method for accelerating convolution calculation in a convolutional neural network (CNN).

Convolutional neural networks have been widely deployed in deep learning applications, such as computer vision applications. However, the scale of workloads of the CNNs has grown larger and larger due to high demands for computation capabilities, and the data transfer between the hardware accelerators and the memory has become the main bottleneck. Moreover, the same hardware accelerator of an existing CNN may be not suitable for both the standard convolution and depth-wise convolution. It may lead to low utilization of processing elements in the hardware accelerators.

DETAILED DESCRIPTION

Further, it will be understood that when an element is referred to as being “connected to” or “coupled to” another element, it can be directly connected to or coupled to the other element, or intervening elements can be present.

Embodiments, or examples, illustrated in the drawings are disclosed as follows using specific language. It will nevertheless be understood that the embodiments and examples are not intended to be limiting. Any alterations or modifications in the disclosed embodiments, and any further applications of the principles disclosed in this document are contemplated as would normally occur to one of ordinary skill in the pertinent art.

Further, it is understood that several processing steps and/or features of a device can be only briefly described. Also, additional processing steps and/or features can be added, and certain of the following processing steps and/or features can be removed or changed while still implementing the claims. Thus, it is understood that the following descriptions represent examples only, and are not intended to suggest that one or more steps or features are required.

FIG.1is a diagram showing operations of a standard convolution in a convolutional layer in accordance with an embodiment of the disclosure.

In an embodiment, in the convolutional layer100, an input activation cube150is applied to every weight cube (e.g., weight cubes110,120,130, and140) so as to perform multiply-accumulate (MAC) operations to generate an output cube160. For example, the weight cubes110,120,130, and140may be regarded as filters, and the input activation cube150may be regarded as activation data. In addition, the number of layers in the weight cubes110,120,130, and140, and the input activation cube150may refer to the number of channels of the activation data.

In this embodiment, the filter is a 3×3 filter, which slides over the activation data by a specific stride (e.g., 1) in a raster scan order (e.g., from top to bottom, and from left to right). For example, in the beginning of the convolutional operation, the weight elements A1 to I1 in array111are respectively multiplied with elements a1, b1, c1, f1, g1, h1, k1, l1, and m1 of the input activation data in window170of array151, and the multiplication products are accumulated to generate a first accumulated value. Similarly, the weight elements A2 to I2 in array112are respectively multiplied with elements a2, b2, c2, f2, g2, h2, k2, l2, and m2 of the input activation data in window170of array152, and the multiplication products are accumulated to generate a second accumulated value. The weight elements A3 to I3 in array113are respectively multiplied with elements a3, b3, c3, f3, g3, h3, k3, l3, and m3 of the input activation data in window170of array153, and the multiplication products are accumulated to generate a third accumulated value. The first accumulated value, the second accumulated value, and the third accumulated value are summed to generate the element OA1in array161of the output cube160.

Then, the filter slides right by the specific stride (e.g., 1). That is, window170is shifted right by one pixel. Accordingly, the weight elements (e.g., A1 to I1, A2 to I2, and A3 to I3) in arrays111,112, and113are multiplied with respective elements b1, c1, d1, g1, h1, i1, l1, m1, and n1 to generate a fourth accumulated value, a fifth accumulated value, and a sixth accumulated value. The fourth accumulated value, the fifth accumulated value, and the sixth accumulated value are summed to generate the element OB1in array161of the output cube160. Accordingly, other elements OC1to OI1in array161can be calculated in a similar fashion.

It should be noted that each of the weight cubes110to140can be regarded as the weight cube in an independent channel, and thus the convolutional calculations of each of the weight cubes120,130, and140with the input activation cube150can be performed independently in a fashion similar to that described so as to obtain elements in arrays162,163, and164.

It should also be noted that the number of layers in the weight cubes110to140and input activation cube150shown inFIG.1is for purposes of description, and it can be changed according to actual needs. The number of layers in the output cube160depends on the volume convolutional operations of the weight cubes110to140and the input activation cube150. Therefore, for a standard convolution, each filter (e.g., weight) is convolved with all of the input's channels to produce a single channel filter output. These outputs are concatenated to obtain the channel output of the standard convolution.

FIG.2is a diagram showing operations of a standard convolution in a convolutional layer in accordance with another embodiment of the disclosure.FIG.3shows the architecture of the hardware circuit of the convolutional layer in accordance with the embodiment ofFIG.2.FIG.4shows the architecture of a sub PE array in the hardware circuit of the convolutional layer in accordance with the embodiment ofFIG.3. Please refer toFIGS.1to4.

Referring toFIG.2, in some embodiments, the convolutional layer100shown inFIG.1can be modified to the convolutional layer200shown inFIG.2. For example, convolutional operations in the convolutional layer200may be based on the reshaped weight and separated input activation. Specifically, arrays211to21N inFIG.2, which are different layers in the same weight cube, may be similar to arrays111to113in the weight cube110inFIG.1. Arrays221to22N inFIG.2, which are different layers in another weight cube, may be similar to arrays121to123in weight cube120inFIG.1, and so on.

In block201, each of arrays211to2N1may be the topmost layer or first layer in the respective weight cube, and the elements A1 to I1 in each of arrays211to2N1are respectively multiplied with the elements a1, b1, c1, f1, g1, h1, k1, l1, and m1 in window271in array251. Similarly, the elements A2 to I2 in each of arrays212to2N2are respectively multiplied with the elements a2, b2, c2, f2, g2, h2, k2, l2, and m2 in window272in array252, and so on.

The multiplication products for arrays211to21N are accumulated to generate the element OA1in array261of the output cube260. The elements OA2to OAMin arrays262to26M in the output cube260can be calculated in a similar manner. Afterwards, the windows271to27N may be shifted by a specific stride (e.g., 1), and similar MAC operations are performed to generate the elements OB1to OBMin array261of the output cube260. Likewise, after the windows271to27N respectively move throughout the arrays251to25N in the raster scan order, all elements in arrays261to26M of the output cube260can be calculated.

It should be noted that the result of the standard convolution using the convolutional layer200inFIG.2is the same as that using the convolutional layer100inFIG.1. In addition, a specific pair of weight and activation data will not require the data from other pairs in the flow of the standard convolution inFIG.2, and thus the dedicated hardware circuit for the convolutional layer200can have local computation and data storage for independent computation.

Attention now is directed toFIG.3, which shows the architecture of the hardware circuit of the convolutional layer200inFIG.2. In an embodiment, the accelerator circuit300may include a memory310, a router320, an activation circuit330, a multiplexer (MUX)340, a demultiplexer (DEMUX)350, an accumulator360, and a processing-element (PE) array370. The PE array370may include a plurality of sub PE arrays380.

The memory310is configured to store the weights for the convolution operations (i.e., including standard convolution and depth-wise convolution). The router320may be configured to distribute the activation data from the activation circuit330to the respective activation buffer383in each sub PE array380. The activation circuit330may be a circuit with collective functions including activation, pooling, batch normalization (BN), and quantization, and thus the activation circuit330can be regarded as an activation/pool/BN/quantizer unit.

Specifically, the activation function of the activation circuit330may be enabled while the functions of pooling, batch normalization, and quantization of the activation circuit330are selectively enabled depending on the operating requirements of the convolutional neural network. For example, in some cases, the functions of pooling, batch normalization, and quantization may be disabled. In some other cases, the functions of pooling, batch normalization, and quantization may be enabled. In yet some other cases, part of the functions of pooling, batch normalization, and quantization may be enabled.

As shown inFIG.3andFIG.4, each of the sub PE arrays380may include a plurality of processing elements381, a weight buffer382, an activation buffer383, and an accumulator384. The processing elements381in each sub PE array are arranged into a two-dimensional systolic array, and each of the processing elements381may perform a multiply-accumulate (MAC) operation. In addition, the number of processing elements381in each row and each column of each sub PE array380may depend on the width of the input activation data and weights. Each of the memory310, the weight buffer382, and the activation buffer383may be implemented by a volatile memory or a non-volatile memory. The volatile memory may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), but the disclosure is not limited thereto.

For example, referring toFIG.4, the data-processing flow of the processing elements381may be weight stationary or activation stationary (input stationary). Given that the weight-stationary data-processing flow is used, upon start of the convolutional operation, the weights are loaded from the memory310to the respective weight buffer382in each sub PE array380, and the activations are loaded from the router320to the respective activation buffer383in each sub PE array380. For example, the weights in each array in blocks201to20N are unrolled and preloaded from the respective weight buffer382to the processing elements381in each sub PE array380.

Then, for each sub PE array380, the weight for each processing element381is loaded from the weight buffer382to the corresponding processing element381. In addition, for each sub PE array380, the activation data for the convolutional operation are sequentially loaded from the activation buffer383to the left-most column of processing elements381in parallel, and the activation data is forwarded to the processing elements381in the next column every one clock cycle.

Accordingly, in each sub PE array380, each processing element381can perform a MAC operation by multiplying the input activation data with the preloaded weight to generate a multiplication product, and adding the multiplication product to the incoming partial sum from the processing element381in the neighboring upper row (i.e., previous row) to generate an output partial sum. In other words, the output partial sums of the processing elements381in a given row, which is not the last row, are transmitted to the processing elements381in the next row. When the given row is the last row, the output partial sums calculated by the processing elements381in the given row are sent to the accumulator384(i.e., a local accumulator in each sub PE array380), and the accumulator384may accumulate output partial sums from the processing elements381in the last row (i.e., bottom row) to generate a partial sum (e.g., partial sums391to394) for the sub PE array380. For purposes of description, there are four partial sums (e.g., partial sums391to394) of the sub PE arrays380labeled onFIG.3.

The partials sum generated by each of sub PE arrays380may be concatenated into a data bus that is input to the demultiplexer350. The multiplexer340and the demultiplexer350are controlled by a control signal CTRL. When the control signal CTRL is in a low logic state, the accelerator circuit300may be used to perform the standard convolution (i.e., standard CONV). In other words, the processing elements381in each sub PE array380implement a standard convolutional layer during a first configuration (i.e., the control signal CTRL is in the low logic state).

When the control signal CTRL is in a high logic state, the accelerator circuit300may be used to perform the depth-wise convolution (i.e., DW CONV). In other words, the processing elements381in each sub PE arrays380implement a depth-wise convolutional layer during a second configuration (i.e., the control signal CTRL is in the high logic state). It should be noted that the concatenated partial sums351and352output by the demultiplexer350may be substantially the same, but they are for different convolution modes. For example, the concatenated partial sum351is for the standard convolution, and the concatenated partial sum352is for the depth-wise convolution.

In response to the control signal CTRL being in the low logic state, the demultiplexer350may output the concatenated partial sum351to the accumulator360so as to perform element-wise accumulation for the standard convolution. For example, the concatenated partial sum351includes the partial sums generated by each of the sub PE arrays380, and each partial sum of the concatenated partial sum351can be regarded as an input element of the accumulator360(i.e., a global accumulator for the accelerator circuit300). Thus, the elements in the concatenated partial sum351(i.e., the partial sums generated by the sub PE arrays380) may be accumulated by the accumulator360to generate an accumulation result361, which is an input of the multiplexer340. At this time, since the control signal CTRL is in the low logic state, the multiplexer340may select the accumulation result361as its output to the activation circuit330.

In response to the control signal CTRL being in the high logic state, the demultiplexer350may output the concatenated partial sum352as a whole to the multiplexer340as another input. At this time, since the control signal CTRL is in the high logic state, the multiplexer340may select the concatenated partial sum352as its output to the activation circuit330.

The activation circuit330may perform activation operations (e.g., alone or with pooling, batch normalization, quantization, or a combination thereof) using the output from the multiplexer340to generate activation data331. Thus, the router320may distribute the activation data from the activation circuit330to the respective activation buffer383in each sub PE array380for computation of the next layer in the convolutional neural network.

More specifically, since each sub PE array380has its own local processing elements381, weight buffer382, activation buffer383, and accumulator384, the architecture of the accelerator circuit300shown inFIG.3can have localized computation within each sub PE array. The activation data and weight are used by its own sub PE array.

FIG.5is a diagram showing the weight-stationary data flow in the hardware circuit of the convolutional layer in accordance with the embodiment ofFIG.3. Please refer toFIGS.2to5.

In an embodiment, the separated activation and weight pairs (e.g., in blocks201to20N inFIG.2) are assigned to different sub PE arrays380. The sub PE arrays380may process the corresponding activation and weight pair in the same fashion such as the weight-stationary method or the activation-stationary method. For purposes of description, the weight-stationary data flow in the accelerator circuit300of the convolutional layer200is shown inFIG.5.

For example, as shown inFIG.5, the sub PE array380at the upper-left position in the accelerator circuit300may process the MAC operations and accumulation operations in block201shown inFIG.2, and the sub PE array380at the bottom-right position in the accelerator circuit300may process the MAC operations and accumulation operations in block20N inFIG.2. In addition, other sub PE arrays380may process the MAC operations in their corresponding blocks inFIG.2.

For example, given that a weight-stationary data flow is used, the weights are preloaded to each processing element381upon start of the standard convolution. Then, the input activations for each window (e.g., window271) in activation array251are fetched from the activation buffer383to the leftmost column of processing element381. Given that window271is shifted by a specific stride of 1 each time, there may be nine locations of window271on array251, and nine combinations of elements are to be fetched from the activation buffer383to the leftmost column of the processing elements381. The certain fetching rule is shown in registers385, for example, the input activations from al to ml (i.e., the top-left window in array251) will be fetched to the left-most column of the processing elements381at 1stcycle, and the input activations from m1 to y1 (i.e., the bottom-right window in array251) will be fetched to the left-most column of the processing elements381at 9thcycle. During this process, the activation data at the left-most column of the processing elements381will be transferred to its neighboring columns of the processing elements381iteratively, till to the right-most column of the processing elements381.

Similarly, there are also nine combinations of the corresponding window on each of arrays252to25N, and nine combinations of elements to be fetched from the activation buffer383to the left-most columns of processing elements381.

FIG.6is a diagram showing operations of a depth-wise convolution in a convolutional layer in accordance with an embodiment of the disclosure.

In an embodiment, a depth-wise convolution may refer to a type of convolution in which a single convolutional filter is applied for each input channel rather than for multiple input channels in the standard convolution. It should be noted that each of the arrays601to60N can be regarded as the input activation of a corresponding channel, and each of the arrays611to61N can be regarded as the weight (i.e., filter) of the corresponding channel. In the depth-wise convolution shown inFIG.6, each of the arrays601to60N is convolved with the respective array in the arrays611to61N. For example, arrays601is convolved with array611to generate an output array621, where window651is shifted by a specific stride (e.g., 1) on the array601each time to perform the MAC operation with the weight (e.g., array611), and the MAC result can be put to the corresponding element OA1in the output array621. Other elements in the output array621can be calculated in a similar manner.

The convolutional operations for other channels can also be performed in a similar manner. The output array of the convolutional operations in each channel can be stacked to obtain the output (e.g., output cube620including output arrays621to62N) of the depth-wise convolution.

FIG.7is a diagram showing the input-stationary data flow in the hardware circuit of the convolutional layer in accordance with the embodiment ofFIG.6. Please refer toFIG.3,FIG.6, andFIG.7.

In an embodiment, the activation and weight pairs in each channel are assigned to different sub PE arrays380of the accelerator circuit300while performing the depth-wise convolution. Each of the sub PE arrays380may process the activation and weight pair in the corresponding channel in the same fashion such as the weight-stationary method or the activation-stationary method. For purposes of description, the input-stationary data flow in the accelerator circuit300of the convolutional layer200that performs the depth-wise convolution is shown inFIG.7. In the input-stationary (i.e., activation stationary) data flow, the input activations are loaded from the activation buffer383to each processing element381in each sub PE array380upon start of the depth-wise convolution. The processing elements381in the leftmost column may receive the input weight from the weight buffer382, and these processing elements381may perform the MAC operation by multiplying the incoming weight by the preloaded activation, and adding the multiplication product to the incoming partial sum to generate an output partial sum. Then, these processing elements381pass the output partial sum to the processing elements381in the next row, and pass the weight to the processing elements381in the next column.

In other words, the output partial sums of the processing elements381in a given row, which is not the last row, are transmitted to the processing elements381in the next row. When the given row is the last row, the output partial sums calculated by the processing elements381in the given row are sent to the accumulator384, and the accumulator384may accumulate the output partial sums from the processing elements381in the last row to generate a partial sum (e.g., partial sums701to704) for the sub PE array380. For purposes of description, there are four partial sums (e.g., partial sums701to704) of the sub PE arrays380labeled onFIG.7.

As depicted inFIG.7, the sub PE array380at the upper-left position in the accelerator circuit300may process the MAC operations for the activation and weight pair in the first channel (e.g., arrays601and611) shown inFIG.6, and the sub PE array380at the bottom-right position in the accelerator circuit300may process the MAC operations for the activation and weight pair in the N-th channel (e.g., arrays60N and61N) inFIG.6. In addition, other sub PE arrays380may process the MAC operations in their corresponding channels inFIG.6.

For example, given that an input-stationary data flow is used, the activation data is preloaded to each processing element381upon start of the depth-wise convolution. Then, the weights for each window (e.g., window651) in the arrays611to61N of the weight cube are fetched from the weight buffer382to the left-most column of processing element381.

More specifically, for the depth-wise convolution, since the processing elements381in each sub PE array380are dedicated for MAC operations of the corresponding channel, the accelerator circuit300can achieve high utilization of the processing elements381in each sub PE array380.

FIG.8is a diagram of a perspective view of the hardware circuit of the convolutional layer in accordance with the embodiment ofFIG.3. Please refer toFIG.3andFIG.8.

In an embodiment, the accelerator circuit300may be implemented by a system-on-chip (SoC) or a system-in-package (SiP). The components other than the weight buffer382in each sub PE array380may be implemented on a die plane800, and the weight buffer382in each sub PE array380may be implemented by another die plane810. For example, the weight buffer382in each sub PE array380can be implemented by a three-dimensional-stacked (3D-stacked) DRAM over the die plane800of the sub PE arrays380. In addition, the proposed structure can be implemented as 2D-IC as well, while 3D-IC could bring more benefits due to shorter interconnect. Also, the memory type is not limited by the design described in this disclosure.

As depicted inFIG.8, the weight buffer382in each sub PE array380is in a top tier, and the input/output (I/O) bonds802of the weight buffer382connected to the I/O bonds805of the processing elements381in each sub PE array380through the TSVs (through-silicon via)804. In other words, each sub PE array380has its own local TSV array (i.e., including the TSVs804) that connects the weight buffer382to the processing elements381in each sub PE array380. The length of the intra-buffer interconnect803(i.e., the distance between the I/O bonds802and the farthest memory cells3821) is shorter in comparison with a global weight buffer for a monolithic PE array, and there is no need to distribute weight and activation data over larger global buffers and the global PE array. In addition, with the assistance of localized high-density SoIC (system of integrated circuits) bonds (e.g., I/O bonds802and805), the accelerator circuit300can transfer the weight and activation data on a smaller scale so as to improve throughput and energy efficiency, thereby achieving fast and energy-efficient data-transfer of weight/activation data, and mitigating the use of larger buffer and long interconnects. Moreover, overall system performance can be improved across various workloads since the architecture shown inFIG.7andFIG.8can support both the standard convolution and depth-wise convolution with high utilization of the processing elements381.

FIG.9shows the architecture of the hardware circuit of the convolutional layer in accordance with the embodiment ofFIG.1. Please refer toFIG.1andFIG.9.

In an embodiment, the data flow of the standard convolution inFIG.1can be implemented using the accelerator circuit900inFIG.9. The accelerator circuit900may include a PE array910, a weight buffer920, an activation buffer930, an accumulator940, and registers950. The PE array910may include a plurality of processing elements911that are arranged in a two-dimensional systolic array. For example, given that a weight-stationary data flow is used, the weights are preloaded to each processing element911(e.g., through arrow921) upon start of the standard convolution. Then, the input activations for each window (e.g., window170) in the arrays151to153of the input activation cube150are fetched from the activation buffer930to the left-most column of the processing elements911, following a certain rule as shown in the registers950.

For example, given that window170is shifted by a specific stride of 1 each time, there may be nine locations of window170on array151, and nine combinations of elements are to be fetched from the activation buffer930to the left-most column of the processing elements911. Similarly, there are also nine combinations of the corresponding window on each of arrays152and153, and nine combinations of elements to be fetched from the activation buffer930to the left-most column of the processing elements911for each of arrays152and153.

Moreover, the weights in the weight cubes110,120,130, and140are unrolled and preloaded into each processing element911. For example, the weights in arrays111,112, and113of the weight cube110are unrolled and preloaded into the left-most column of processing elements911(e.g., weights A1 to I1 of array111, weights A2 to I2 of array112, and weights A3 to I3 of array113). Similarly, the weights in arrays121,122, and123of the weight cube120are unrolled and preloaded into the second column of processing elements911(e.g., weights A1 to I1 of array121, weights A2 to I2 of array122, and weights A3 to I3 of array123). The weights in arrays131,132, and133of the weight cube130are unrolled and preloaded into the third column of processing elements911(e.g., weights A1 to I1 of array131, weights A2 to I2 of array132, and weights A3 to I3 of array133). The weights in arrays141,142, and143of the weight cube140are unrolled and preloaded into the fourth column of processing elements911(e.g., weights A1 to I1 of array141, weights A2 to I2 of array142, and weights A3 to I3 of array143). If additional weight cubes are used, the weights in arrays of the additional weight cubes can be unrolled and preloaded into the subsequent column of processing elements911.

In the accelerator circuit900shown inFIG.9, the weight buffer920and the activation buffer930can be regarded as a global weight buffer and a global activation buffer, respectively. Thus, the weight buffer920and the activation buffer930may be much larger than the weight buffer382and activation383in each sub PE array380shown inFIG.3, and it may indicate that weight and activation data are distributed over large buffers and the PE array910. In other words, the accelerator circuit300shown inFIG.3can transfer the weight and activation data on a smaller scale so as to improve throughput and energy efficiency, thereby achieving fast and energy-efficient data-transfer of weight/activation data, and mitigating the use of larger buffer and long interconnects.

FIG.10is a diagram of a method for accelerating convolution in a convolutional neural network in accordance with an embodiment of the disclosure, the method including the following Steps. Please refer toFIG.3andFIG.10.

Step1010: providing an accelerator circuit comprising a plurality of sub processing-element (PE) arrays, wherein each sub PE array comprises a plurality of processing elements.

Step1020: utilizing the processing elements in each sub PE array to implement a standard convolutional layer during a first configuration applied to the accelerator circuit.

Step1030: utilizing the processing elements in each sub PE array to implement a depth-wise convolutional layer during a second configuration applied to the accelerator circuit.

For example, the first configuration and the second configuration may refer to the standard convolution and the depth-wise convolution, respectively. The control signal CTRL for the multiplexer340and demultiplexer350shown inFIG.3can be switched between the first configuration and the second configuration.

In an embodiment, the present disclosure provides an accelerator circuit for use in a convolutional layer of a convolutional neural network. The accelerator circuit includes a plurality of sub processing-element (PE) arrays, and each of the plurality of sub PE arrays comprises a plurality of processing elements. The processing elements in each of the plurality sub PE arrays implement a standard convolutional layer during a first configuration, and implement a depth-wise convolutional layer during a second configuration.

In another embodiment, the present disclosure provides a semiconductor device. The semiconductor devices includes plurality of sub processing-element (PE) arrays, and each sub PE array comprises a plurality of processing elements and a weight buffer. The processing elements in each sub PE array are implemented on a first die plane, and the weight buffer in each sub PE array is implemented on a second die plane that is on top of the first die plane. The processing elements (381) in each of the plurality sub PE arrays (380) implement a standard convolutional layer during a first configuration, and implement a depth-wise convolutional layer during a second configuration.

In yet another embodiment, the present disclosure provides a method for accelerating convolution in a convolutional neural network. The method includes the following steps: providing an accelerator circuit comprising a plurality of sub processing-element (PE) arrays, wherein each sub PE array comprises a plurality of processing elements; utilizing the processing elements in each sub PE array to implement a standard convolutional layer during a first configuration (S1020); and utilizing the processing elements in each sub PE array to implement a depth-wise convolutional layer during a second configuration.

The methods and features of the present disclosure have been sufficiently described in the provided examples and descriptions. It should be understood that any modifications or changes without departing from the spirit of the present disclosure are intended to be covered in the protection scope of the present disclosure.

Moreover, the scope of the present application is not intended to be limited to the particular embodiments of the process, machine, manufacture, and composition of matter, means, methods and steps described in the specification. As those skilled in the art will readily appreciate from the present disclosure, processes, machines, manufacture, composition of matter, means, methods or steps presently existing or later to be developed, that perform substantially the same function or achieve substantially the same result as the corresponding embodiments described herein, can be utilized according to the present disclosure.

Accordingly, the appended claims are intended to include within their scope: processes, machines, manufacture, compositions of matter, means, methods or steps. In addition, each claim constitutes a separate embodiment, and the combination of various claims and embodiments are within the scope of the present disclosure.