Patent Publication Number: US-10331983-B1

Title: Artificial intelligence inference computing device

Description:
FIELD 
     The invention generally relates to the field of machine learning and more particularly to artificial intelligence (AI) inference computing device using Cellular Neural Networks (CNN) based digital integrated circuits (ICs). 
     BACKGROUND 
     Cellular Neural Networks or Cellular Nonlinear Networks (CNN) have been applied to many different fields and problems including, but limited to, image processing since 1988. However, most of the prior art CNN approaches are either based on software solutions (e.g., Convolutional Neural Networks, Recurrent Neural Networks, etc.) or based on hardware that are designed for other purposes (e.g., graphic processing, general computation, etc.). As a result, CNN prior approaches are too slow in term of computational speed and/or too expensive thereby impractical for processing large amount of imagery data. The imagery data can be any two-dimensional data (e.g., still photo, picture, a frame of a video stream, converted form of voice data, etc.). 
     Until now, prior art machine inference has been done by feeding data to a machine learning model through a long-distance cloud connection or by keeping the data and the deep learning model in a single computing system. However the requirement of a single computing system limits the commercial availability of machine learning to only a small fraction of the population, as few people own the well-trained models needed to generate reliable inferences and vast computing power to process large sets of data. Although prior art cloud solution solves the matter of computing power by transferring the data processing to a capable site. It poses two different problems to the client. First, since the data travels to a central warehouse, it is exposed to the people working with it makes it impossible to process data privately through cloud. Second, since the cloud is such a long distance away from the client, the data will be processed at a much slower pace. Therefore, it would be desirable to have an improved artificial intelligence inference computing device that overcomes the problems, shortcoming and defects described above. 
     SUMMARY 
     This section is for the purpose of summarizing some aspects of the invention and to briefly introduce some preferred embodiments. Simplifications or omissions in this section as well as in the abstract and the title herein may be made to avoid obscuring the purpose of the section. Such simplifications or omissions are not intended to limit the scope of the invention. 
     Artificial intelligence inference computing devices are disclosed. According to one aspect of the disclosure, an artificial intelligence inference computing device contains a printed circuit board (PCB) and a number of electronic components mounted thereon. Electronic components include a wireless communication module, a controller module, a memory module, a storage module and at least one cellular neural networks (CNN) based integrated circuit (IC) configured for performing convolutional operations in a deep learning model for extracting features out of input data. Each CNN based IC includes a number of CNN processing engines operatively coupled to at least one input/output data bus. CNN processing engines are connected in a loop with a clock-skew circuit. Wireless communication module is configured for transmitting pre-trained filter coefficients of the deep learning model, input data and classification results. 
     Objects, features, and advantages of the invention will become apparent upon examining the following detailed description of an embodiment thereof, taken in conjunction with the attached drawings. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       These and other features, aspects, and advantages of the invention will be better understood with regard to the following description, appended claims, and accompanying drawings as follows: 
         FIGS. 1A-1B  are block diagrams illustrating an example integrated circuit designed for extracting features from input imagery data in accordance with one embodiment of the invention; 
         FIG. 2  is a function block diagram showing an example controller configured for controlling operations of one or more CNN processing engines according to an embodiment of the invention; 
         FIG. 3  is a diagram showing an example CNN processing engine in accordance with one embodiment of the invention; 
         FIG. 4  is a diagram showing M×M pixel locations within a (M+2)-pixel by (M+2)-pixel region, according to an embodiment of the invention; 
         FIGS. 5A-5C  are diagrams showing three example pixel locations, according to an embodiment of the invention; 
         FIG. 6  is a diagram illustrating an example data arrangement for performing 3×3 convolutions at a pixel location, according to one embodiment of the invention; 
         FIG. 7  is a function block diagram illustrating an example circuitry for performing 3×3 convolutions at a pixel location, according to one embodiment of the invention; 
         FIG. 8  is a diagram showing an example rectification according to an embodiment of the invention; 
         FIGS. 9A-9B  are diagrams showing two example 2×2 pooling operations according to an embodiment of the invention; 
         FIG. 10  is a diagram illustrating a 2×2 pooling operation reduces M-pixel by M-pixel block to a (M/2)-pixel by (M/2)-pixel block in accordance with one embodiment of the invention; 
         FIGS. 11A-11C  are diagrams illustrating examples of M-pixel by M-pixel blocks and corresponding (M+2)-pixel by (M+2)-pixel region in an input image, according to one embodiment of the invention; 
         FIG. 12  is a diagram illustrating an example of a first set of memory buffers for storing received imagery data in accordance with an embodiment of the invention; 
         FIG. 13A  is a diagram showing two operational modes of an example second set of memory buffers for storing filter coefficients in accordance with an embodiment of the invention; 
         FIG. 13B  is a diagram showing example storage schemes of filter coefficients in the second set of memory buffers, according to an embodiment of the invention; 
         FIG. 14  is a diagram showing a plurality of CNN processing engines connected as a loop via an example clock-skew circuit in accordance of an embodiment of the invention; 
         FIG. 15  is a schematic diagram showing an example image processing technique based on convolutional neural networks in accordance with an embodiment of the invention; 
         FIG. 16  is a flowchart illustrating an example process of achieving a trained convolutional neural networks model having bi-valued 3×3 filter kernels in accordance with an embodiment of the invention; 
         FIG. 17  is a diagram showing an example filter kernel conversion scheme in accordance with the invention; 
         FIG. 18  is a diagram showing an example data conversion scheme; 
         FIG. 19  is a function diagram showing salient components of an example artificial intelligence inference computing device in accordance with an embodiment of the invention; 
         FIG. 20A  is a diagram showing a first example data pattern of imagery data and filter coefficients for a CNN based digital IC with four CNN processing engines and two I/O data bus in accordance with one embodiment of the invention; and 
         FIG. 20B  is a diagram showing a second example data pattern of imagery data and filter coefficients for a CNN based digital IC with four CNN processing engines and two I/O data bus in accordance with one embodiment of the invention. 
     
    
    
     DETAILED DESCRIPTIONS 
     In the following description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will become obvious to those skilled in the art that the invention may be practiced without these specific details. The descriptions and representations herein are the common means used by those experienced or skilled in the art to most effectively convey the substance of their work to others skilled in the art. In other instances, well-known methods, procedures, and components have not been described in detail to avoid unnecessarily obscuring aspects of the invention. 
     Reference herein to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the invention. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Further, the order of blocks in process flowcharts or diagrams or circuits representing one or more embodiments of the invention do not inherently indicate any particular order nor imply any limitations in the invention. Used herein, the terms “top”, “bottom”, “right” and “left” are intended to provide relative positions for the purposes of description, and are not intended to designate an absolute frame of reference 
     Embodiments of the invention are discussed herein with reference to  FIGS. 1A-20B . However, those skilled in the art will readily appreciate that the detailed description given herein with respect to these figures is for explanatory purposes as the invention extends beyond these limited embodiments. 
     Referring first to  FIG. 1A , it is shown a block diagram illustrating an example digital integrated circuit (IC)  100  for extracting features out of an input image in accordance with one embodiment of the invention. 
     The integrated circuit  100  is implemented as a digital semi-conductor chip and contains a CNN processing engine controller  110 , and one or more neural networks (CNN) processing engines  102  operatively coupled to at least one input/output (I/O) data bus  120 . Controller  110  is configured to control various operations of the CNN processing engines  102  for extracting features out of an input image based on an image processing technique by performing multiple layers of 3×3 convolutions with rectifications or other nonlinear operations (e.g., sigmoid function), and 2×2 pooling operations. To perform 3×3 convolutions requires imagery data in digital form and corresponding filter coefficients, which are supplied to the CNN processing engine  102  via input/output data bus  120 . It is well known that digital semi-conductor chip contains logic gates, multiplexers, register files, memories, state machines, etc. 
     According to one embodiment, the digital integrated circuit  100  is extendable and scalable. For example, multiple copy of the digital integrated circuit  100  can be implemented on one semiconductor chip. 
     All of the CNN processing engines are identical. For illustration simplicity, only few (i.e., CNN processing engines  122   a - 122   h ,  132   a - 132   h ) are shown in  FIG. 1B . The invention sets no limit on the number of CNN processing engines on a digital semi-conductor chip. 
     Each CNN processing engine  122   a - 122   h ,  132   a - 132   h  contains a CNN processing block  124 , a first set of memory buffers  126  and a second set of memory buffers  128 . The first set of memory buffers  126  is configured for receiving imagery data and for supplying the already received imagery data to the CNN processing block  124 . The second set of memory buffers  128  is configured for storing filter coefficients and for supplying the already received filter coefficients to the CNN processing block  124 . In general, the number of CNN processing engines on a chip is 2 n , where n is an integer (i.e., 0, 1, 2, 3, . . . ). As shown in  FIG. 1B , CNN processing engines  122   a - 122   h  are operatively coupled to a first input/output data bus  130   a  while CNN processing engines  132   a - 132   h  are operatively coupled to a second input/output data bus  130   b . Each input/output data bus  130   a - 130   b  is configured for independently transmitting data (i.e., imagery data and filter coefficients). In one embodiment, the first and the second sets of memory buffers comprise random access memory (RAM). Each of the first and the second sets are logically defined. In other words, respective sizes of the first and the second sets can be reconfigured to accommodate respective amounts of imagery data and filter coefficients. 
     The first and the second I/O data bus  130   a - 130   b  are shown here to connect the CNN processing engines  122   a - 122   h ,  132   a - 132   h  in a sequential scheme. In another embodiment, the at least one I/O data bus may have different connection scheme to the CNN processing engines to accomplish the same purpose of parallel data input and output for improving performance. 
       FIG. 2  is a diagram showing an example controller  200  for controlling various operations of at least one CNN processing engine configured on the integrated circuit. Controller  200  comprises circuitry to control imagery data loading control  212 , filter coefficients loading control  214 , imagery data output control  216 , and image processing operations control  218 . Controller  200  further includes register files  220  for storing the specific configuration (e.g., number of CNN processing engines, number of input/output data bus, etc.) in the integrated circuit. 
     Image data loading control  212  controls loading of imagery data to respective CNN processing engines via the corresponding I/O data bus. Filter coefficients loading control  214  controls loading of filter coefficients to respective CNN processing engines via corresponding I/O data bus. Imagery data output control  216  controls output of the imagery data from respective CNN processing engines via corresponding I/O data bus. Image processing operations control  218  controls various operations such as convolutions, rectifications and pooling operations which can be defined by user of the integrated circuit via a set of user defined directives (e.g., file contains a series of operations such as convolution, rectification, pooling, etc.). 
     More details of a CNN processing engine  302  are shown in  FIG. 3 . A CNN processing block  304  contains digital circuitry that simultaneously obtains M×M convolution operations results by performing 3×3 convolutions at M×M pixel locations using imagery data of a (M+2)-pixel by (M+2)-pixel region and corresponding filter coefficients from the respective memory buffers. The (M+2)-pixel by (M+2)-pixel region is formed with the M×M pixel locations as an M-pixel by M-pixel central portion plus a one-pixel border surrounding the central portion. M is a positive integer. In one embodiment, M equals to 14 and therefore, (M+2) equals to 16, M×M equals to 14×14=196, and M/2 equals 7. 
       FIG. 4  is a diagram showing a diagram representing (M+2)-pixel by (M+2)-pixel region  410  with a central portion of M×M pixel locations  420  used in the CNN processing engine  302 . 
     Imagery data may represent characteristics of a pixel in the input image (e.g., one of the color (e.g., RGB (red, green, blue)) values of the pixel, or distance between pixel and observing location). Generally, the value of the RGB is an integer between 0 and 255. Values of filter coefficients are floating point integer numbers that can be either positive or negative. 
     In order to achieve faster computations, few computational performance improvement techniques have been used and implemented in the CNN processing block  304 . In one embodiment, representation of imagery data uses as few bits as practical (e.g., 5-bit representation). In another embodiment, each filter coefficient is represented as an integer with a radix point. Similarly, the integer representing the filter coefficient uses as few bits as practical. As a result, 3×3 convolutions can then be performed using fixed-point arithmetic for faster computations. 
     Each 3×3 convolution produces one convolution operations result, Out(m, n), based on the following formula: 
                     Out   ⁡     (     m   ,   n     )       =         ∑       1   ≤   i     ,     j   ≤   3         ⁢       In   ⁡     (     m   ,   n   ,   i   ,   j     )       ×     C   ⁡     (     i   ,   j     )           -   b             (   1   )               
where:
         m, n are corresponding row and column numbers for identifying which imagery data (pixel) within the (M+2)-pixel by (M+2)-pixel region the convolution is performed;   In(m,n,i,j) is a 3-pixel by 3-pixel area centered at pixel location (m, n) within the region;   C(i, j) represents one of the nine weight coefficients C(3×3), each corresponds to one of the 3-pixel by 3-pixel area;   b represents an offset coefficient; and   i, j are indices of weight coefficients C(i, j).       

     Each CNN processing block  304  produces M×M convolution operations results simultaneously and, all CNN processing engines perform simultaneous operations. 
       FIGS. 5A-5C  show three different examples of the M×M pixel locations. The first pixel location  531  shown in  FIG. 5A  is in the center of a 3-pixel by 3-pixel area within the (M+2)-pixel by (M+2)-pixel region at the upper left corner. The second pixel location  532  shown in  FIG. 5B  is one pixel data shift to the right of the first pixel location  531 . The third pixel location  533  shown in  FIG. 5C  is a typical example pixel location. M×M pixel locations contain multiple overlapping 3-pixel by 3-pixel areas within the (M+2)-pixel by (M+2)-pixel region. 
     To perform 3×3 convolutions at each sampling location, an example data arrangement is shown in  FIG. 6 . Imagery data (i.e., In(3×3)) and filter coefficients (i.e., weight coefficients C(3×3) and an offset coefficient b) are fed into an example CNN 3×3 circuitry  600 . After 3×3 convolutions operation in accordance with Formula (1), one output result (i.e., Out(1×1)) is produced. At each sampling location, the imagery data In(3×3) is centered at pixel coordinates (m, n)  605  with eight immediate neighbor pixels  601 - 604 ,  606 - 609 . 
       FIG. 7  is a function diagram showing an example CNN 3×3 circuitry  700  for performing 3×3 convolutions at each pixel location. The circuitry  700  contains at least adder  721 , multiplier  722 , shifter  723 , rectifier  724  and pooling operator  725 . In a digital semi-conductor implementation, all of these can be achieved with logic gates and multiplexers, which are generated using well-known methods (e.g., hardware description language such as Verilog, etc.). Adder  721  and multiplier  722  are used for addition and multiplication operations. Shifter  723  is for shifting the output result in accordance with fixed-point arithmetic involved in the 3×3 convolutions. Rectifier  724  is for setting negative output results to zero. Pooling operator  725  is for performing 2×2 pooling operations. 
     Imagery data are stored in a first set of memory buffers  306 , while filter coefficients are stored in a second set of memory buffers  308 . Both imagery data and filter coefficients are fed to the CNN block  304  at each clock of the digital integrated circuit. Filter coefficients (i.e., C(3×3) and b) are fed into the CNN processing block  304  directly from the second set of memory buffers  308 . However, imagery data are fed into the CNN processing block  304  via a multiplexer MUX  305  from the first set of memory buffers  306 . Multiplexer  305  selects imagery data from the first set of memory buffers based on a clock signal (e.g., pulse  312 ). 
     Otherwise, multiplexer MUX  305  selects imagery data from a first neighbor CNN processing engine (from the left side of  FIG. 3  not shown) through a clock-skew circuit  320 . 
     At the same time, a copy of the imagery data fed into the CNN processing block  304  is sent to a second neighbor CNN processing engine (to the right side of  FIG. 3  not shown) via the clock-skew circuit  320 . Clock-skew circuit  320  can be achieved with known techniques (e.g., a D flip-flop  322 ). 
     The first neighbor CNN processing engine may be referred to as an upstream neighbor CNN processing engine in the loop formed by the clock-skew circuit  320 . The second neighbor CNN processing engine may be referred to as a downstream CNN processing engine. In another embodiment, when the data flow direction of the clock-skew circuit is reversed, the first and the second CNN processing engines are also reversed becoming downstream and upstream neighbors, respectively. 
     After 3×3 convolutions for each group of imagery data are performed for predefined number of filter coefficients, convolution operations results Out(m, n) are sent to the first set of memory buffers via another multiplex MUX  307  based on another clock signal (e.g., pulse  311 ). An example clock cycle  310  is drawn for demonstrating the time relationship between pulse  311  and pulse  312 . As shown pulse  311  is one clock before pulse  312 , as a result, the 3×3 convolution operations results are stored into the first set of memory buffers after a particular block of imagery data has been processed by all CNN processing engines through the clock-skew circuit  320 . 
     After the convolution operations result Out(m, n) is obtained from Formula (1), rectification procedure may be performed as directed by image processing control  218 . Any convolution operations result, Out(m, n), less than zero (i.e., negative value) is set to zero. In other words, only positive value of output results are kept.  FIG. 8  shows two example outcomes of rectification. A positive output value 10.5 retains as 10.5 while −2.3 becomes 0. Rectification causes non-linearity in the integrated circuits. 
     If a 2×2 pooling operation is required, the M×M output results are reduced to (M/2)×(M/2). In order to store the (M/2)×(M/2) output results in corresponding locations in the first set of memory buffers, additional bookkeeping techniques are required to track proper memory addresses such that four (M/2)×(M/2) output results can be processed in one CNN processing engine. 
     To demonstrate a 2×2 pooling operation,  FIG. 9A  is a diagram graphically showing first example output results of a 2-pixel by 2-pixel block being reduced to a single value 10.5, which is the largest value of the four output results. The technique shown in  FIG. 9A  is referred to as “max pooling”. When the average value 4.6 of the four output results is used for the single value shown in  FIG. 9B , it is referred to as “average pooling”. There are other pooling operations, for example, “mixed max average pooling” which is a combination of “max pooling” and “average pooling”. The main goal of the pooling operation is to reduce the size of the imagery data being processed.  FIG. 10  is a diagram illustrating M×M pixel locations, through a 2×2 pooling operation, being reduced to (M/2)×(M/2) locations, which is one fourth of the original size. 
     An input image generally contains a large amount of imagery data. In order to perform image processing operations. The input image  1100  is partitioned into M-pixel by M-pixel blocks  1111 - 1112  as shown in  FIG. 11A . Imagery data associated with each of these M-pixel by M-pixel blocks is then fed into respective CNN processing engines. At each of the M×M pixel locations in a particular M-pixel by M-pixel block, 3×3 convolutions are simultaneously performed in the corresponding CNN processing block. 
     Although the invention does not require specific characteristic dimension of an input image, the input image may be required to resize to fit into a predefined characteristic dimension for certain image processing procedures. In an embodiment, a square shape with (2 K ×M)-pixel by (2 K ×M)-pixel is required. K is a positive integer (e.g., 1, 2, 3, 4, etc.). When M equals 14 and K equals 4, the characteristic dimension is 224. In another embodiment, the input image is a rectangular shape with dimensions of (2 I ×M)-pixel and (2 J ×M)-pixel, where I and J are positive integers. 
     In order to properly perform 3×3 convolutions at pixel locations around the border of a M-pixel by M-pixel block, additional imagery data from neighboring blocks are required.  FIG. 11B  shows a typical M-pixel by M-pixel block  1120  (bordered with dotted lines) within a (M+2)-pixel by (M+2)-pixel region  1130 . The (M+2)-pixel by (M+2)-pixel region is formed by a central portion of M-pixel by M-pixel from the current block, and four edges (i.e., top, right, bottom and left) and four corners (i.e., top-left, top-right, bottom-right and bottom-left) from corresponding neighboring blocks. Additional details are shown in  FIG. 12  and corresponding descriptions for the first set of memory buffers. 
       FIG. 11C  shows two example M-pixel by M-pixel blocks  1122 - 1124  and respective associated (M+2)-pixel by (M+2)-pixel regions  1132 - 1134 . These two example blocks  1122 - 1124  are located along the perimeter of the input image. The first example M-pixel by M-pixel block  1122  is located at top-left corner, therefore, the first example block  1122  has neighbors for two edges and one corner. Value “0”s are used for the two edges and three corners without neighbors (shown as shaded area) in the associated (M+2)-pixel by (M+2)-pixel region  1132  for forming imagery data. Similarly, the associated (M+2)-pixel by (M+2)-pixel region  1134  of the second example block  1124  requires “0”s be used for the top edge and two top corners. Other blocks along the perimeter of the input image are treated similarly. In other words, for the purpose to perform 3×3 convolutions at each pixel of the input image, a layer of zeros (“0”s) is added outside of the perimeter of the input image. This can be achieved with many well-known techniques. For example, default values of the first set of memory buffers are set to zero. If no imagery data is filled in from the neighboring blocks, those edges and corners would contain zeros. 
     Furthermore, an input image can contain a large amount of imagery data, which may not be able to be fed into the CNN processing engines in its entirety. Therefore, the first set of memory buffers is configured on the respective CNN processing engines for storing a portion of the imagery data of the input image. The first set of memory buffers contains nine different data buffers graphically illustrated in  FIG. 12 . Nine buffers are designed to match the (M+2)-pixel by (M+2)-pixel region as follows: 
     1) buffer-0 for storing M×M pixels of imagery data representing the central portion; 
     2) buffer-1 for storing 1×M pixels of imagery data representing the top edge; 
     3) buffer-2 for storing M×l pixels of imagery data representing the right edge; 
     4) buffer-3 for storing 1×M pixels of imagery data representing the bottom edge; 
     5) buffer-4 for storing M×1 pixels of imagery data representing the left edge; 
     6) buffer-5 for storing 1×1 pixels of imagery data representing the top left corner; 
     7) buffer-6 for storing 1×1 pixels of imagery data representing the top right corner; 
     8) buffer-7 for storing 1×1 pixels of imagery data representing the bottom right corner; and 
     9) buffer-8 for storing 1×1 pixels of imagery data representing the bottom left corner. 
     Imagery data received from the I/O data bus are in form of M×M pixels of imagery data in consecutive blocks. Each M×M pixels of imagery data is stored into buffer-0 of the current block. The left column of the received M×M pixels of imagery data is stored into buffer-2 of previous block, while the right column of the received M×M pixels of imagery data is stored into buffer-4 of next block. The top and the bottom rows and four corners of the received M×M pixels of imagery data are stored into respective buffers of corresponding blocks based on the geometry of the input image (e.g.,  FIGS. 11A-11C ). 
     An example second set of memory buffers for storing filter coefficients are shown in  FIG. 13A . In one embodiment, a pair of independent buffers Buffer0  1301  and Buffer1  1302  is provided. The pair of independent buffers allow one of the buffers  1301 - 1302  to receive data from the I/O data bus  1330  while the other one to feed data into a CNN processing block (not shown). Two operational modes are shown herein. 
     Example storage schemes of filter coefficients are shown in  FIG. 13B . Each of the pair of buffers (i.e., Buffer0  1301  or Buffer1  1302 ) has a width (i.e., word size  1310 ). In one embodiment, the word size is 120-bit. Accordingly, each of the filter coefficients (i.e., C(3×3) and b) occupies 12-bit in the first example storage scheme  1311 . In the second example storage scheme  1312 , each filter coefficient occupies 6-bit thereby 20 coefficients are stored in each word. In the third example scheme  1313 , 3-bit is used for each coefficient hence four sets of filter coefficients (40 coefficients) are stored. Finally, in the fourth example storage scheme  1314 , 80 coefficients are stored in each word, each coefficient occupies 1.5-bit. In other words, data architecture is flexible depending upon applications. 
     In another embodiment, a third memory buffer can be set up for storing entire filter coefficients to avoid I/O delay. In general, the input image must be at certain size such that all filter coefficients can be stored. This can be done by allocating some unused capacity in the first set of memory buffers to accommodate such a third memory buffer. Since all memory buffers are logically defined in RAM (Random-Access Memory), well known techniques may be used for creating the third memory buffer. In other words, the first and the second sets of memory buffers can be adjusted to fit different amounts of imagery data and/or filter coefficients. Furthermore, the total amount of RAM is dependent upon what is required in image processing operations. 
     When more than one CNN processing engine is configured on the integrated circuit. The CNN processing engine is connected to first and second neighbor CNN processing engines via a clock-skew circuit. For illustration simplicity, only CNN processing block and memory buffers for imagery data are shown. An example clock-skew circuit  1440  for a group of CNN processing engines are shown in  FIG. 14 . The CNN processing engines connected via the second example clock-skew circuit  1440  to form a loop. In other words, each CNN processing engine sends its own imagery data to a first neighbor and, at the same time, receives a second neighbor&#39;s imagery data. Clock-skew circuit  1440  can be achieved with well-known manners. For example, each CNN processing engine is connected with a D flip-flop  1442 . 
     A special case with only two CNN processing engines are connected in a loop, the first neighbor and the second neighbor are the same. 
     Referring now to  FIG. 15 , it is a schematic diagram showing an example image processing technique based on convolutional neural networks in accordance with an embodiment of the invention. Based on convolutional neural networks, multi-layer input imagery data  1511   a - 1511   c  is processed with convolutions using a first set of filters or weights  1520 . Since the imagery data  1511   a - 1511   c  is larger than the filters  1520 . Each corresponding overlapped sub-region  1515  of the imagery data is processed. After the convolutional results are obtained, activation may be conducted before a first pooling operation  1530 . In one embodiment, activation is achieved with rectification performed in a rectified linear unit (ReLU). As a result of the first pooling operation  1530 , the imagery data is reduced to a reduced set of imagery data  1531   a - 1531   c . For 2×2 pooling, the reduced set of imagery data is reduced by a factor of 4 from the previous set. 
     The previous convolution-to-pooling procedure is repeated. The reduced set of imagery data  1531   a - 1531   c  is then processed with convolutions using a second set of filters  1540 . Similarly, each overlapped sub-region  1535  is processed. Another activation can be conducted before a second pooling operation  1540 . The convolution-to-pooling procedures are repeated for several layers and finally connected to a Fully-connected (FC) Layers  1560 . In image classification, respective probabilities of predefined categories can be computed in FC Layers  1560 . 
     This repeated convolution-to-pooling procedure is trained using a known dataset or database. For image classification, the dataset contains the predefined categories. A particular set of filters, activation and pooling can be tuned and obtained before use for classifying an imagery data, for example, a specific combination of filter types, number of filters, order of filters, pooling types, and/or when to perform activation. In one embodiment, convolutional neural networks are based on Visual Geometry Group (VGG16) architecture neural nets, which contains 13 convolutional layers and three fully-connected network layers. 
     A trained convolutional neural networks model is achieved with an example set of operations  1600  shown in  FIG. 16 . At action  1602 , a convolutional neural networks model is first obtained by training the convolutional neural networks model based on image classification of a labeled dataset, which contains a sufficiently large number of input data (e.g., imagery data, converted voice data, optical character reorganization (OCR) data, etc.). For example, there are at least 4,000 data for each category. In other words, each data in the labeled dataset is associated with a category to be classified. The convolutional neural networks model includes multiple ordered filter groups (e.g., each filter group corresponds to a convolutional layer in the convolutional neural networks model). Each filter in the multiple ordered filter groups contains a standard 3×3 filter kernel (i.e., nine coefficients in floating point number format (e.g., standard 3×3 filter kernel  1710  in  FIG. 17 )). Each of the nine coefficients can be any negative or positive real number (i.e., a number with fraction). The initial convolutional neural networks model may be obtained from many different frameworks including, but not limited to, Mxnet, caffe, tensorflow, etc. 
     Then, at action  1604 , the convolutional neural networks model is modified by converting respective standard 3×3 filter kernels  1710  to corresponding bi-valued 3×3 filter kernels  1720  of a currently-processed filter group in the multiple ordered filter groups based on a set of kernel conversion schemes. In one embodiment, each of the nine coefficients C(i,j) in the corresponding bi-valued 3×3 filter kernel  1720  is assigned a value ‘A’ which equals to the average of absolute coefficient values multiplied by the sign of corresponding coefficients in the standard 3×3 filter kernel  1710  shown in following formula: 
     
       
         
           
             
               
                 
                   A 
                   = 
                   
                     
                       ∑ 
                       
                         
                           1 
                           ≤ 
                           i 
                         
                         , 
                         
                           j 
                           ≤ 
                           3 
                         
                       
                     
                     ⁢ 
                     
                       
                          
                         
                           C 
                           ⁡ 
                           
                             ( 
                             
                               i 
                               , 
                               j 
                             
                             ) 
                           
                         
                          
                       
                       / 
                       9 
                     
                   
                 
               
               
                 
                   ( 
                   2 
                   ) 
                 
               
             
           
         
       
     
     Filter groups are converted one at a time in the order defined in the multiple ordered filter groups. In certain situation, two consecutive filter groups are optionally combined such that the training of the convolutional neural networks model is more efficient. 
     Next, at action  1606 , the modified convolutional neural networks model is retrained until a desired convergence criterion is met or achieved. There are a number of well known convergence criteria including, but not limited to, completing a predefined number of retraining operation, converging of accuracy loss due to filter kernel conversion, etc. In one embodiment, all filter groups including already converted in previous retraining operations can be changed or altered for fine tuning. In another embodiment, the already converted filter groups are frozen or unaltered during the retraining operation of the currently-processed filter group. 
     Process  1600  moves to decision  1608 , it is determined whether there is another unconverted filter group. If ‘yes’, process  1600  moves back to repeat actions  1604 - 1606  until all filter groups have been converted. Decision  1608  becomes ‘no’ thereafter. At action  1610 , coefficients of bi-valued 3×3 filter kernels in all filter groups are transformed from a floating point number format to a fixed point number format to accommodate the data structure required in the CNN based integrated circuit. Furthermore, the fixed point number is implemented as reconfigurable circuits in the CNN based integrated circuit. In one embodiment, filter coefficients are implemented using 12-bit fixed point number format. In another embodiment, filter coefficients are implemented using specialized floating point format with 12-bit mantissa and 2-bit exponents. In yet another embodiment, imagery data in activation is implemented with 5-bit mantissa and 4-bit exponents. 
       FIG. 18  is a diagram showing an example data conversion scheme for converting data from 8-bit [0-255] to 5-bit [0-31] per pixel. For example, bits 0-7 becomes 0, bits 8-15 becomes 1, etc. 
     As described in process  1600  of  FIG. 16 , a convolutional neural networks model is trained for the CNN based integrated circuit. The entire set of trained coefficients or weights are pre-configured to the CNN based integrated circuit as a feature extractor for a particular data format (e.g., imagery data, voice spectrum, fingerprint, palm-print, optical character recognition (OCR), etc.). In general, there are many convolutional layers with many filters in each layer. In one embodiment, VGG16 model contains 13 convolutional layers. In a software based image classification task, computations for the convolutional layers take majority of computations (e.g., 90%) traditionally. This computations is drastically reduced with a dedicated hardware such as CNN based IC  100 . 
     For better extracting features in different domains, like speech, face recognition, gesture recognition and etc, different sets of configured convolution layer coefficients are provided for that domain. And the particular set of convolution layers is used as a general feature extractor for the specific tasks in that domain. For example, the specific task of family members face recognition in the domain of face recognition, and the specific task of company employee face recognition also in the same domain of face recognition. And these two specific tasks can share the same set of convolution layers coefficients used for face detection. 
     An example artificial intelligence (AI) inference computing device  1900  is shown in  FIG. 19 . The example AI computing device  1900  contains a printed circuit board (PCB)  1920  with a number of electronic components mounted thereon. Electronic components include a wireless communication interface  1902 , a CNN based integrated circuit (IC)  1904  (e.g., CNN based IC  100  of  FIG. 1A ), a controller  1906 , a memory  1908  and a storage  1910 . Examples of wireless communication interface  1902  may include, but are not limited to, WiFi, Bluetooth®, etc. Examples controller  1906  may include, but are not limited to, central processing unit such as MIPS, ARM, etc. Examples of memory  1908  may include, but are not limited to, Dynamic Random Access Memory, Static Random Access Memory, etc. Examples of storage  1910  may include, but are not limited to, non-volatile memory, etc. 
     CNN based IC  1904  is configured for performing convolutional operations in a deep learning model for extracting features out of input data. Examples of the deep learning model may include, but are not limited to, VGG16, ResNet, MobileNet, etc. Wireless communication interface module  1902  is configured for receiving pre-trained filter coefficients  1941  of the deep learning model and input data  1942  from a smart client device  1930 , and for sending classification results  1951  to the smart client device  1930 . Smart client device  1930  is capable of obtaining input data and transmitting filter coefficients. Examples of smart client device may include, but are not limited to, smart phone, tablet, etc. The deep learning model further include activation and pooling layers. 
     Controller  1906  is configured for loading the pre-trained filter coefficients  1941  of the deep learning model into each CNN based IC  1904 . In addition, controller  1906  is configured for executing the deep learning model on each CNN based IC for the received input data  1942 . Finally, controller  1906  is configured for performing fully-connected layers (e.g., FC Layer  1560  of  FIG. 15 ) from the extracted features out of each CNN based IC  1904  to obtain classification results  1951 . 
     Each CNN based IC  1904  requires specific data structure (i.e., the order of input data and the order of filter coefficients, respectively).  FIGS. 20A and 20B  are diagrams showing two example data structures. 
     Referring now to  FIG. 20A , it is shown the order of convolution operations performed in a first example CNN based digital IC for extracting features out of an input imagery data. The example CNN based digital IC contains four CNN processing engines connected with a clock-skew circuit (e.g., clock-skew circuit  1440  of  FIG. 14 ) and two I/O/data bus. The I/O data bus #I serves CNN processing engines 1 and 2, while the I/O data bus #II serves CNN processing engines 3 and 4. The direction of the data access in the clock-skew circuit is Engine#1→Engine#2→Engine#3→Engine#4→Engine #1. In the first example, the upstream neighbor CNN processing engine for CNN processing engine #1 is CNN processing engine #4. 
     Eight sets of imagery data with 12 filters are used in the first example in  FIG. 20A . Eight sets of imagery data is divided into two imagery data groups with each imagery data group containing 4 sets of imagery data. Filter coefficients of 12 filters are divided into three filter groups each filter groups containing 4 sets of filter coefficients. Each filter group is further divided into two subgroups corresponding to two imagery data groups. Each subgroup contains a portion of the 4 sets of filter coefficients correlating to a corresponding one of the two imagery data groups. 
     The order of the convolution operations for each block of the input image (e.g., block  1111  of the input image  1100  of  FIG. 11A ) starts with a first imagery data group of imagery data (i.e., Im(1), Im(2), Im(3) and Im(4)) being loaded (load-1) to respective CNN processing engines (i.e., Engines #1-4). To perform the convolution operations in cyclic manner based on the connectivity of the clock-skew circuit (e.g., clock-skew circuit  1440  of  FIG. 14 ), filter coefficients of the first portion of the first filter group (i.e., F(i,j) for filters 1-4 correlating to Im(1)-Im(4)) are loaded. The order of the first portion is decided by cyclic access of imagery data from an upstream neighbor CNN processing engine. After four rounds of convolution operations, a second imagery data group (i.e., Im(5), Im(6), Im(7) and Im(8)) is loaded (load-2). Filter coefficient of a second portion of the first filter group (i.e, F(i,j) for filters 1-4 correlating to Im(5)-Im(8)) are loaded and used. After four rounds of convolution operations, the convolution operations results for filters 1-4 are outputted (output-1) and stored into a designated area of the first set of memory buffers of respective CNN processing engines. 
     Then, the convolution operations continue for remaining filter groups. The first imagery data group (i.e., Im(1)-Im(4)) is loaded (load-3) again into respective CNN processing engines. Filter coefficients of the first portion of the second filter group (i.e., F(i,j) for filters 5-8 correlating to Im(1)-Im(4)) are loaded. Four rounds of convolution operations are performed. The second imagery data group (i.e., Im(5)-Im(8)) is loaded (load-4). Filter coefficients of the second portion of the second filter group (i.e., F(i,j) for filters 5-8 correlating to Im(5)-Im(8)) are loaded for four more rounds of convolution operations. Convolution operations results for filters 5-8 are then outputted (output-2). This process continues for filter coefficients of the third filter group (i.e., filters 9-12) again using first and second portions. And the convolution operations results for filters 9-12 are outputted (output-3). 
     The order of convolution operations of a second example CNN based digital IC is shown in  FIG. 20B . The second example IC is the same as the first example IC except the direction of data access in the clock-skew circuit is reversed (i.e., Engine#1→Engine#4→Engine#3→Engine#2→Engine #1). In other words, the upstream neighbor CNN processing engine for CNN processing engine #1 is CNN processing engine #2. As a result, the order of filter coefficients are different. However, the final convolution operations results are the same. 
     There can be other connection schemes to form a loop. Similar to the two examples shown in  FIGS. 20A-20B , corresponding order of filter coefficients can be derived by those having ordinary skill in the art. 
     It is evident from the examples shown in  FIGS. 20A-20B  that any set of filter coefficients can be discarded after an output (i.e., output-1, output-2, output-3). As a results, the filter coefficient may be stored in first-in-first-out manner. However, each group of imagery data must be preserved as they may be reloaded for next set of filters. Since imagery data are stored in RAM (i.e., the first set of memory buffers), reloading operations can be performed with well known techniques. 
     The convolution operations between filter coefficients and imagery data are represented in the following formula:
 
Out( i )=Σ F ( i,j )⊗Im( j )  (3)
 
where
 
F(i,j): filter coefficients of the i-th filter correlating to the j-th imagery data.
 
Im(j): the j-th imagery data.
 
Out(i): the i-th convolution operations result.
 
     In examples shown in  FIGS. 20A-20B , i=1,12 while j=1,8, hence there are 12 Out(i), 8 Im(j) and 12×8=96 F(i,j) filter coefficients. Other combinations of different numbers of imagery data, filters, CNN processing engines and I/O data bus can be similarly derived for those having ordinary skill in the art. If the number of imagery data is not a multiple of the number of CNN processing engines, any unfilled part is filled with zeros. 
     Also, two I/O data bus have been shown in the example connecting to CNN processing engines sequentially (i.e., the first half of the CNN processing engines to the first I/O data bus, the second half of the CNN processing engines to the second I/O data bus). However, I/O data bus may be connected to CNN processing engines differently, for example, in an alternating manner (i.e., CNN processing engines with odd number to the first I/O data bus, the others to the second I/O data bus). 
     Although the invention has been described with reference to specific embodiments thereof, these embodiments are merely illustrative, and not restrictive of, the invention. Various modifications or changes to the specifically disclosed example embodiments will be suggested to persons skilled in the art. For example, whereas the wireless communication interface module has been shown for basing on current technologies, any local short distance data transmission may be used for achieving the same. Additionally, image classification has been shown and described as application in the AI computing device, other types of AI tasks may be used for achieving the same. for example, image detection. In summary, the scope of the invention should not be restricted to the specific example embodiments disclosed herein, and all modifications that are readily suggested to those of ordinary skill in the art should be included within the spirit and purview of this application and scope of the appended claims.