Patent ID: 12210952

DETAILED DESCRIPTION OF THE EMBODIMENTS

In the following detailed description, only certain exemplary embodiments of the present invention have been shown and described, simply by way of illustration. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. Accordingly, the drawings and description are to be regarded as illustrative in nature and not restrictive. Like reference numerals designate like elements throughout the specification.

Through the specification, in addition, unless explicitly described to the contrary, the word “comprise” and variations such as “comprises” or “comprising” will be understood to imply the inclusion of stated elements but not the exclusion of any other elements.

The expressions described in the singular of this specification may be construed as singular or plural unless explicit expressions such as “one” or “singular” are used.

Hereinafter, a reorganizable neural network computing device according to an exemplary embodiment of the present invention will be described.

An exemplary embodiment of the present invention provides a semiconductor device for neural network (NN) computing, and the semiconductor device has a structure that is capable of re-organization.

FIG.1shows a structure of a neural network computing device according to the first exemplary embodiment of the present invention.

The neural network computing device1according to the first exemplary embodiment of the present invention has a structure of a systolic array. Specifically, the neural network computing device1includes, as shown inFIG.1, a plurality of data control lefters (DCL) (11,12,13,14, given a representative reference numeral “10”), a plurality of data control uppers (DCU) (21,22, . . . ,27,28, given a representative reference numeral “20”), and a data processing array unit30which is disposed between the plurality of DCLs10and the plurality of DCUs20.

The plurality of DCUs20are sequentially arranged in a first direction, and the plurality of DCLs10are sequentially arranged in a second direction.

The data processing array unit30has a two-dimensional systolic array structure including a plurality of cells, and includes a plurality of neural cores (NC) (given a representative reference numeral “31”) which are disposed between the plurality of DCLs10arranged in the second direction and the plurality of DCUs20arranged in the first direction, respectively. The first dimension of the data processing array unit30corresponds to a row of the cells and the second dimension of the data processing array unit30corresponds to a column of the cells. The NCs31are disposed at the cell of the two-dimensional systolic array structure, respectively. Each NC31may include an internal register for storing the result of computation, and may also be referred to as “an operator”.

The neural network computing device1stores data for each row of an N matrix in the DCLs10, and stores data for each column of an M matrix in the DCUs20to perform N×M=Y computations. A layer may have a plurality of kernels, and the kernels may have a matrix structure. Kernel data corresponding to each kernel is provided to the DCU20, and for example, the kernel data for the kernel A is provided to a DCU corresponding to the first column, and the kernel data for the kernel B is provided to a DCU corresponding to the second column.

The DCL10is configured to transmit data (also referred to as first input data) for the layers constituting the neural network along the first dimension of the data processing array unit30. The DCU20is configured to transmit data (also referred to as second input data) for the layers constituting the neural network along the second dimensions of the data processing array unit30. The DCL10may be referred to as “a first data input processor”, and the DCU20may be referred to as “a second data input processor”.

Each NC31of the data processing array unit30is configured to compute on data input from the DCL10and data input from the DCU20and store the result. Each NC31stores the computation result itself and accumulates the computation result. That is, a value obtained by adding the computation result of the current computation and the computation result stored according to the previous computation is stored as the resulting value. The resulting value stored in each NC31thus functions as a point in the result matrix.

Each NC31transfers input data to a neighboring NC. More specifically, data input from the DCL10is transferred along the first dimension of the data processing array unit30, and data input from the DCU20is transferred along the second dimension of the data processing array unit30. Accordingly, the data input from the DCL10is sequentially transferred to the NCs arranged in the first dimension of the data processing array unit30, and the data input from the DCL20is sequentially transferred to the NCs arranged in the second dimension of the data processing array unit30.

Here, for convenience, 4 DCLs10, 8 DCUs20, and 32 NCs31are used inFIG.1, but the present invention is not limited thereto.

In the neural network computing device1according to the first exemplary embodiment of the present invention having such a structure, the first input data is output from the DCL10to the NC31one by one. That is, the DCL10outputs the first input data (e.g., data for each row of the N matrix), which has been input and then stored, in the direction (here, a first direction) of the NC31of the data processing array unit30, and then the output first input data proceeds in the right direction (here, the first direction) via the NC31which is disposed at the corresponding row of the data processing array unit30. At this time, the first input data is output one by one from the DCL10per clock cycle, and may proceed in the right direction.

The DCU20sequentially outputs the second input data (e.g., data for each column of the M matrix) one by one, so that the second input data proceeds in the lower direction (here, the second direction) via the NC31of the data processing array unit30.

The first input data sequentially output from the DCL10and the second input data sequentially output from the DCU20are computed by each NC31of the data processing array unit30, and the computation result is stored in the corresponding NC31. Specifically, the first input data and the second input data are processed by NC31, for example, multiplied and accumulated, and the result data is stored in a register inside the NC31.

The result data (also referred to as accumulated data) stored in each NC31may be sequentially output to the DCL10after all data computations have been performed, for instance. The result data output to the DCL10may be stored in an external memory (not shown) via an on-chip memory bus (not shown).

Through the neural network computing device1according to the first exemplary embodiment of the present invention, the matrix computation for the neural network can be performed efficiently at a high speed.

On the other hand, the size of the matrix data for the matrix computation varies depending on the type of the layer. In the case of the convolution layer, as shown inFIG.1, both the DCLs and the DCUs are used for computation of data input in the first direction and the second direction, but only one DCL is used for a fully-connected layer (FCL). When performing a computation for the FCL using the neural network computing device1having the structure ofFIG.1, only one DCL (for example, a DCL0inFIG.1) disposed at one row and a plurality of NCs disposed at the one row are used and the remaining DCLs (e.g., DCL1˜DCL3) and the NCs corresponding to these rows are not used. Thus, some NCs may be wasted, thereby deteriorating the utilization rate of the neural network computing device1.

The second exemplary embodiment of the present invention provides a network computing device having a structure that is capable of re-organization according to a neural network.

FIG.2shows the structure of a neural network computing device according to the second exemplary embodiment of the present invention.

The neural network computing device1according to the second exemplary embodiment of the present invention has a structure of a systolic array. Specifically, the neural network computing device1includes, as shown inFIG.2, a plurality of DCLs (11,12,13,14, given a representative reference numeral “10”), a plurality of DCUs (21,22,23,24, given a representative reference numeral “20”) and a data processing array unit30′ which is disposed between the plurality of DCLs10and the plurality of DCUs20. The plurality of DCUs20are sequentially arranged in a first direction, and the plurality of DCLs10are sequentially arranged in a second direction. Here, a detailed description is omitted for the same portion as the first exemplary embodiment.

The data processing array unit30′ has a two-dimensional systolic array structure including a plurality of cells, and includes a plurality of NCs (given a representative reference numeral “31”) which are disposed between the plurality of DCLs10arranged in the second direction and the plurality of DCUs20arranged in the first direction, respectively. The data processing array unit30may further include a neural core buffer (NCB) (given a representative reference numeral “32”) corresponding to each NC.

The first dimension of the data processing array unit30′ corresponds to the rows of the cells and the second dimensions of the data processing array unit30corresponds to the columns of the cells. The NCs31are disposed at the cells of the two-dimensional systolic array structure, respectively, and the NCB32is disposed adjacent to each NC31. The number of NCBs32may be equal to the number of NCs31.

Each NC31is configured to compute on data input from the DCL10and data input from the DCU20, and store and accumulate the computation result. In addition, each NC31is configured to compute on data input from a corresponding NCB32and the data input from the DCL10, and store and accumulate the computation result.

The DCL10is configured to transfer data (first input data) for the layers constituting the neural network along the first dimension of the data processing array unit30. The DCU20is configured to transfer data (second input data) for the layers constituting the neural network along the second dimensions of the data processing array unit30. Particularly, in the second exemplary embodiment of the present invention, the DCU20outputs the second input data to the NC31disposed at the corresponding column or to the NCB32disposed at the corresponding column, so that the second input data is transferred along the second dimensions of the data processing array unit30. The NCB32is configured to buffer the data provided from the DCU20and transfer the buffered data to the neighboring NC31.

When the computation of N×M=Y is performed on the FCL neural network in the neural network computing device1, the number of rows (N (r)) is 1, and the number of columns (N (c)) is a large number of 16˜1024. Also, the number of rows (M (r)) and the number of columns (M (c)) in the M matrix are large numbers, of 16˜1024. In this case, since the number of kernels corresponding to the M (c) is increased while the N (r) is 1, the data (the first input data) for the N matrix is stored in one DCL (e.g., DCL0) and the data (the second input data) corresponding to each kernel of the M (c) is stored in the NCB32.

In the case of FCL, the size of the NCB32does not have to be as large as the size of the DCU20, as data is not reused, and for example, it may be sized to store eight 16-bit data. The data stored in the NCB32may be data supplied from an external on-chip memory. For example, data (second input data) corresponding to the M (c) for the FCL supplied from an external on-chip memory may be buffered in the NCB32through the DCU20and then provided to the NC31. Alternatively, data corresponding respectively to the M (c) may be provided to the NC31through the DCU20, or buffered to the NCB32via the DCU20and then provided to the NC31. It is difficult for data to be transferred along second dimensions while the data is being scanned at once in the DCU20, and transmission speed may be slower. However, transmission speed delay can be eliminated by using the NCB32.

In this data processing array unit30′, unlike the first exemplary embodiment, a chaining path (given a representative reference numeral “33”) may be formed by units for the plurality of NCs31, which are arranged in the first and second dimensions, so that reorganization is possible in the second exemplary embodiment according to the present invention. For example, a chaining path may be selectively formed for each of two row units.

The chaining path33may be selectively formed to transfer data from the NC of the odd-numbered rows to the NC of the even-numbered rows. Specifically, the chaining path33is formed between the NCs disposed at the even-numbered (e.g., i-th, where i is a positive even number) row and the NC disposed at the previous odd-numbered row (e.g., i−1th). In particular, the NCs which are last disposed at the even-numbered rows and NCs which are last disposed at the previous odd-numbered rows are selectively connected through the chaining path. The NC last disposed at the row represents the NC disposed farthest from the DCL. The fact that an NC disposed at an even-numbered row is connected with an NC disposed at an odd-numbered row through the chaining path means that the NC disposed at the even-numbered row receive data from the NC disposed at the odd-numbered row, and then transfer it to a neighboring NC.

Thus, an NC last disposed at an odd-numbered row may be operated to transfer data input from a neighboring NC that is disposed at the same row to the NC last disposed at an even-numbered row. Further, the NC last disposed at an even-numbered row can be operated to receive data from the NC last disposed at the previous odd-numbered row and to transfer it to a neighboring NC disposed at the same row.

If necessary, one or more chaining paths may be formed in the data processing array unit30′.

When two or more chaining paths are formed per two row units, the NC first disposed in the even-numbered row (the NC closest to the DCL) may be operated to transfer data input from the third direction (e.g., a left direction opposite to the first direction in the right direction), which is opposite to the first direction, that is, data that is input from the neighboring NC in the third direction of the same row to the NC first disposed at the next odd-numbered ((i+1)-th) row. The first disposed NC in the next odd-numbered ((i+1)-th) row may be operated to receive the data from the first disposed NC of the previous even-numbered (i-th) row and transfer it the neighboring NC in the first direction of the same row.

Each NC, based on the input instruction, may receive and transfer data as above, and perform computation on the data. Each NC performs computation on any input data according to the input instruction, and transfers any other input data to a neighboring NC.

In the data processing array unit30′ of the neural network computing device1having such a structure, the chaining path may be selectively formed according to the type of the neural network. For example, a chaining path is not formed during computation for the convolution layer, and a chaining path is formed during computation for the FCL.

Here, for convenience, 4 DCUs10, 4 DCUs20and 16 NCs31are used inFIG.2, but the present invention is not limited thereto.

Based on this structure, the operation of the neural network computing device1according to the second exemplary embodiment of the present invention will be described referring toFIG.2.

When performing a computation for the FCL, a chaining path is formed, but the present invention is not necessarily limited thereto.

For example,FIG.2, a chaining path331may be formed between the NC (31ainFIG.2) last disposed at the odd-numbered first row and the NC (31binFIG.2) last disposed at the even-numbered second row. A chaining path332may be formed between the NC (31cinFIG.2) last disposed at the odd-numbered third row and the NC (31dinFIG.2) last disposed at the even-numbered fourth row.

For computation on the FCL neural network, the data for the N matrix (first input data) is stored in one DCL (e.g., DCL0), and the data corresponding to each kernel of M (c) (second input data) is stored in the DCU20and/or the NCB32, respectively. Here, the DCL1, the DCL2, and the DCL3do not operate.

In this state, the first input data from DCL0is transferred in the right direction (the first direction) through the NCs of the first row along the first dimension and the last NC31bof the second row receives and transfers the data from the last NC31aof the first row to the input. Thus, the data transferred along the NCs of the first row is provided to the last NC31bof the second row via the chaining path331, and then is transferred in the left direction (third direction) along the first dimension through the last NC31bof the second row.

The data transferred in the left direction along the first dimension through the last NC31bof the second row is then transferred to the first NC31c′ of the third row through the first NC31b′ of the second row, and the data transferred to the first NC31cof the third row is moved in the first direction along the first dimension. Thereafter, the data moved in the right direction along the first dimension through the first NC31c′ of the third row is transferred to the last NC31dof the fourth row through the chaining path332by the last NC31cof the third row. The data transferred to the last NC31dmoves along the first dimension in the left direction (third direction) and moves to the first NC31d′ of the fourth row. As above, the data input from DCL0may be processed through all the NCs of the data processing array unit30along the first dimension via chaining paths331and332which are formed by two row units.

Each NC31computes the first input data input from the first direction or the third direction and the data (data input from the DCU20or data from the NCB) input along the second dimension and stores the computation result. The computation result of each NC can be cumulatively processed and subsequently sequentially output to the DCL0.

According to the second exemplary embodiment of the present invention, each NC of the neural network computing device1having a systolic array structure is used for computation of the convolution layer widely used for vision processing and image recognition, and all NCs can also be used efficiently for the computation of the FCL. As a result, the utilization rate of the neural network computing device1having a systolic array structure is improved. In addition, a large matrix computation can efficiently perform convolution computation and FCL computation while optimizing memory access time and computation time.

In an exemplary embodiment of the present invention, instructions for controlling the operation of each NC may be provided to each NC by a controller (e.g., a flow controller (FC), not shown). For example, an NC last disposed at an even-numbered row may determine whether to receive data from the NC last disposed at the previous odd-numbered row, depending on the instruction being entered.

FIG.3shows the structure of the NC according to another exemplary embodiment of the present invention.

As shown inFIG.3, an NC31according to another exemplary embodiment of the present invention includes a computation circuit311for performing a computation, an input interface unit312, an output interface unit313, and a storage314.

The input interface unit312may be configured to provide data for computation to the computation circuit311. For example, the input interface unit312may be configured to provide first input data from the DCL, which is the first data input processor, and second input data from the DCU, which is the second data input processor, to the computation circuit311.

The computation circuit311may be configured to perform computation described referring toFIGS.1and2, as above. The computation circuit311may be configured to perform computation according to an instruction input through the input interface unit312. The computation circuit311may perform the computation of the first input data and the second input data according to the inputted instruction, or may output the first input data or the second input data through the output interface unit313to be transmitted to the other NC.

The storage314may be configured to store a computation result of the computation circuit311, and may be configured in a register form, for example. The computation result is the accumulated resulting value of the computations performed by the computation circuit311.

The output interface unit313may be configured to output data from the computation circuit311. For example, the output interface313may output the first input data or the second input data from the computation circuit311so that the first input data or the second input data is transferred to the other NC. In addition, the output interface unit313may be configured to output the computation result stored in the storage314.

According to the embodiments of the present invention, it is possible to provide a neural network computing device capable of being reorganized according to a layer type of a neural network. Thus, the reorganized systolic array structure can be used to the full extent for the FCL as well as a convolution layer.

In addition, large matrix computation can be used to efficiently perform convolution computation and FCL computation while optimizing memory access time and computation time.

Exemplary embodiments of the present invention may be implemented through a program for performing a function corresponding to a configuration according to an exemplary embodiment of the present invention and a recording medium with the program recorded therein, as well as through the aforementioned apparatus and/or method, and may be easily implemented by one of ordinary skill in the art to which the present invention pertains from the above description of the exemplary embodiments.

While this invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.