Patent Description:
An artificial intelligence (Al) system may be a computer system that enables a machine to learn and judge and become smart, unlike conventional rule-based smart systems. As used more, the Al system may have an improved recognition rate and may more accurately understand a preference of a user.

Al technology may include machine learning (e.g., deep learning) and element techniques that utilize machine learning. Machine learning may be an algorithm technology that classifies/learns features of input data, and the element techniques may be techniques that implement functions (such as, cognition and judgment) by using machine learning algorithms such as deep learning, and may be implemented in technical fields such as linguistic understanding, visual understanding, inference/prediction, knowledge representation, and motion control.

Artificial intelligence technology may be applied to various fields as follows. Linguistic understanding may be a technique of recognizing and applying/processing language/characters, and may include natural language processing, machine translation, dialogue system, question and answer, and speech recognition/synthesis. Visual understanding may be a technique of recognizing and processing objects like vision, and may include object recognition, object tracking, image retrieval, person recognition, scene understanding, spatial understanding, and image enhancement. Inference/prediction may be a technique of judging information and performing logical inference and prediction, and may include knowledge/probability-based inference, optimization prediction, preference-based planning, and recommendation. Knowledge representation may be a technique of automatically processing human experience information into knowledge data, and may include knowledge construction (data generation/classification) and knowledge management (data utilization). Motion control may be a technique of controlling autonomous driving of a vehicle and movement of a robot, as a non-limiting example, and may include movement control (navigation, collision, driving) and operation control (action control), for example. <CIT> discloses a method and device for controlling data input and output of fully connected network.

The invention is what is claimed in the independent claims.

Although terms of "first" or "second" are used herein to describe various members, components, regions, layers, or sections, these members, components, regions, layers, or sections are not to be limited by these terms.

As used herein, the singular forms are intended to include the plural forms as well, unless the context clearly indicates otherwise.

Unless otherwise defined, all terms, including technical and scientific terms, used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains and based on an understanding of the disclosure of the present application. Terms, such as those defined in commonly used dictionaries, are to be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the disclosure of the present application and are not to be interpreted in an idealized or overly formal sense unless expressly so defined herein. The use of the term "may" herein with respect to an example or embodiment (e.g., as to what an example or embodiment may include or implement) means that at least one example or embodiment exists where such a feature is included or implemented, while all examples are not limited thereto.

The examples may be implemented as, or with, various types of products such as, for example, a personal computer, a laptop computer, a tablet computer, a smart phone, a television, a smart home appliance, an intelligent vehicle, a kiosk, and a wearable device. Hereinafter, the examples will be described in detail with reference to the accompanying drawings, wherein like drawing reference numerals are used for like elements.

<FIG> illustrates an example of a method of training a neural network.

Referring to <FIG>, a neural network <NUM> may include an input layer <NUM>, hidden layers <NUM>, and an output layer <NUM>. In <FIG>, the neural network <NUM> may be a fully connected network that classifies and outputs information included in input data is illustrated. In detail, if the input data is image data, the neural network <NUM> may output, as output data, result data obtained by classifying types of image objects included in the image data.

The plurality of layers forming the neural network <NUM> may each include a plurality of nodes (for example, nodes <NUM>) that receive data. Two neighboring layers may be connected by a plurality of edges or connections (for example, edges <NUM>) as shown in <FIG>. Each of the nodes may include a weight, and the neural network <NUM> may determine the output data based on a value determined by performing an operation (for example, a multiplication operation) between an input signal and the weight.

Referring to <FIG>, the input layer <NUM> may receive the input data, for example, input data <NUM> including a cat as an image object.

Further, the neural network <NUM> may include a first edge layer <NUM> formed between the input layer <NUM> and a first hidden layer, a second edge layer <NUM> formed between the first hidden layer and a second hidden layer, a third edge layer <NUM> formed between the second hidden layer and a third hidden layer, and a fourth edge layer <NUM> formed between the third hidden layer and the output layer <NUM>.

The plurality of nodes included in the input layer <NUM> of the neural network <NUM> may receive signals corresponding to the input data <NUM>. Through the operations at the plurality of layers included in the hidden layers <NUM>, the output layer <NUM> may output output data <NUM> corresponding to the image data <NUM>. In an example of <FIG>, the neural network <NUM> may output the output data <NUM> of "Cat probability: <NUM>%" by performing operations to classify types of image objects included in an input image. To increase the accuracy of the output data <NUM> output from the neural network <NUM>, weights may be corrected to increase the accuracy of output data by performing learning or training in a direction from the output layer <NUM> to the input layer <NUM> (e.g., through one or more learning techniques such as backpropagation of non-dropped out remaining nodes of a neural network).

As described above, the neural network <NUM> may adjust connection weights of one or more nodes in a layer through learning. In an example, overfitting may occur during the process of adjusting the weights. Overfitting may refer to a situation in which the output accuracy with respect to newly input data decreases due to excessive concentration on training data. To solve such overfitting issues, an operation using dropout or pruning may be used. The operation such as dropout or pruning may be a technique that improves the learning performance by omitting operations (e.g., operations determined to be unnecessary) in a neural network.

<FIG> illustrates an example of omitting an operation (e.g., an operation determined to be unnecessary) in a neural network to improve the learning performance.

Referring to <FIG>, a fully connected neural network <NUM> and a partially connected neural network <NUM> are illustrated. The partially connected neural network <NUM> may have fewer nodes and fewer edges than the fully connected neural network <NUM>. For example, the partially connected neural network <NUM> may be a network to which dropout is applied.

A model combination may be used to improve the learning performance of the fully connected neural network <NUM>. For the model combination, training may be performed using different training data, or models may have different structures. However, when deep networks are used, training one or more networks to accurately estimate, interpret, or classify different types of image objects may include training a plurality of networks (e.g., wherein each of the networks is trained based on a respective type of image object), which may include performing a large amount of computation. To reduce the amount of computation performed to train one or more networks to accurately classify different types of image objects, dropout may omit a portion of neurons at random during a learning cycle of a network, rather than training the plurality of networks. In this example, training a network using dropout configures the network to accurately classify different types of image objects, such as training exponentially various models, as many as combinations of the omitted neurons is produced, and thus the effect of model combination is achieved.

Referring to <FIG>, the partially connected neural network <NUM> may have fewer edges than the fully connected neural network <NUM>. Thus, the partially connected neural network <NUM> may include multiple bit values of "<NUM>" indicating "disconnection" in an edge sequence indicating connections among nodes.

Hereinafter, practical methods of omitting operations (e.g., operations determined to be unnecessary) in the neural network operation will be described in detail. According to one or more embodiments of the present disclosure, multiplication of a matrix and a vector used in the operation process of a neural network (for example, a fully connected network) may be performed quickly at low power.

<FIG> illustrates an example of a control apparatus.

Referring to <FIG>, a control apparatus <NUM> may include a memory <NUM>, an encoding circuit <NUM>, and a decoding circuit <NUM>. The control apparatus <NUM> may be connected to a neural network circuit <NUM> that performs a deep learning operation of a neural network. The control apparatus <NUM> may receive information output during the operation process of the neural network circuit <NUM> and transmit the information generated by the control apparatus <NUM> to the neural network circuit <NUM>.

The neural network circuit <NUM> may perform operations through the neural network including an input layer, a hidden layer, and an output layer. Here, the hidden layer may include a plurality of layers, for example, a first layer, a second layer, and a third layer. Non-limiting example operations of the neural network performed by the neural network circuit <NUM> are described above with reference to <FIG>.

The control apparatus <NUM> may receive data from the neural network circuit <NUM> and output data to the neural network circuit <NUM> in a first-in, first-out (FIFO) manner. The neural network circuit <NUM> may process information in a unit of each layer of the neural network. In an example, there may be a waiting time for each layer during the information processing process of the neural network circuit <NUM>. For example, a result of an operation of the first layer may be processed or reprocessed for a predetermined waiting time after the operation of the first layer. The process of processing or reprocessing the operation result may be performed through the control apparatus <NUM>. The control apparatus <NUM> may process the operation result of the first layer and transmits the processed operation result to the neural network circuit <NUM>. The processed operation result received by the neural network circuit <NUM> from the control apparatus <NUM> may be used for an operation of the second layer. The control apparatus <NUM> may sequentially receive data from the neural network circuit <NUM> and sequentially output processed data to the neural network circuit <NUM>.

The neural network circuit <NUM> may perform the operations of the neural network. For example, the neural network may be a fully connected network. Nodes included in each layer of the fully connected network may have weights. In the fully connected network, a signal input to a current layer may be output (e.g., to a subsequent layer) after an operation with a weight matrix (for example, a multiplication operation). Here, the signal input into the current layer may be matrix data in the size of N × <NUM> (N denoting the number of nodes of the current layer). Further, a weight matrix multiplied by the signal input into the current layer may be matrix data in the size of M × N (M denoting the number of nodes of a layer subsequent to the current layer; N denoting the number of nodes of the current layer). A signal output from the current layer may be input into the subsequent layer. Here, the signal output from the current layer may be input into the subsequent layer through the control by the control apparatus <NUM>. For example, the signal output from the current layer may be processed by the control apparatus <NUM> and the processed signal may be input into the subsequent layer.

The memory <NUM> may store a sequence or a bitstream having a predetermined size.

The sequence may be a sequence including information related to an input feature map and/or a sequence including information related to a weight of a filter.

For example, the sequence may include information regarding whether nodes constituting a plurality of layers of the neural network are connected by edges. In detail, the sequence may include information indicating connections or disconnections of a plurality of edges formed in a layer included in the neural network. For example, referring to <FIG>, the sequence may include information related to an edge sequence indicating connections of the plurality of edges <NUM> included in a predetermined layer, for example, the first layer <NUM>.

A bit value of each bit string of the sequence may indicate a connection strength of a predetermined edge. For example, a greater bit value may indicate a higher connection strength of a predetermined edge, and a smaller bit value may indicate a lower connection strength of the predetermined edge. Hereinafter, information, as a sequence, indicating connection strengths of predetermined edges may be referred to as a "data sequence".

The sequence may include information related to a sequence that distinguishes a valid bit and an invalid bit in a bit string of the data sequence. For example, a value "<NUM>" included in the bit string of the sequence may indicate that a bit corresponding to an address of a corresponding bit in the data sequence is an invalid bit. Further, a value "<NUM>" included in the bit string of the sequence may indicate that a bit corresponding to an address of a corresponding bit in the data sequence is a valid bit. Whether a bit in the data sequence is valid or invalid may be determined by comparing a size of the bit to a predetermined threshold value. Hereinafter, a sequence that determines a valid bit and an invalid bit in the bit string of the data sequence may be referred to as a "validity determination sequence".

The memory <NUM> may store the data sequence and/or the validity determination sequence described above. The data sequence may be compressed and stored in the memory <NUM> in the form of a compressed data sequence. Non-limiting examples of the data sequence, the compressed data sequence, and the validity determination sequence will be described in detail later with reference to <FIG>.

When the neural network circuit <NUM> terminates or completes an operation of a predetermined layer, the control apparatus <NUM> may receive an operation result of the layer from the neural network circuit <NUM>. In an example, the operation result for the layer may be a data sequence for the layer.

The encoding circuit <NUM> may process the data sequence received by the control apparatus <NUM> and store the processed data sequence in the memory <NUM>. For example, the processed sequence may be a compressed sequence obtained by compressing the data sequence. Further, for example, the processed sequence may be a validity determination sequence that distinguishes a valid bit and an invalid bit in a bit string of the compressed data sequence. The encoding circuit <NUM> may generate a processed sequence corresponding to an operation cycle of the neural network circuit <NUM>. The encoding circuit <NUM> may write the processed sequence to the memory <NUM>. The compressed sequence may include fewer bit strings than a sequence before compression, and thus the encoding circuit <NUM> may reduce the number of writes to the memory <NUM>. Thus, due to the reduction in the number of writes, the power consumption of the control apparatus <NUM> may be advantageously reduced by the control apparatus <NUM> of one or more embodiments. Accordingly, the control apparatus <NUM> of one or more embodiments may improve the technical field of neural network training by reducing power consumption used by the control apparatus <NUM> to train a neural network through dropout, compared to a typical control apparatus.

The decoding circuit <NUM> may transmit the processed sequence generated by the encoding circuit <NUM> to the neural network circuit <NUM>, such that the neural network circuit <NUM> may determine (or redetermine) a connection state (e.g., a connection strength) of an edge in the neural network. The decoding circuit <NUM> may read the processed sequence from the memory <NUM>, such that the control apparatus <NUM> may sequentially output bit strings in the processed sequence. The compressed sequence may include fewer bit strings than a sequence before compression, and thus the decoding circuit <NUM> may reduce the number of reads from the memory <NUM>. Thus, due to the reduction in the number of reads, the power consumption of the control apparatus <NUM> may be advantageously reduced by the control apparatus <NUM> of one or more embodiments. Accordingly, the control apparatus <NUM> of one or more embodiments may improve the technical field of neural network training by reducing the power consumption used by the control apparatus <NUM> to train a neural network through dropout, compared to a typical control apparatus.

Further, the decoding circuit <NUM> may determine a bit to be transmitted to the neural network circuit in the bit string of the compressed data sequence, such that the neural network circuit omits an operation with respect to non-consecutive invalid bits. When the operation with respect to the non-consecutive invalid bits is omitted, the decoding circuit <NUM> may advantageously improve the operation processing rate. Non-limiting example operations of the decoding circuit <NUM> omitting the operation with respect to the non-consecutive invalid bits will be described in detail later with reference to <FIG>.

<FIG> illustrates an example of a sequence generated by an encoding circuit (e.g., the encoding circuit <NUM> of <FIG>).

Referring to <FIG>, examples of a data sequence <NUM>, compressed data sequences <NUM> and <NUM>, and validity determination sequences <NUM> and <NUM> are illustrated.

The data sequence <NUM> may include information indicating a connection strength of predetermined edges. The data sequence <NUM> may include a bit string. A great bit value of a bit included in the bit string may indicate a high connection strength of predetermined edges, and a small bit value may indicate a low connection strength of predetermined edges.

The data sequence <NUM> may include valid bits and invalid bits. Whether a bit in the data sequence <NUM> is valid or invalid may be determined by comparing a size of the bit to a predetermined threshold value. When the bit in the data sequence <NUM> has a value less than or equal to the threshold value, the bit may be determined to be invalid. Being invalid may indicate that edges corresponding to the bit are disconnected. When a bit having a value less than or equal to the predetermined threshold value is determined to be invalid, computations using the bit may be determined as unnecessary to improve learning performance using pruning or dropout, and therefore such computations may be omitted from the pruning or dropout.

The encoding circuit <NUM> may generate the compressed data sequences <NUM> and <NUM> in which consecutive invalid bits in the bit string of the data sequence <NUM> are compressed into a single bit.

In an example of generating the compressed data sequence <NUM> and the validity determination sequences <NUM>, when the predetermined threshold value is "<NUM>", a bit having a value less than or equal to "<NUM>" in the bit string of the data sequence <NUM> may be determined as invalid, and a bit having a value greater than "<NUM>" in the bit string of the data sequence <NUM> may be determined as valid. Further, the data sequence <NUM> may include consecutive bits having values less than or equal to the threshold value "<NUM>". When there are consecutive bits having values less than or equal to the threshold value "<NUM>", the encoding circuit <NUM> may generate the compressed data sequence <NUM> by expressing the consecutive bits with a single bit value. In an example, the single bit value may indicate the number of consecutive bits having values less than or equal to the threshold value "<NUM>" in the data sequence <NUM>. For example, when the data sequence <NUM> includes three consecutive bits having values less than or equal to the threshold value "<NUM>", such as "<NUM>", "<NUM>" of the data sequence <NUM> may be expressed as "<NUM>" in the compressed data sequence <NUM>. The encoding circuit <NUM> may compress consecutive invalid bits into a single bit as described above, thereby improving the operation speed of the neural network circuit <NUM>. Further, when a bit of the data sequence <NUM> is greater than the predetermined threshold value "<NUM>", the bit may be included in the compressed data sequence <NUM>. Accordingly, the encoding circuit <NUM> may compress the data sequence <NUM> of "<NUM>" to generate the compressed data sequence <NUM> of "<NUM>".

In an example of generating the compressed data sequence <NUM> and the validity determination sequences <NUM>, when the predetermined threshold value is "<NUM>", a bit having a value less than or equal to "<NUM>" in the bit string of the data sequence <NUM> may be determined as invalid, and a bit having a value greater than "<NUM>" may be determined as valid. Further, the data sequence <NUM> may include consecutive bits having values less than or equal to the threshold value "<NUM>". When there are consecutive bits having values less than or equal to the threshold value "<NUM>", the encoding circuit <NUM> may generate the compressed data sequence <NUM> by expressing the consecutive bits with a single bit value. In this example, the single bit value may indicate the number of consecutive bits having values less than or equal to the threshold value "<NUM>" in the data sequence <NUM>. For example, when the data sequence <NUM> includes eight consecutive bits having values less than or equal to the threshold value "<NUM>", such as "<NUM>", "<NUM>" of the data sequence <NUM> may be expressed as "<NUM>" in the compressed data sequence <NUM>. The encoding circuit <NUM> may compress consecutive invalid bits into a single bit as described above, thereby improving the operation speed of the neural network circuit <NUM>. Further, when a bit of the data sequence <NUM> is greater than the predetermined threshold value "<NUM>", the bit may be included in the compressed data sequence <NUM>. Accordingly, the encoding circuit <NUM> may compress the data sequence <NUM> of "<NUM>" to generate the compressed data sequence <NUM> of "<NUM>".

The encoding circuit <NUM> may generate the validity determination sequences <NUM> and <NUM> respectively indicating valid bits and invalid bits in the bit strings of the compressed data sequences <NUM> and <NUM>.

The validity determination sequences <NUM> and <NUM> may be binary sequences expressed by "<NUM>" and "<NUM>". For example, a value "<NUM>" included in the bit strings of the validity determination sequences <NUM> and <NUM> may indicate that a bit corresponding to an address of a corresponding bit in the compressed data sequences <NUM> and <NUM> is an invalid bit. Further, a value "<NUM>" included in the bit strings of the validity determination sequences <NUM> and <NUM> may indicate that a bit corresponding to an address of a corresponding bit in the compressed data sequences <NUM> and <NUM> is a valid bit.

The decoding circuit <NUM> may read the compressed data sequences <NUM> and <NUM> and the validity determination sequences <NUM> and <NUM> from the memory <NUM>. The decoding circuit <NUM> may determine bits to be transmitted to the neural network circuit <NUM> in the bit strings of the compressed data sequences <NUM> and <NUM> based on the validity determination sequences <NUM> and <NUM>, such that the neural network circuit <NUM> may omit the operation with respect to the non-consecutive invalid bits.

<FIG> illustrates an example of a sequence generated by an encoding circuit (e.g., encoding circuit <NUM> of <FIG>).

Referring to <FIG>, examples of a data sequence <NUM>, compressed data sequences <NUM>, <NUM>, and <NUM>, and validity determination sequences <NUM>, <NUM>, <NUM>, and <NUM> are illustrated.

The data sequence <NUM>, the compressed data sequence <NUM>, and the validity determination sequences <NUM> and <NUM> may respectively be generated by the same operations used to generate to the data sequence <NUM>, the compressed data sequence <NUM> and <NUM>, and the validity determination sequences <NUM> and <NUM> of <FIG>. For example, the encoding circuit <NUM> may generate the compressed data sequence <NUM> by expressing consecutive bits of the data sequence <NUM>, having values less than or equal to a threshold value "<NUM>", with a single bit indicating the number of the consecutive bits.

The encoding circuit <NUM> may generate the compressed data sequences <NUM> and <NUM> by compressing consecutive invalid bits in a bit string of the data sequence <NUM> into a single bit and further compressing consecutive valid bits having the same bit value into a single bit.

For example, the data sequence <NUM> may include consecutive bits having the same value greater than a threshold value "<NUM>". When there are consecutive bits having the same value greater than the threshold value "<NUM>" in the data sequence <NUM>, the encoding circuit <NUM> may generate the compressed data sequences <NUM> and <NUM> by expressing the consecutive bits with a single bit value. In an example, the single bit value may be expressed by the bit value of the consecutive bits in the data sequence <NUM>. For example, when the data sequence <NUM> includes four consecutive bits having values greater than the threshold value "<NUM>", such as "<NUM>", "<NUM>" of the data sequence <NUM> may be expressed as "<NUM>" in the compressed data sequences <NUM> and <NUM>. As another example, when the data sequence <NUM> includes three consecutive bits having values greater than the threshold value "<NUM>", such as "<NUM>", "<NUM>" of the data sequence <NUM> may be expressed as "<NUM>" in the compressed data sequences <NUM> and <NUM>. The encoding circuit <NUM> may compress consecutive valid bits into a single bit as described above, thereby improving the operation speed of the neural network circuit <NUM>. Accordingly, the encoding circuit <NUM> may compress the data sequence <NUM> of "<NUM>" to generate the compressed data sequences <NUM> and <NUM> of "<NUM>".

The encoding circuit <NUM> may generate the validity determination sequences <NUM> and <NUM> respectively indicating valid bits and invalid bits in the bit strings of the compressed data sequences <NUM> and <NUM>. Further, the encoding circuit <NUM> may generate the validity determination sequence <NUM> indicating the number of consecutive valid bits having the same bit value in the bit strings of the compressed data sequences <NUM> and <NUM>.

For example, a value "<NUM>" included in the bit string of the validity determination sequence <NUM> may indicate that a bit corresponding to an address of a corresponding bit in the compressed edge sequence <NUM> is an invalid bit. In this example, a bit corresponding to the invalid bit in the validity determination sequence <NUM> has a value "<NUM>".

When a bit value in the validity determination sequence <NUM> is "<NUM>", the number of consecutive valid bits having the same bit value may be determined using the validity determination sequence <NUM>. For example, a value "<NUM>" included in the bit string of the validity determination sequence <NUM> may indicate that a bit corresponding to an address of a corresponding bit in the compressed data sequence <NUM> appears consecutively four times in the compressed data sequence <NUM> and in the data sequence <NUM>.

The encoding circuit <NUM> may generate the validity determination sequence <NUM> that indicates valid bits and invalid bits in the bit strings of the compressed data sequences <NUM> and <NUM> and that simultaneously indicates the number of consecutive valid bits having the same bit value.

For example, a value "<NUM>" included in the bit string of the validity determination sequence <NUM> may indicate that a bit corresponding to an address of a corresponding bit in the compressed edge sequence <NUM> is an invalid bit. In this example, a bit corresponding to the invalid bit in the validity determination sequence <NUM> has a value "<NUM>". Further, a non-zero value included in the bit string of the validity determination sequence <NUM> may indicate that a bit corresponding to an address of a corresponding bit in the compressed edge sequence <NUM> is a valid bit and may indicate that the bit corresponding to the address of the corresponding bit in the compressed edge sequence <NUM> appears consecutively a number of times corresponding to the bit value.

For example, a value "<NUM>" included in the bit string of the validity determination sequence <NUM> indicates that a bit corresponding to an address of a corresponding bit in the compressed data sequence <NUM> is valid and indicates that the valid bit appears consecutively four times in the compressed data sequence <NUM> and in the data sequence <NUM>. Further, a value "<NUM>" included in the bit string of the validity determination sequence <NUM> may indicate that a bit corresponding to an address of a corresponding bit in the compressed data sequence <NUM> is invalid.

<FIG> illustrate examples of performing an operation in a neural network based on an output of a control apparatus.

Referring to <FIG>, a control apparatus <NUM> may output data for an operation of a neural network (for example, a fully connected network) to a neural network circuit <NUM>. For example, the data output from the control apparatus <NUM> may be input data for a current layer of the neural network. An operation may be performed using the input data for the current layer output from the control apparatus <NUM> and a weight sequence <NUM> of the current layer. Although <FIG> illustrate an example in which the control apparatus <NUM> outputs the input data for the current layer, an operation may be performed using the weight sequence <NUM> of the current layer output from the control apparatus <NUM> and the input data for the current layer, in some examples.

The neural network circuit <NUM> may perform a multiplication operation between matrices using a processing element. The neural network circuit <NUM> may output, as an output of the current layer, a result of performing an operation using the data output from the control apparatus <NUM> and the weight sequence <NUM>.

A decoding circuit <NUM> may include a buffer that sequentially stores a compressed data sequence and a validity determination sequence. The buffer may be a ring buffer.

The decoding circuit <NUM> may store a first pointer (for example, "c" of <FIG>) indicating a location at which a current bit of a compressed data sequence to be transmitted to the neural network circuit is stored in the buffer, a second pointer (for example, "n" of <FIG>) indicating a location at which a next bit of the compressed data sequence to be transmitted to the neural network circuit at a next cycle of the current bit is stored in the buffer, and a third pointer (for example, "w" of <FIG>) indicating a location at which the compressed data sequence and the validity determination sequence are to be stored in the buffer. Here, the first pointer may be referred to as the current pointer, the second pointer may be referred to as the next pointer, and the third pointer may be referred to as the write pointer.

The decoding circuit <NUM> may determine bits to be transmitted to the neural network circuit <NUM> in a bit string of the compressed data sequence based on the validity determination sequence, such that the neural network circuit omits an operation with respect to non-consecutive invalid bits.

The decoding circuit <NUM> may read the compressed data sequence and the validity determination sequence from the memory and sequentially store the compressed data sequence and the validity determination sequence in the buffer in the FIFO manner. In detail, the decoding circuit <NUM> may read a bit indicated by a read pointer (for example, "r" of <FIG>) in the compressed data sequence and the validity determination sequence from the memory and write the bit to the location corresponding to the third (or write) pointer in the buffer. The decoding circuit <NUM> may move the read pointer and the third pointer by one space.

The decoding circuit <NUM> may move the first (or current) pointer and the second (or next) pointer by one space when a bit value corresponding to the second pointer in the validity determination sequence is "<NUM>", and may move the first pointer and the second pointer by two spaces when the bit value corresponding to the second pointer in the validity determination sequence is "<NUM>".

The decoding circuit <NUM> may determine a bit value corresponding to the first pointer in the compressed data sequence and the validity determination sequence to be a bit waiting to be output.

Referring to <FIG>, an example of a compressed data sequence "<NUM>" and a validity determination sequence "<NUM>" to be output from the control apparatus <NUM> to the neural network circuit <NUM> is illustrated. The compressed data sequence "<NUM>" and the validity determination sequence "<NUM>" may be generated based on a data sequence <NUM> by an encoding circuit <NUM> and may be written to the memory.

The decoding circuit <NUM> may input a value obtained by adding "a" to the bit value corresponding to the second pointer in the validity determination sequence as data of a multiplexer, and may input the bit value corresponding to the second pointer in the validity determination sequence as a control signal of the multiplexer.

Referring to <FIG>, the decoding circuit <NUM> may move the first pointer and the second pointer by one space when the bit value corresponding to the second pointer in the validity determination sequence is "<NUM>" in the cycle of <FIG>.

In <FIG>, a current address counter of the weight sequence <NUM> may indicate an address corresponding to a first bit of the weight sequence <NUM>, and the decoding circuit <NUM> may transmit "<NUM>" being an output of the multiplexer to the address counter when the bit value corresponding to the second pointer in the validity determination sequence is "<NUM>" in the cycle of <FIG>. The value transmitted to the address counter may indicate a difference between a bit address of the weight sequence <NUM> of the current operation and a bit address of the weight sequence <NUM> of the next operation. For example, when the value "<NUM>" is transmitted to the address counter, a bit after one space participates in the next operation.

In <FIG>, the decoding circuit <NUM> may identify that a bit value of a bit waiting to be output in the compressed data sequence is "<NUM>" (e.g., when the bit value corresponding to the first pointer in the compressed data sequence is "<NUM>" in the cycle of <FIG>).

In <FIG>, the decoding circuit <NUM> may transmit the corresponding compressed data sequence to the neural network circuit <NUM> when a bit value of a bit waiting to be output in the validity determination sequence is "<NUM>" in the cycle of <FIG>, and may not transmit the corresponding compressed data sequence to the neural network circuit <NUM> when the bit value of the bit waiting to be output in the validity determination sequence is "<NUM>" in the cycle of <FIG>.

In <FIG>, a current address counter of the weight sequence <NUM> may indicate an address corresponding to a second bit of the weight sequence <NUM>, and the decoding circuit <NUM> may transmit "<NUM>" to the address counter when the bit value corresponding to the second pointer in the validity determination sequence is "<NUM>" in the cycle of <FIG>.

In <FIG>, the decoding circuit <NUM> may identify that a bit value of a bit waiting to be output in the compressed data sequence is "<NUM>" (e.g., when the bit value corresponding to the first pointer in the compressed data sequence is "<NUM>" in the cycle of <FIG>), and the bit value "<NUM>" of the bit waiting to be output in the compressed data sequence in <FIG> and the first bit "<NUM>" of the weight sequence <NUM> indicated by the address counter of <FIG> may participate in the operation of the neural network circuit <NUM> in the cycle of <FIG>.

Thus, in <FIG>, the bit value "<NUM>" of the compressed data sequence output from the decoding circuit <NUM> and the bit value "<NUM>" output from the weight sequence <NUM> may be input into the neural network circuit <NUM>. The neural network circuit <NUM> may store a value "<NUM>" by multiplying the bit value "<NUM>" of the compressed data sequence and the bit value "<NUM>" of the weight sequence <NUM>.

Referring to <FIG>, the decoding circuit <NUM> may move the first pointer and the second pointer by two spaces when the bit value corresponding to the second pointer in the validity determination sequence is "<NUM>" in the cycle of <FIG>.

In <FIG>, a current address counter of the weight sequence <NUM> may indicate an address corresponding to a third bit of the weight sequence <NUM>, and the decoding circuit <NUM> may transmit "<NUM>" (obtained by adding "<NUM>" to the bit value corresponding to the second pointer in the compressed data sequence of <FIG>) to the address counter when the bit value corresponding to the second pointer in the validity determination sequence is "<NUM>" in the cycle of <FIG>.

In <FIG>, the decoding circuit <NUM> may identify that a bit value of a bit waiting to be output in the compressed data sequence is "<NUM>" (e.g., when the bit value corresponding to the first pointer in the compressed data sequence is "<NUM>" in the cycle of <FIG>), and the bit value "<NUM>" of the bit waiting to be output in the compressed data sequence in <FIG> and the second bit "<NUM>" of the weight sequence <NUM> indicated by the address counter of <FIG> may participate in the operation of the neural network circuit <NUM> in the cycle of <FIG>.

Thus, in <FIG>, the bit value "<NUM>" of the compressed data sequence output from the decoding circuit <NUM> and the bit value "<NUM>" output from the weight sequence <NUM> may be input into the neural network circuit <NUM>. The neural network circuit <NUM> may store a value "<NUM>" by adding "<NUM>" (obtained by multiplying the bit value "<NUM>" of the compressed data sequence by the bit value "<NUM>" of the weight sequence <NUM>) to "<NUM>" that is the already stored intermediate result value from the cycle of <FIG>.

In <FIG>, a current address counter of the weight sequence <NUM> may indicate an address corresponding to a sixth bit of the weight sequence <NUM>, and the decoding circuit <NUM> may transmit "<NUM>" to the address counter when the bit value corresponding to the second pointer in the validity determination sequence is "<NUM>" in the cycle of <FIG>.

In <FIG>, the decoding circuit <NUM> may identify that a bit value of a bit waiting to be output in the compressed data sequence is "<NUM>" (e.g., when the bit value corresponding to the first pointer in the compressed data sequence is "<NUM>" in the cycle of <FIG>), and the bit value "<NUM>" of the bit waiting to be output in the compressed data sequence in <FIG> and the third bit "<NUM>" of the weight sequence <NUM> indicated by the address counter of <FIG> may participate in the operation of the neural network circuit <NUM> in the cycle of <FIG>.

In <FIG>, a current address counter of the weight sequence <NUM> may indicate an address corresponding to a seventh bit of the weight sequence <NUM>, and the decoding circuit <NUM> may transmit "<NUM>" (obtained by adding "<NUM>" to the bit value corresponding to the second pointer in the compressed data sequence of <FIG>) to the address counter when the bit value corresponding to the second pointer in the validity determination sequence is "<NUM>" in the cycle of <FIG>.

In <FIG>, the decoding circuit <NUM> may identify that a bit value of a bit waiting to be output in the compressed data sequence is "<NUM>" (e.g., when the bit value corresponding to the first pointer in the compressed data sequence is "<NUM>" in the cycle of <FIG>), and the bit value "<NUM>" of the bit waiting to be output in the compressed data sequence in <FIG> and the sixth bit "<NUM>" of the weight sequence <NUM> indicated by the address counter of <FIG> may participate in the operation of the neural network circuit <NUM> in the cycle of <FIG>.

However, in <FIG>, when the bit of the weight sequence <NUM> participating in the operation is "<NUM>", the bit may be determined to be invalid and thus, may not transmitted to the neural network circuit <NUM>. Accordingly, the neural network circuit <NUM> may not perform a multiplication operation with the compressed data sequence. Thus, the bit value indicated by the data sequence of the weight sequence <NUM> stored in the neural network circuit <NUM> may be maintained as a value "<NUM>", and the bit value indicated by the validity determination sequence of the weight sequence <NUM> may be set to "<NUM>". In addition, the neural network circuit <NUM> may still retain "<NUM>" that is the already stored intermediate result value from the cycle of <FIG>.

In <FIG>, a current address counter of the weight sequence <NUM> may indicate an address corresponding to an eleventh bit of the weight sequence <NUM>, and the decoding circuit <NUM> may transmit "<NUM>" to the address counter when the bit value corresponding to the second pointer in the validity determination sequence is "<NUM>" in the cycle of <FIG>.

In <FIG>, the decoding circuit <NUM> may identify that a bit value of a bit waiting to be output in the compressed data sequence is "<NUM>" (e.g., when the bit value corresponding to the first pointer in the compressed data sequence is "<NUM>" in the cycle of <FIG>), and the bit value "<NUM>" of the bit waiting to be output in the compressed data sequence in <FIG> and the seventh bit "<NUM>" of the weight sequence <NUM> indicated by the address counter of <FIG> may participate in the operation of the neural network circuit <NUM> in the cycle of <FIG>.

According to the example described with reference to <FIG>, invalid bits may not be output to the neural network circuit <NUM>, and thus the control apparatus of one or more embodiments may be configured to omit operations with respect to not only consecutive invalid bits but also non-consecutive invalid bits.

<FIG> illustrates an example of performing an operation in a neural network based on an output of a control apparatus.

Referring to <FIG>, a decoding circuit <NUM> of a control apparatus may output data for training of a neural network (for example, a fully connected network) to a neural network circuit <NUM>. The data output from the decoding circuit <NUM> of the control apparatus may be input data for a current layer of the neural network. An operation may be performed by the neural network circuit <NUM> using the input data for the current layer output from the decoding circuit <NUM> of the control apparatus and weight data <NUM>, <NUM>, <NUM>, and <NUM> of the current layer. Such an operation may include a multiplication operation between a weight matrix (for example, in the size of <NUM> × <NUM>) and an input matrix (for example, in the size of <NUM> × <NUM>). The neural network circuit <NUM>, also referred to as the training circuit <NUM>, may perform a multiplication operation between matrices using a plurality of processing elements. Data may move sequentially from left to right between the processing elements.

The process of performing an operation in a neural network based on an output of a control apparatus according to the example of <FIG> may be performed by iteratively performing the process of performing an operation in a neural network circuit based on an output of a control apparatus according to the example of <FIG> a number of times corresponding to the number of items of weight data.

<FIG> illustrates an example of zero gating.

Referring to <FIG>, a validity determination sequence may be used as a clock gating signal to perform an operation of a neural network circuit. A decoding circuit of a control apparatus may determine whether a current bit corresponding to a first pointer is valid, and may not transmit the current bit to a neural network circuit when the current bit is invalid (for example, if a bit value of the bit in a validity determination sequence is "<NUM>"). The decoding circuit may initiate an operation when data input into the neural network circuit <NUM> is valid, thereby reducing the power consumption of the neural network circuit.

As described above with reference to <FIG>, the decoding circuit may move the first pointer and a second pointer by two spaces when a bit value corresponding to the second pointer in the validity determination sequence is "<NUM>", and thus the current value corresponding to the first pointer generally has a valid value.

However, since the first pointer and the second pointer do not overtake a third pointer, the current bit corresponding to the first pointer may have an invalid value in a predetermined circumstance.

For example, in an example <NUM>, a bit value corresponding to a second pointer (e.g., "N") in a compressed data sequence may be invalid, and thus a first pointer (e.g., "C") and the second pointer may be moved by two spaces. However, in an example <NUM>, the first and second pointers may be moved by one space when the first pointer and the second pointer do not overtake a third pointer (e.g., "W").

Accordingly, in the example <NUM>, the current bit corresponding to the first pointer may have an invalid value. In an example <NUM>, the current bit corresponding to the first pointer may be determined to be an invalid value by the clock gating signal "<NUM>", and thus the decoding circuit maintains the bit value indicated by the previous data sequence to be "<NUM>", without transmitting the current bit to the neural network circuit.

Each of the examples <NUM>, <NUM>, and <NUM> may include a data sequence corresponding to the first row and a validity determination sequence corresponding to the second row.

<FIG> illustrates an example of increasing a reuse rate by storing a range of a value iteratively used.

Referring to <FIG>, a decoding circuit may store a fourth pointer for identifying a plurality of reused data when a data sequence includes the plurality of reused data. Here, the fourth pointer may also be referred to as the iteration pointer.

The decoding circuit may place multiple fourth pointers expressing iteration intervals when inserting a data sequence to be reused into a buffer, thereby facilitating iteration. The decoding circuit may split invalid data into two segments and store the segments separately in the buffer when the invalid data exceeds the iteration range.

The decoding circuit may store the fourth pointer indicating the range of iteration, and iteratively decode data of a next iteration interval after reusing data in the range until the data are reused to the maximum.

<FIG> illustrates an example of reducing power consumption using zero gating in a systolic array.

Referring to <FIG>, an encoding circuit of a control apparatus may separately compress input data for a current layer and weight data of the current layer depending on a range of reuse.

<FIG> illustrates an example of controlling data input and output when data are stored in parallel.

Referring to <FIG>, in a case of a memory with a great data bit width, data may be stored in parallel. The data stored in parallel may be highly likely to include different numbers of consecutive "<NUM>"s at the same address, and thus a decoding circuit may insert a dummy value to match the data to a sequence with a longest range.

Through this, the neural network circuit may omit an operation with respect to non-consecutive invalid bits in common for the data stored in parallel.

<FIG> illustrates an example of application of a method of controlling data input and output.

The method of controlling data input and output may be applicable to all schemes of sequentially reading consecutive data.

Referring to <FIG>, a control method of a control apparatus that is connected to a neural network circuit performing a deep learning operation to control data input and output may also applicable to a systolic array <NUM>.

Further, the control method of the control apparatus that is connected to the neural network circuit performing a deep learning operation to control data input and output may also applicable to an adder tree architecture <NUM>.

The control apparatuses, memories, encoding circuits, decoding circuits, neural network circuits, systolic arrays, adder tree architectures, address counters, accumulators, data control apparatuses, weight control apparatuses, input/weight control apparatuses, output control apparatuses, control apparatus <NUM>, memory <NUM>, encoding circuit <NUM>, decoding circuit <NUM>, neural network circuit <NUM>, control apparatus <NUM>, encoding circuit <NUM>, decoding circuit <NUM>, neural network circuit <NUM>, decoding circuit <NUM>, neural network circuit <NUM>, systolic array <NUM>, adder tree architecture <NUM>, apparatuses, units, modules, devices, and other components described herein with respect to FIGS. <NUM>-<NUM> are implemented by or representative of hardware components. Examples of hardware components that may be used to perform the operations described in this application where appropriate include controllers, sensors, generators, drivers, memories, comparators, arithmetic logic units, adders, subtractors, multipliers, dividers, integrators, and any other electronic components configured to perform the operations described in this application. In other examples, one or more of the hardware components that perform the operations described in this application are implemented by computing hardware, for example, by one or more processors or computers. A processor or computer may be implemented by one or more processing elements, such as an array of logic gates, a controller and an arithmetic logic unit, a digital signal processor, a microcomputer, a programmable logic controller, a field-programmable gate array, a programmable logic array, a microprocessor, or any other device or combination of devices that is configured to respond to and execute instructions in a defined manner to achieve a desired result. In one example, a processor or computer includes, or is connected to, one or more memories storing instructions or software that are executed by the processor or computer. Hardware components implemented by a processor or computer may execute instructions or software, such as an operating system (OS) and one or more software applications that run on the OS, to perform the operations described in this application. The hardware components may also access, manipulate, process, create, and store data in response to execution of the instructions or software. For simplicity, the singular term "processor" or "computer" may be used in the description of the examples described in this application, but in other examples multiple processors or computers may be used, or a processor or computer may include multiple processing elements, or multiple types of processing elements, or both. For example, a single hardware component or two or more hardware components may be implemented by a single processor, or two or more processors, or a processor and a controller. One or more hardware components may be implemented by one or more processors, or a processor and a controller, and one or more other hardware components may be implemented by one or more other processors, or another processor and another controller. One or more processors, or a processor and a controller, may implement a single hardware component, or two or more hardware components. A hardware component may have any one or more of different processing configurations, examples of which include a single processor, independent processors, parallel processors, single-instruction single-data (SISD) multiprocessing, single-instruction multiple-data (SIMD) multiprocessing, multiple-instruction single-data (MISD) multiprocessing, and multiple-instruction multiple-data (MIMD) multiprocessing.

The methods illustrated in FIGS. <NUM>-<NUM> that perform the operations described in this application are performed by computing hardware, for example, by one or more processors or computers, implemented as described above executing instructions or software to perform the operations described in this application that are performed by the methods.

The instructions or software may be written using any programming language based on the block diagrams and the flow charts illustrated in the drawings and the corresponding descriptions used herein, which disclose algorithms for performing the operations that are performed by the hardware components and the methods as described above.

Claim 1:
A neural network deep learning data control apparatus comprising:
a memory (<NUM>);
an encoding circuit (<NUM>, <NUM>) configured to:
receive a data sequence (<NUM>, <NUM>, <NUM>) being a sequence of bits having values,
generate a compressed data sequence (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>) in which consecutive invalid bits in a bit string of the data sequence are compressed into a single bit of the compressed data sequence, wherein an invalid bit is a bit having a value less than or equal to a predetermined threshold value while a bit having a value greater than the predetermined threshold value is a valid bit,
generate a validity determination sequence (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>) indicating valid and invalid bits in a bit string of the compressed data sequence, and
write the compressed data sequence and the validity determination sequence to the memory; and
a decoding circuit (<NUM>, <NUM>, <NUM>) comprising a buffer configured to sequentially store the compressed data sequence and the validity determination sequence, the decoding circuit being configured to:
read the compressed data sequence and the validity determination sequence from the memory, and
determine a bit in the bit string of the compressed data sequence set for transmission to a neural network circuit (<NUM>, <NUM>, <NUM>), based on the validity determination sequence, such that the neural network circuit omits an operation with respect to non-consecutive invalid bits,
wherein the decoding circuit is further configured to:
store a first pointer indicating a location at which a current bit of the compressed data sequence to be transmitted to the neural network circuit is stored in the buffer, and a second pointer indicating a location at which a next bit of the compressed data sequence to be transmitted to the neural network circuit at a next cycle of the current bit is stored in the buffer,
determine whether the next bit corresponding to the second pointer is valid based on the validity determination sequence,
move the first pointer to the location at which the next bit is stored in the buffer in response to the next bit being valid, and
move the first pointer to a location at which a bit to be transmitted to the neural network circuit at a next cycle of the next bit is stored in the buffer in response to the next bit being invalid.