Patent Description:
A neural network may be a computing system based on a computational architecture. Input data may be analyzed by using a neural network apparatus in various types of electronic systems and valid information may be extracted.

A typical neural network apparatus may need to perform a large amount of computations in order to process complicated input data. Moreover, when the neural network apparatus needs to perform such large amount of computations, the typical neural network apparatus may not efficiently process network operations and therefore may not analyze (or may not efficiently analyze) input data and/or extract (or efficiently extract) information in real time, e.g., when embedded or implemented by a low-power and high-performance system such as a smartphone having limited performance resources. <CIT> discloses a method of generating a fixed-point quantized neural network including analyzing a statistical distribution for each channel of floating-point parameter values of feature maps and a kernel for each channel from data of a pre-trained floating-point neural network, determining a fixed-point expression of each of the parameters for each channel statistically covering a distribution range of the floating-point parameter values based on the statistical distribution for each channel, determining fractional lengths of a bias and a weight for each channel among the parameters of the fixed-point expression for each channel based on a result of performing a convolution operation, and generating a fixed-point quantized neural network in which the bias and the weight for each channel have the determined fractional lengths. It is the object of the present invention to provide an improved neural network data quantizing method.

In one general aspect, a neural network data quantizing method includes: obtaining local quantization data by firstly quantizing, based on a local maximum value for each output channel of a current layer of a neural network, global recovery data obtained by recovering output data of an operation of the current layer based on a global maximum value corresponding to a previous layer of the neural network; storing the local quantization data in a memory to perform an operation of a next layer of the neural network; obtaining global quantization data by secondarily quantizing, based on a global maximum value corresponding to the current layer, local recovery data obtained by recovering the local quantization data based on the local maximum value for each output channel of the current layer; and providing the global quantization data as input data for the operation of the next layer.

The recovering of the output data of the operation of the current layer may include recovering the output data of the operation of the current layer for each output channel based on the global maximum value corresponding to the previous layer, and the global maximum value may be generated based on data obtained by recovering the output data of an operation of the previous layer.

The global maximum value may be a greatest value among values of output channels of the recovered output data of the operation of the previous layer.

The firstly quantizing of the global recovery data may include firstly quantizing the global recovery data for each output channel respectively based on the local maximum value that is a maximum value of the global recovery data for the output channel, such that the maximum value of the global recovery data for each output channel is quantized into a same value.

The recovering of the local quantization data may include recovering the local quantization data for each output channel respectively based on the local maximum value that is a maximum value of the global recovery data for the output channel, such that the local quantization data is recovered to data previous to the first quantization.

The data previous to the first quantization may correspond to the global recovery data.

The secondarily quantizing of the local recovery data may include secondarily quantizing the local recovery data for each output channel based on the global maximum value that is a maximum value of the local recovery data, such that the local recovery data is quantized to a same scale.

The obtaining of the local quantization data further may include: recovering the output data of the operation of the current layer based on the global maximum value corresponding to the previous layer; and obtaining the global recovery data by performing a pooling operation on the recovered data.

The method may include: storing the local maximum value for each output channel of the current layer, the local maximum values being obtained by calculating a maximum value of the global recovery data for each output channel of the current layer; and storing, in a register, the global maximum value corresponding to the current layer, the global maximum value corresponding to the current layer being obtained by calculating a maximum value of the local maximum values.

The obtaining of the local quantization data may be sequentially performed for each output channel.

The operation of the current layer and the operation of the next layer may be based on backward propagation, and the output data of the operation of the current layer and the input data for the operation of the next layer may include either one of an activation, a loss gradient, and a weight gradient.

The method may include training the neural network by performing the operation of the next layer based on the global quantization data.

A non-transitory computer-readable storage medium may store instructions that, when executed by a processor, configure the processor to perform the method.

In another general aspect, a neural network data quantizing apparatus includes: a processor configured to: obtain local quantization data by firstly quantizing, based on a local maximum value for each output channel of a current layer of a neural network, global recovery data obtained by recovering output data of an operation of the current layer based on a global maximum value corresponding to a previous layer of the neural network; store the local quantization data in a memory to perform an operation of a next layer of the neural network; obtain global quantization data by secondarily quantizing, based on a global maximum value corresponding to the current layer, local recovery data obtained by recovering the local quantization data based on the local maximum value for each output channel of the current layer; and provide the global quantization data as input data for an operation of a next layer.

For the recovering of the output data of the operation of the current layer, the processor may be configured to recover the output data of the operation of the current layer for each output channel based on the global maximum value corresponding to the previous layer, and the global maximum value may be generated based on data obtained by recovering the output data of an operation of the previous layer.

For the firstly quantizing of the global recovery data, the processor may be configured to firstly quantize the global recovery data for each output channel respectively based on the local maximum value that is a maximum value of the global recovery data for the output channel, such that the maximum value of the global recovery data for each output channel is quantized into a same value.

For the recovering of the local quantization data, the processor may be configured to recover the local quantization data for each output channel respectively based on the local maximum value that is a maximum value of the global recovery data for the output channel, such that the local quantization data is recovered to data previous to the first quantization.

For the secondarily quantizing of the local recovery data, the processor may be configured to secondarily quantize the local recovery data for each output channel based on the global maximum value that is a maximum value of the local recovery data, such that the local recovery data is quantized to a same scale.

For the obtaining of the local quantization data, the processor may be configured to: recover the output data of the operation of the current layer based on the global maximum value corresponding to the previous layer; and obtain the global recovery data by performing a pooling operation on the recovered data.

The processor may be configured to: store the local maximum value for each output channel of the current layer, the local maximum values being obtained by calculating the maximum value of the global recovery data for each of all output channels of the current layer; and store, in a register, the global maximum value corresponding to the current layer, the global maximum value corresponding to the current layer being obtained by calculating a maximum value of the local maximum values.

For the obtaining of the local quantization data, the processor may be configured to sequentially obtain the local quantization data for each output channel.

The processor may be configured to perform the operation of the current layer and the operation of the next layer based on backward propagation, and the output data of the operation of the current layer and the input data for the operation of the next layer may each include any one of an activation, a loss gradient, and a weight gradient.

The apparatus may be any one of a personal computer (PC), a server device, a mobile device, a smartphone, a tablet device, an augmented reality (AR) device, an Internet of Things (IoT) device, an autonomous vehicle, a robotic device, and a medical device, and the processor may be configured to perform any one of voice recognition, image recognition, and image classification based on an output of the operation of the next layer.

The apparatus may include a memory storing instructions that, when executed by the processors, configure the processor to perform the obtaining of the local quantization data, the storing of the local quantization data, the obtaining of the global quantization, and the providing of the global quantization data.

In another general aspect, a neural network data quantizing method includes: globally recovering output data of a current layer of a neural network based on a global maximum value of a previous layer of the neural network; locally quantizing the globally recovered output data of the current layer based on local maximum values of output channels of the current layer; locally recovering the locally quantized data based on the local maximum values; globally quantizing the locally recovered data based on a global maximum value of output channels of the current layer; and providing the globally quantization data as input data for a next layer of the neural network.

In this regard, the one or more embodiments may have different forms and should not be construed as being limited to the descriptions set forth herein.

Unless otherwise defined, all terms, including technical and scientific terms, used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains and after an understanding of the disclosure of this application. Terms, such as those defined in commonly used dictionaries, are to be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the disclosure of this application, and are not to be interpreted in an idealized or overly formal sense unless expressly so defined herein.

The term used in the embodiments such as "unit", etc., indicates a unit for processing at least one function or operation, and where the unit is hardware or a combination of hardware and software.

Herein, it is noted that use of the term "may" with respect to an example or embodiment, e.g., as to what an example or embodiment may include or implement, means that at least one example or embodiment exists where such a feature is included or implemented while all examples and embodiments are not limited thereto.

Hereinbelow, embodiments will be described in detail with reference to the attached drawings. However, embodiments may be implemented in various forms, and are not limited to examples described herein.

<FIG> illustrates a neural network according to one or more embodiments of the present disclosure.

A neural network <NUM> illustrated in <FIG> may be, but is not limited to, an example of a deep neural network (DNN). For example, the neural network <NUM> may be a DNN including an input layer (Layer1), a plurality of hidden layers (Layer2, Layer3, Layer <NUM>, and Layer5), and an output layer (Layer6).

When the neural network <NUM> is implemented as a DNN, the neural network <NUM> may include many layers capable of processing valid information. Thus, with the neural network <NUM>, data sets that are more complex may be processed than with a neural network including a single layer. Meanwhile, the neural network <NUM> is illustrated as including six layers in <FIG>, but this illustration is merely an example. In other words, the neural network <NUM> may include layers that are more than or less than those illustrated in <FIG>. That is, the neural network <NUM> may include layers of various structures that are different from those illustrated in <FIG>.

<FIG> illustrates an architecture of a neural network according to one or more embodiments of the present disclosure.

Some convolutional layers in a neural network <NUM> are illustrated in <FIG>, the neural network <NUM> may further include pooling layers and fully connected layers, etc., as well as the illustrated convolutional layers and pooling (sub-sampling) layers.

The neural network <NUM> may be implemented as architecture having a plurality of layers that output feature maps. In the neural network <NUM>, an operation is performed on an input image with a filter referred to as a kernel, and as a result, the feature maps are output to a next layer. The operation is performed again using another filter on the output feature maps, as input feature maps to a next layer resulting in new output feature maps. When the operation is repeatedly performed as such, a recognition result with respect to features of the input image may be finally output through the neural network <NUM>.

For example, when an input image having a 24x24 pixel size is input to the neural network <NUM> of <FIG>, the input image may be output as feature maps of four channels each having a 20x20 pixel size, through an operation with a first filter. That is, feature maps may be output for each of four output channels. The output feature maps of the four channels may be output as feature maps having a 10x10 pixel size for the four channels through a sub-sampling (or pooling) operation. Thereafter, sizes of the feature maps may be reduced through repeated operations using respective filters and sub-sampling (or pooling) operations, and finally, features each having a 1x1 pixel size may be output. In the neural network <NUM>, the operations and the sub-sampling (or pooling) operations may be repeatedly performed in several stages so as to respectively filter and output robust features, which may represent the entire input image, from the input image, and to derive a recognition result, as a non-limiting example, of the input image through final features that are output. Based on the above, it is understood that example embodiments discussed herein include embodiments with one or more, in various combinations, or all such layers, stages, edges, etc. in operations of the neural network <NUM>.

Referring to <FIG> and <FIG>, forward propagation and backward propagation will be described according to one or more embodiments.

<FIG> illustrates forward propagation and backward propagation according to one or more embodiments of the present disclosure.

In <FIG>, a part of a convolution layer <NUM> is illustrated. The convolution layer <NUM> of <FIG> may include an input feature map X, a filter F, and an output feature map O.

The input feature map X, the filter F, and the output feature map O may respectively correspond to input activations (as output activation, from a previous layer), weights, and output activations. A convolution operation may be performed on the input feature map using the filter F through forward propagation, resulting in the output feature map O. This convolution forward propagation is also referred to an inference operation.

For example, the filter F and the input feature map X may be subject to the convolution operation using a sliding window scheme. More specifically, input activations in a first window may be multiplied by weights of the filter, and products thereof may be summed. The window moves or slides in any axis direction (e.g., an x axis, a y axis, or a z axis) from the first window to a second window. Input activations in the second window may be multiplied by weights of the filter, and products thereof may be summed. When the convolution operation is continuously performed in this way, the output feature map O may be generated.

While one input feature map X and one filter F are illustrated in <FIG>, the convolution operation may be performed on a plurality of input feature maps and a plurality of filters F to generate the output feature map O.

In a training process, once a final output feature map is generated, the final output feature map may be compared with an expected training result, thus generating a loss function. This process of training the neural network may be iteratively executed to minimize a loss of the loss function.

To minimize the loss of the loss function, a loss gradient <MAT> may be propagated backward. The loss gradient may mean a gradient of activation.

Thus, during this training, the convolution operation may be performed on the backward-propagating loss gradient ao using a filter having elements rearranged from filter F, resulting in a loss gradient aX for backward propagation to a previous layer.

The convolution operation may also be performed on the loss gradient ao and the input feature map X, resulting in a weight gradient <MAT>.

<FIG> illustrates an operation of a convolution layer in forward propagation and backward propagation according to one or more embodiments of the present disclosure.

The convolution operation may be performed on the input feature map X and the filter F through forward propagation, resulting in the output feature map O. The input feature map may be, or based on, training data for example. For example, activation of the output feature map O may be generated based on a multiplication and accumulation (MAC) operation performed using activation of the input feature map X and a weight of the filter F.

For example, an output feature map <MAT> of an input feature map X = <MAT> and a filter <MAT> may be generated using Equation <NUM> below, for example.

xii indicates activation of the input feature map X, fii indicates the weight of the filter F, and oii indicates the resultant activation of the output feature map O.

The backward-propagating loss gradient ao and a filter F' having rearranged elements may be subject to the convolution operation, resulting in a loss gradient aX for backward propagation to a next previous layer, e.g., as the corresponding loss gradient <MAT> for the previous layer to adjust the filter of the previous layer. For example, an element of the loss gradient aX may be generated based on the MAC operation performed using an element of the loss gradient <MAT> and a weight of the filter F'. In an example, the filter F' may be generated by rotating the elements of the filter F by <NUM> degrees.

For example, a loss gradient <MAT> for backward propagation to a next layer from a backward-propagating loss gradient <MAT> and a filter <MAT> having rearranged elements may be generated using Equation <NUM> below, for example.

The convolution operation may also be performed on the backward-propagating loss gradient ao and the input feature map X, resulting in a weight gradient <MAT>. For example, an element of the weight gradient <MAT> may be generated based on the MAC operation performed using an element of the loss gradient <MAT> and activation of the input feature map X.

For example, a weight gradient, <MAT>, for backward propagation to a next layer may be generated from the input feature map <MAT> and a loss gradient <MAT> <MAT> backward-propagating from a previous layer, by using Equation <NUM> below, for example.

<FIG> illustrates a neural network learning device and a neural network inference device according to one or more embodiments of the present disclosure.

Referring to <FIG>, a neural network learning device <NUM> may be a computing device having various processing functions such as functions of creating and training (or learning) a neural network, quantizing data of the neural network, or retraining the neural network. For example, the neural network learning device <NUM> may be implemented with, or as, various types of devices such as a personal computer (PC), a server device, a mobile device, etc..

The neural network learning device <NUM> may create a trained neural network <NUM> by repeatedly training (learning) a given initial neural network. Data in a floating-point format may be quantized into a fixed-point format in a training process, and/or data in the fixed-point format may undergo an operation process and then quantized back into the fixed-point format.

The neural network learning device <NUM> may deliver the trained neural network <NUM> to a hardware accelerator such as a neural network inference device <NUM>. The neural network inference device <NUM> may be, or be included in, a mobile device, an embedded device, etc. The neural network inference device <NUM>, which is hardware for driving a neural network <NUM>, may be implemented with relatively low power or low performance, and thus may be properly implemented for a fixed-point operation rather than a floating-point operation. A function of the neural network inference device <NUM> is not limited to driving of the neural network <NUM>, and may include functions of the neural network learning device <NUM>. In an example, the neural network inference device <NUM> may perform training operations herein and the neural network learning device <NUM> may perform inference operations described herein.

The neural network inference device <NUM> may correspond to, but not limited to, a dedicated module for driving a neural network, such as a neural processing unit (NPU), a tensor processing unit (TPU), a neural engine, etc. The neural network inference device <NUM> may be included in, but not limited to, an autonomous vehicle, robotics, a smartphone, a tablet device, an augmented reality (AR) device, an Internet of Things (IoT) device, etc., which performs voice recognition, image recognition, etc., using a neural network.

<FIG> illustrates quantizing data of a neural network according to one or more embodiments of the present disclosure.

In driving or training of a neural network, a process of performing (by a calculator <NUM>) an operation of a current layer, quantizing data output as a result of the operation, storing the quantized data in a memory <NUM>, and reading data stored in the memory <NUM>, and inputting the read data into the calculator <NUM> for use in an operation to a next layer may be repeatedly performed.

In forward propagation, output data resulting from an operation may include activation. In backward propagation, output data resulting from an operation may include a loss gradient and a weight gradient.

Meanwhile, prior to quantization, the output data resulting from the operation may be stored in an on-chip memory serving as a cache.

A neural network inference device included in a smartphone, a tablet, a wearable device, etc., having relatively low processing performance may have a limited space of an on-chip memory. Accordingly, a method of quantizing data of a neural network according to one or more embodiments may quantize data to sufficiently reduce a space used by the on-chip memory in storing the data, such that the neural network may be efficiently driven and/or trained using the neural network inference device having a limited resource (such as the limited space of the on-chip memory), thereby improving the functioning of the neural network inference device with which the neural network may be driven and/or trained. Moreover, by sequentially quantizing output data for each channel of a neural network layer, the method of quantizing data of the neural network according to one or more embodiments may reduce the amount of resources used by the on-chip memory compared to a typical quantization method which may quantize all output data of the layer, thereby improving.

Hereinbelow, a method of quantizing data of a neural network according to one or more embodiments will be described with reference to <FIG>.

<FIG> illustrates a method of quantizing data of a neural network according to one or more embodiments of the present disclosure.

The flowchart illustrated in <FIG> may include operations processed in time series by a neural network apparatus <NUM> to be described later with reference to <FIG>. Matters described below may be applied to the neural network apparatus <NUM>.

In operation <NUM>, a processor <NUM> of the neural network device <NUM> may primarily (e.g., firstly) quantize global recovery data (recovered from output data of an operation of a current layer based on a global maximum value corresponding to a previous layer) based on a local maximum value for each output channel of the current layer, thereby obtaining local quantization data.

The previous layer and the current layer may be, but not limited to, an input layer, a hidden layer, and an output layer, etc., of a DNN. The previous layer and the current layer may be, but not limited to, a layer included in a convolution neural network. The previous layer and the current layer may be, but not limited to, a convolution layer, a pooling layer, a flatten layer, a normalization layer, a fully connected layer, etc..

When the neural network apparatus <NUM> drives the neural network through forward propagation, output data resulting from the operation of the current layer may include activation. Further, when the neural network apparatus <NUM> trains the neural network through backward propagation, output data resulting from the operation of the current layer may include a loss gradient or a weight gradient.

The processor <NUM> recovers the output data resulting from the operation of the current layer based on the global maximum value corresponding to the previous layer, thus generating global recovery data. Data recovery means that data is dequantized into a scale previous to quantization.

The processor <NUM> may recover the output data resulting from the operation of the current layer based on the global maximum value corresponding to the previous layer and perform sub-operations such as the pooling operation, arrangement normalization, activation using the activation function, etc., thus generating the global recovery data.

The processor <NUM> may add noise to the output data resulting from the operation of the current layer to generate the global recovery data. By performing primary (first) quantization with respect to the global recovery data generated by addition of the noise to the output data, the learning performance of the neural network may be improved. Noise may be, but not limited to, Gaussian noise, uniform noise, white noise, etc..

The global maximum value corresponding to the previous layer may be a value generated based on the output data resulting from the operation of the previous layer. For example, the global maximum value corresponding to the previous layer may be a maximum value of recovered output data when the output data resulting from the operation of the previous layer is recovered.

The output data resulting from the operation of the current layer may be recovered based on Equation <NUM> below, for example. To recover the output data resulting from the operation of the current layer, the global maximum value corresponding to the previous layer may be used as max(|x|) in Equation <NUM>, and the output data of the operation of the current layer may be used as maxqn(x).

The processor <NUM> may recover the output data resulting from the operation of the current layer for each output channel of the current layer. As the output data resulting from the operation of the current layer is recovered for each output channel of the current layer, the global recovery data may be generated for each output channel of the current layer. The global recovery data generated for each output channel of the current layer may be quantized based on a separate reference value for each output channel of the current layer.

The processor <NUM> may sequentially recover the output data resulting from the operation of the current layer for each output channel of the current layer. For example, when the current layer includes three output channels, the processor <NUM> may recover output data of a second output channel after recovering output data of a first output channel, and then recover output data of a third output channel.

The processor <NUM> may generate local quantization data by primarily quantizing the global recovery data based on the local maximum value for each output channel of the current layer. Primary quantization based on the local maximum value for each output channel of the current layer may mean quantization based on a separate reference value for each output channel of the current layer. The local maximum value may be a maximum value for each output channel of the current layer of the global recovery data.

The global recovery data may be quantized based on Equation <NUM> and Equation <NUM>. To primarily quantize the global recovery data, the local maximum value for each output channel of the current layer may be used as of Equation <NUM>, and the global recovery data may be used as x. In Equations <NUM> and <NUM> below, n may indicate the number of bits of a level for quantization. <MAT> <MAT>.

The processor <NUM> may sequentially perform primary quantization on the global recovery data for each output channel of the current layer. For example, when the current layer includes three output channels, the processor <NUM> may quantize global recovery data of the second output channel after quantizing global recovery data of the first output channel, and then quantize global recovery data of the third output channel. As the global recovery data is sequentially quantized for each output channel, the amount of a resource used by the on-chip memory for a quantization process may be reduced, thereby improving the functioning of a device including the on-chip memory with which the neural network may be driven and/or trained.

The processor <NUM> may sequentially perform recovery and primary quantization with respect to the output data resulting from the operation of the current layer for each output channel of the current layer. For example, when the current layer includes three output channels, the processor <NUM> may recover and quantize the output data of the second output channel after recovering and quantizing the output data of the first output channel, and then recovering and quantizing the output data of the third output channel.

In operation <NUM>, the processor <NUM> may store the local quantization data in the memory to perform the operation of the next layer. The memory may be the memory <NUM> of the neural network apparatus <NUM> or the on-chip memory included in the processor <NUM>.

The next layer may be, but not limited to, an input layer, a hidden layer, and an output layer, etc., of a DNN. The next layer may be, but not limited to, a layer included in a convolution neural network. The next layer may be, but not limited to, a convolution layer, a pooling layer, a flatten layer, a normalization layer, a fully connected layer, etc..

In operation <NUM>, the processor <NUM> may perform secondary quantization on the local recovery data, recovered from the local quantization data based on the local maximum value for each output channel, based on the global maximum value corresponding to the current layer, thus obtaining global quantization data.

The processor <NUM> may recover the local quantization data based on the local maximum value for each output channel, thus generating local recovery data. The local quantization data may be recovered for each output channel based on the same local maximum value as in primary quantization. The local quantization data may be dequantized to a scale previous to quantization by being recovered based on the local maximum value for each output channel. For example, the local recovery data may be the same as the global recovery data.

The local quantization data may be recovered based on Equation <NUM>. To recover the local quantization data, the local maximum value for each output channel of the current layer may be used as max(|x|) of Equation <NUM>, and the local quantization data may be used as maxqn(x).

The processor <NUM> may sequentially recover the local quantization data for each output channel of the current layer. For example, when the current layer includes three output channels, the processor <NUM> may recover the local quantization data of the second output channel after recovering the local quantization data of the first output channel, and then recover the local quantization data of the third output channel.

The processor <NUM> may perform secondary quantization on the local recovery data based on the global maximum value corresponding to the current layer, thus generating global quantization data.

The global maximum value corresponding to the current layer may be a maximum value of the global recovery data. The global maximum value corresponding to the current layer may be calculated based on the local maximum value for each output channel of the current layer. For example, the global maximum value corresponding to the current layer may be a maximum value among local maximum values for all output channels of the current layer.

The local recovery data may be secondarily quantized based on Equation <NUM> and Equation <NUM>. To secondarily quantize the local recovery data, the global maximum value corresponding to the current layer may be used as max(|x|) of Equation <NUM>.

The processor <NUM> may sequentially perform secondary quantization on the local recovery data for each output channel of the current layer. For example, when the current layer includes three output channels, the processor <NUM> may quantize local recovery data of the second output channel after quantizing local recovery data of the first output channel, and then quantize local recovery data of the third output channel.

The processor <NUM> may sequentially perform recovery and secondary quantization with respect to the local quantization data for each output channel of the current layer. For example, when the current layer includes three output channels, the processor <NUM> may recover and quantize the local quantization data of the second output channel after recovering and quantizing the local quantization data of the first output channel, and then recover and quantize the local quantization data of the third output channel.

In operation <NUM>, the processor <NUM> may provide the global quantization data as input data for an operation of the next layer.

<FIG> illustrates storing a local maximum value and a global maximum value according to one or more embodiments of the present disclosure.

In operation <NUM>, the processor <NUM> may store local maximum values for all output channels of the current layer in a register of the processor <NUM>. For example, the processor <NUM> may store the local maximum values in a register file.

The processor <NUM> may obtain the local maximum values by calculating a maximum value of the global recovery data for each of all the output channels of the current layer.

The processor <NUM> may sequentially store the local maximum values in the register. For example, when the current layer includes three output channels, the processor <NUM> may store a local maximum value calculated from the global recovery data of the first output channel, store a local maximum value calculated from the global recovery data of the second output channel, and then store a local maximum value calculated from the global recovery data of the third output channel.

The processor <NUM> may read the local maximum values stored in the register to recover the local quantization data in a subsequent process.

In operation <NUM>, the processor <NUM> may store a global maximum value corresponding to the current layer, which is the maximum value among the local maximum values, in the register.

The processor <NUM> may obtain the global maximum value corresponding to the current layer, by calculating the maximum value of the local maximum values for all the output channels of the current layer.

The processor <NUM> may read the global maximum value stored in the register to secondarily quantize the local recovery data in a subsequent process.

In the neural network, a calculator <NUM> may perform a convolution operation. Calculators described below may be included in a processing unit of the processor <NUM>. The calculator <NUM> may perform an operation of the next layer after performing an operation of the current layer. Output data resulting from the operation of the current layer may be used in the operation of the next layer, after passing through primary quantization and secondary quantization.

When the neural network operates through forward propagation, output data <NUM> resulting from the operation of the current layer may include activation. When the neural network is trained through backward propagation, output data <NUM> resulting from the operation of the current layer may include a loss gradient or a weight gradient.

The processor <NUM> may recover the output data <NUM> based on the global maximum value corresponding to the previous layer, thus generating global recovery data <NUM>. The processor <NUM> may recover the output data <NUM> based on the global maximum value corresponding to the previous layer and perform sub-operations such as the pooling operation, arrangement normalization, activation using the activation function, etc., thus generating the global recovery data <NUM>. The processor <NUM> may add noise to the output data <NUM> to generate the global recovery data.

The processor <NUM> may recover the output data <NUM> for each output channel of the current layer. The processor <NUM> may sequentially recover the output data <NUM> for each output channel of the current layer. For example, when the current layer includes n output channels, the processor <NUM> may recover output data <NUM> of the second output channel after recovering output data <NUM> of the first output channel, and finally recover output data <NUM> of an nth output channel.

The processor <NUM> may calculate local maximum values Local Max #<NUM>, Local Max #<NUM>,. , Local Max #n from global recovery data <NUM>. For example, when the current layer includes n output channels, the processor <NUM> may calculate a maximum value of global recovery data <NUM> of the first output channel as the local maximum value Local Max #<NUM>, calculate a maximum value of global recovery data <NUM> of the second output channel as the local maximum value Local Max #<NUM>, and finally calculate a maximum value of global recovery data <NUM> of the nth output channel as the local maximum value Local Max #n. The calculated local maximum values Local Max #<NUM>, Local Max #<NUM>,. , Local Max #n may be stored in a register <NUM> of the processor <NUM>.

The processor <NUM> may calculate a global maximum value Global Max corresponding to the current layer from the local maximum values Local Max #<NUM>, Local Max #<NUM>,. , Local Max #n. For example, the processor <NUM> may calculate a maximum value of the local maximum values Local Max #<NUM>, Local Max #<NUM>,. , Local Max #n as the global maximum value Global Max corresponding to the current layer. The calculated global maximum value Global Max may be stored in the register <NUM>.

The processor <NUM> may primarily quantize the global recovery data <NUM> based on the local maximum values Local Max #<NUM>, Local Max #<NUM>,. , Local Max #n, thus generating local quantization data <NUM>.

The processor <NUM> may primarily quantize the global recovery data <NUM> for each output channel of the current layer. The processor <NUM> may sequentially perform primary quantization on the global recovery data <NUM> for each output channel of the current layer. For example, when the current layer includes n output channels, the processor <NUM> may quantize the global recovery data <NUM> of the second output channel based on the local maximum value Local Max #<NUM> after quantizing the global recovery data <NUM> of the first output channel based on the local maximum value Local Max #<NUM>, and finally quantize the global recovery data <NUM> of the nth output channel based on the local maximum value Local Max #n.

The local quantization data <NUM> may be stored in a memory <NUM> for use in the operation of the next layer. The memory <NUM> may be the memory <NUM> of the neural network apparatus <NUM> or the on-chip memory included in the processor <NUM>.

The processor <NUM> may load local quantization data <NUM> from the memory <NUM> and recover the loaded local quantization data <NUM> based on a local maximum value for each output channel of the current layer, thus generating local recovery data <NUM>. The local maximum value for each output channel of the current layer may be read from the register <NUM>.

The processor <NUM> may recover the local quantization data <NUM> for each output channel of the current layer. The processor <NUM> may sequentially recover the local quantization data <NUM> for each output channel of the current layer. For example, when the current layer includes n output channels, the processor <NUM> may recover local quantization data <NUM> of the second output channel based on the local maximum value Local Max #<NUM> after recovering local quantization data <NUM> of the first output channel based on the local maximum value Local Max #<NUM>, and finally recover local quantization data <NUM> of the nth output channel based on the local maximum value Local Max #n.

The local recovery data <NUM> may be the same as the global recovery data <NUM>. For example, when the current layer includes n output channels, the global recovery data <NUM> of the first output channel may be the same as local recovery data <NUM>, the global recovery data <NUM> of the second output channel may be the same as local recovery data <NUM>, and finally, the global recovery data <NUM> of the nth output channel may be the same as local recovery data <NUM>.

The processor <NUM> may secondarily quantize the local recovery data <NUM> based on the global maximum value Global Max corresponding to the current layer, thus generating global quantization data <NUM>. The global maximum value Global Max corresponding to the current layer may be read from the register <NUM>.

The processor <NUM> may secondarily quantize the local recovery data <NUM> for each output channel of the current layer. The processor <NUM> may sequentially perform secondary quantization on the local recovery data <NUM> for each output channel of the current layer. For example, when the current layer includes n output channels, the processor <NUM> may quantize the local recovery data <NUM> of the second output channel based on the global maximum value Global Max after quantizing the local recovery data <NUM> of the first output channel based on the global maximum value Global Max, and finally quantize the local recovery data <NUM> of the nth output channel based on the global maximum value Global Max.

The processor <NUM> may perform secondary quantization on the local recovery data <NUM> for all the output channels, thus quantizing the local recovery data <NUM> to the same scale.

The global quantization data <NUM> may be input to the calculator <NUM> for use in the operation of the next layer.

<FIG> illustrates obtaining a local maximum value and a global maximum value according to one or more embodiments of the present disclosure.

When the current layer includes four output channels, a local maximum value may be calculated for each of the four output channels.

The processor <NUM> may calculate the local maximum value Local Max #<NUM> for the first output channel from global recovery data <NUM> of the first output channel. For example, the processor <NUM> may calculate a maximum value of <NUM> of the global recovery data <NUM> of the first output channel as the local maximum value Local Max #<NUM>.

Likewise, the processor <NUM> may calculate the local maximum values Local Max #<NUM>, Local Max #<NUM>, and Local Max #<NUM> for the second, third, and fourth output channels from respective global recovery data <NUM>, <NUM>, and <NUM> for the second, third, and fourth output channels.

The processor <NUM> may calculate the global maximum value Global Max corresponding to the current layer from the local maximum values Local Max #<NUM>, Local Max #<NUM>, Local Max #<NUM>, and Local Max #<NUM>. For example, the processor <NUM> may calculate a maximum value of <NUM> of the global maximum values Local Max #<NUM>, Local Max #<NUM>, Local Max #<NUM>, and Local Max #<NUM> as the global maximum value Global Max.

<FIG> illustrates primary quantization and secondary quantization for each output channel according to one or more embodiments of the present disclosure.

More specifically, <FIG> shows an example of primarily quantizing and secondarily quantizing output data <NUM> and <NUM> resulting from an operation of the current layer including two output channels into <NUM> bits.

The processor <NUM> may recover the output data <NUM> of a first channel, thus generating global recovery data <NUM> of the first channel. The processor <NUM> may recover the output data <NUM> of the first output channel based on a global maximum value Global Max@Previous Layer corresponding to the previous layer. For example, the output data <NUM> may be recovered based on Equation <NUM>, and more specifically, data having a value of <NUM>/<NUM><NUM> of the output data <NUM> may be recovered based on the global maximum value, <NUM>, Global Max@Previous Layer corresponding to the previous layer, thus generating global recovery data having a value of <NUM>. For example, using Equation <NUM>, when max(|x|) is <NUM> and maxqn(x) is <NUM>/<NUM><NUM>, global recovery data x having a value of <NUM> may be generated.

The processor <NUM> may calculate the local maximum value Local Max #<NUM> for the first channel. For example, the processor <NUM> may calculate a maximum value of <NUM> of the global recovery data <NUM> as the local maximum value Local Max #<NUM>.

The processor <NUM> may primarily quantize the global recovery data <NUM> of the first channel based on the local maximum value Local Max #<NUM> for the first channel, thus generating local quantization data <NUM> of the first channel. For example, the global recovery data <NUM> may be primarily recovered based on Equation <NUM> and Equation <NUM>, and more specifically, data having a value of <NUM> of the global recovery data <NUM> may be primarily quantized based on the local maximum value, <NUM>, Local Max#<NUM> for the first channel, thus generating local quantization data having a value of <NUM>/<NUM><NUM>. For example, using Equation <NUM>, when x is <NUM> and max(|x|) is <NUM>, local quantization data maxqn(x) having a value of <NUM>/<NUM><NUM> may be generated.

Similarly, the processor <NUM> may recover the output data <NUM> of a second channel, thus generating global recovery data <NUM> of the second channel. The processor <NUM> may recover the output data <NUM> based on a global maximum value Global Max@Previous Layer corresponding to the previous layer. For example, of the output data <NUM>, data having a value of <NUM>/<NUM><NUM> may be recovered based on the global maximum value, <NUM>, Global Max@Previous Layer corresponding to the previous layer, thus generating global recovery data having a value of <NUM>. For example, using Equation <NUM>, when max(|x|) is <NUM> and maxqn(x) is <NUM>/<NUM><NUM>, global recovery data x having a value of <NUM> may be generated.

Likewise, the processor <NUM> may calculate the local maximum value Local Max #<NUM> for the second channel. For example, the processor <NUM> may calculate a maximum value of <NUM> of the global recovery data <NUM> as the local maximum value Local Max #<NUM>.

Likewise, the processor <NUM> may primarily quantize the global recovery data <NUM> of the first channel based on the local maximum value Local Max #<NUM> for the second channel, thus generating local quantization data <NUM> of the second channel. For example, of the global recovery data <NUM>, data having a value of <NUM> may be primarily quantized based on the local maximum value, <NUM>, Local Max#<NUM> for the second channel, thus generating local quantization data having a value of <NUM>/<NUM><NUM>. For example, using Equation <NUM>, when x is <NUM> and max(|x|) is <NUM>, local quantization data maxqn(x) having a value of <NUM>/<NUM><NUM> may be generated.

The processor <NUM> may calculate a global maximum value Global Max@Current Layer corresponding to the current layer from the local maximum value Local Max #<NUM> for the first channel and the local maximum value Local Max #<NUM>. For example, the processor <NUM> may calculate a maximum value, <NUM>, of the local maximum value Local Max #<NUM> for the first channel and the local maximum value Local Max #<NUM> for the second channel, as the global maximum value Global Max@Current Layer corresponding to the current layer.

The local maximum value Local Max #<NUM> for the first channel, the local maximum value Local Max #<NUM> for the second channel, and the global maximum value Global Max@Current Layer corresponding to the current layer may be stored in the register of the processor <NUM>.

The processor <NUM> may recover the local quantization data <NUM> of the first channel, thus generating local recovery data <NUM> of the first channel. The processor <NUM> may recover the local quantization data <NUM> of the first output channel based on the local maximum value Local Max #<NUM> for the first channel. For example, the local quantization data <NUM> may be recovered based on Equation <NUM>, and more specifically, data having a value of <NUM>/<NUM><NUM> of the local quantization data <NUM> may be recovered based on the local maximum value, <NUM>, Local Max #<NUM> for the first channel, thus generating local recovery data having a value of <NUM>. For example, using Equation <NUM>, when max(|x|) is <NUM> and maxqn(x) is <NUM>/<NUM><NUM>, local recovery data x having a value of <NUM> may be generated.

The processor <NUM> may secondarily quantize the local recovery data <NUM> of the first channel based on the global maximum value Global Max@Current Layer corresponding to the current layer, thus generating global quantization data <NUM> of the first channel. For example, the local recovery data <NUM> may be secondarily recovered based on Equation <NUM> and Equation <NUM>, and data having a value of <NUM> of the local recovery data <NUM> may be secondarily quantized based on the global maximum value, <NUM>, Global Max@Current Layer corresponding to the current layer, thus generating global quantization data having a value of <NUM>/<NUM><NUM>. For example, using Equation <NUM>, when x is <NUM> and max(|x|) is <NUM>, global quantization data maxqn(x) having a value of <NUM>/<NUM><NUM> may be generated.

Likewise, the processor <NUM> may recover the local quantization data <NUM> of the second channel, thus generating local recovery data <NUM> of the second channel. The processor <NUM> may recover the local quantization data <NUM> of the second output channel based on the local maximum value Local Max #<NUM> for the second channel. For example, of the local quantization data <NUM>, data having a value of <NUM>/<NUM><NUM> may be recovered based on the local maximum value, <NUM>, Local Max#<NUM> for the second channel, thus generating local recovery data having a value of <NUM>. For example, using Equation <NUM>, when max(|x|) is <NUM> and maxqn(x) is <NUM>/<NUM><NUM>, local recovery data x having a value of <NUM> may be generated.

The processor <NUM> may secondarily quantize the local recovery data <NUM> of the second channel based on the global maximum value Global Max@Current Layer corresponding to the current layer, thus generating global quantization data <NUM> of the second channel. For example, of the local recovery data <NUM>, data having a value of <NUM> may be secondarily quantized based on the global maximum value, <NUM>, Global Max@Current Layer corresponding to the current layer, thus generating global quantization data having a value of <NUM>/<NUM><NUM>. For example, using Equation <NUM>, when x is <NUM> and max(|x|) is <NUM>, global quantization data maxqn(x) having a value of <NUM>/<NUM><NUM> may be generated.

<FIG> illustrates a quantization algorithm of data of a neural network (e.g., using the method illustrated in <FIG>) according to one or more embodiments of the present disclosure.

Referring to <FIG>, in a quantization algorithm <NUM>, a backward-propagating loss gradient may be quantized.

In operation <NUM>, the processor <NUM> may recover local quantization data loss_gradientG for each input channel of the current layer, thus generating local recovery data loss_gradient. In this case, the input channel of the current layer may correspond to an output channel in forward propagation. The processor <NUM> may recover the local quantization data loss_gradientG based on a local maximum value local_maxcin.

In operation <NUM>, the processor <NUM> may quantize local recovery data loss_gradient for each input channel of the current layer, thus generating global quantization data loss_gradientG. The processor <NUM> may quantize the local recovery data loss_gradient based on a global maximum value global_maxi, thus generating global quantization data loss_gradientG.

In operation <NUM>, the processor <NUM> may perform a convolution operation on the global quantization data loss_gradientG.

In operation <NUM>, the processor <NUM> may recover output data loss_gradientG resulting from an operation of the current layer for each output channel of the current layer. In this case, the output channel of the current layer may correspond to an input channel in forward propagation. The output data loss_gradientG may be recovered based on the global maximum value global-maxi.

In operation <NUM>, the processor <NUM> may perform sub-operations such as arrangement normalization, the pooling operation, activation using the activation function, etc., on the recovered data, thus generating global recovery data.

In operation <NUM>, the processor <NUM> may calculate a local maximum value local_maxcout for each output channel of the current layer.

In operation <NUM>, the processor <NUM> may quantize global recovery data loss_gradient for each output channel of the current layer, thus generating local quantization data loss_gradientG. The processor <NUM> may quantize the global recovery data loss_gradient based on the local maximum value local_maxcout,thus generating local quantization data loss_gradientG.

In operation <NUM>, the processor <NUM> may calculate a global maximum value global_maxI-<NUM> from local maximum values local_maxc1, local_maxc2,. , obtained for all the output channels of the current layer.

In the neural network, a calculator <NUM> may perform a convolution operation. The calculator <NUM> may perform an operation of the next layer after performing an operation of the current layer. Output data resulting from the operation of the current layer may be used in the operation of the next layer, after passing through quantization.

When the neural network operates through forward propagation, output data <NUM> resulting from the operation of the current layer may include activation. When the neural network is trained through backward propagation, the output data <NUM> resulting from the operation of the current layer may include a loss gradient or a weight gradient.

The processor <NUM> may predict a global maximum value for recovering the output data <NUM>. For example, the processor <NUM> may predict the global maximum value based on output data resulting from an operation of a previous layer. For example, the processor <NUM> may predict the global maximum value based on an aspect of data change in quantization.

The processor <NUM> may recover the output data <NUM> based on the predicted global maximum value, thus generating global recovery data <NUM>. The processor <NUM> may recover the output data <NUM> based on the predicted global maximum value and perform sub-operations such as the pooling operation, arrangement normalization, activation using the activation function, etc., thus generating the global recovery data <NUM>. The processor <NUM> may recover the output data <NUM> based on the predicted global maximum value and add noise to the recovered data, thus generating the global recovery data <NUM>.

The processor <NUM> may calculate local maximum values from the global recovery data <NUM>. For example, when the current layer includes n output channels, the processor <NUM> may calculate a maximum value of global recovery data <NUM> of the first output channel as a local maximum value for the first output channel, calculate a maximum value of global recovery data <NUM> of the second output channel as a local maximum value for the second output channel, and finally calculate a maximum value of global recovery data <NUM> of the nth output channel as a local maximum value for the nth output channel.

The processor <NUM> may generate local quantization data <NUM> by quantizing the global recovery data <NUM> based on the local maximum values.

The processor <NUM> may quantize the global recovery data <NUM> for each output channel of the current layer. The processor <NUM> may sequentially quantize the global recovery data <NUM> for each output channel of the current layer. For example, when the current layer includes n output channels, the processor <NUM> may quantize the global recovery data <NUM> of the first output channel based on the local maximum value for the first output channel, quantize the global recovery data <NUM> of the second output channel based on the local maximum value for the second output channel, and finally quantize the global recovery data <NUM> of the nth output channel based on the local maximum value for the nth output channel.

The processor <NUM> may compensate for the local quantization data <NUM> based on a difference between a local maximum value for each output channel of the current layer and the predicted global maximum value, thus generating global quantization data <NUM>. For example, when the local maximum value for each output channel and the predicted global maximum value are equal to each other, the processor <NUM> may set the local quantization data <NUM> as the global quantization data <NUM>. For example, when the local maximum value for each output channel is less than the predicted global maximum value, the processor <NUM> may perform compensation in such a way to reduce the local quantization data <NUM> in proportional to the difference between the local maximum value and the predicted global maximum value, thus generating the global quantization data <NUM>.

The processor <NUM> may compensate for the local quantization data <NUM> for each output channel of the current layer. The processor <NUM> may sequentially compensate for the local quantization data <NUM> for each output channel of the current layer. For example, when the current layer includes n output channels, the processor <NUM> may compensate for local quantization data <NUM> of the first output channel based on a difference between a first local maximum value and the predicted global maximum value, compensate for local quantization data <NUM> of the second output channel based on a difference between a second local maximum value and the predicted global maximum value, and finally compensate for local quantization data <NUM> of the nth output channel based on a difference between an nth local maximum value and the predicted global maximum value.

The amount of resources used by the register in quantization may be reduced because the processor <NUM> generates the global quantization data <NUM> without storing the local maximum values in the register.

The global quantization data <NUM> may be stored in and read from the memory <NUM> and input to the calculator <NUM> for use in an operation of the next layer. The memory <NUM> may be the memory <NUM> of the neural network apparatus <NUM> or the on-chip memory included in the processor <NUM>.

<FIG> illustrates a neural network apparatus according to one or more embodiments of the present disclosure.

Referring to <FIG>, a neural network apparatus <NUM> may include a memory <NUM> and a processor <NUM>. The neural network apparatus <NUM> may be connected with an external memory. The neural network apparatus <NUM> illustrated in <FIG> may include components associated with the current embodiment. Thus, it will be understood to one of ordinary skill in the art after an understanding of the present disclosure that other general-purpose components other than the components illustrated in <FIG> may be further included in the neural network apparatus <NUM>.

The neural network apparatus <NUM> may be an apparatus in which the above-described neural network is implemented with reference to <FIG>. For example, the neural network apparatus <NUM> may be implemented as, or with, various types of devices such as a personal computer (PC), a server device, a mobile device, an embedded device, etc. In detail, the neural network apparatus <NUM> may be included in a smartphone, a tablet device, an augmented reality (AR) device, an Internet of Things (IoT) device, an autonomous vehicle, a robotic device, or a medical device, which performs voice recognition, image recognition, and image classification using a neural network, but is not limited thereto. The neural network apparatus <NUM> may correspond to an exclusive hardware (HW) accelerator mounted on such a device, and may be an HW accelerator, such as a neural processing unit (NPU), a tensor processing unit (TPU), or a neural engine, which is an exclusive module for driving a neural network. The neural network apparatus <NUM> may be the neural network device <NUM> or the neural network inference device <NUM> of <FIG>.

The memory <NUM> may store various data processed in the neural network apparatus <NUM>. For example, the memory <NUM> may store data processed or to be processed in the neural network apparatus <NUM> for processing data. Also, the memory <NUM> may store applications or drivers to be driven by the neural network apparatus <NUM>.

For example, the memory <NUM> may include random-access memory (RAM), such as dynamic random-access memory (DRAM) or static random-access memory (SRAM), read-only memory (RAM), electrically erasable programmable read-only memory (EEPROM), a CD-ROM, a Blu-ray disk, optical disk storage, a hard disk drive (HDD), a solid state drive (SSD), or a flash memory.

The processor <NUM> may control overall functions for executing the neural network apparatus <NUM> in a neural network. For example, the processor <NUM> may control the neural network apparatus <NUM> for processing data in general by executing programs stored in the memory <NUM>. The processor <NUM> may be implemented as a central processing unit (CPU), a graphics processing unit (GPU), or an application processor (AP) included in the neural network apparatus <NUM>, but is not limited thereto.

The processor <NUM> may read or write data, for example, image data, feature map data, or kernel data, from or to the memory <NUM>, and execute the neural network by using the read/written data. When the neural network is executed, the processor <NUM> may drive processing units provided therein to repeatedly perform an operation between an input feature map and a kernel, thereby generating data related to an output feature map. Here, an operation count of the operation may be determined based on various factors, such as the number of channels of the input feature map, the number of channels of the kernel, the size of the input feature map, the size of the kernel, and precision of a value.

For example, the processing unit may include a logic circuit for an operation. More specifically, the processing unit may include a calculator implemented with a combination of a multiplier, an adder, and an accumulator. The multiplier may include a combination of a plurality of sub-multipliers, and the adder may also include a combination of a plurality of sub-adders.

The processor <NUM> may further include an on-chip memory in charge of a register (or a register file) and a cache to process an operation and a dispatcher that dispatches various operands, such as pixel values of an input feature map or weights of filters. For example, the dispatcher may dispatch operands such as pixel values and weights required for an operation to be performed by a processing unit from data stored in the memory <NUM> to the on-chip memory. Then, the dispatcher may dispatch the operands dispatched to the on-chip memory again to a processing unit for the operation.

The neural network learning devices, neural network inference devices, calculators, memories, buffers, calculators, registers, neural network apparatuses, processors, neural network learning device <NUM>, neural network inference device <NUM>, calculator <NUM>, memory <NUM>, buffer <NUM>, calculator <NUM>, memory <NUM>, register <NUM>, calculator <NUM>, memory <NUM>, neural network apparatus <NUM>, memory <NUM>, processor <NUM>, and other apparatuses, units, modules, devices, and other components described herein with respect to <FIG> are implemented by or representative of hardware components. Examples of hardware components that may be used to perform the operations described in this application where appropriate include controllers, sensors, generators, drivers, memories, comparators, arithmetic logic modules, adders, subtractors, multipliers, dividers, integrators, and any other electronic components configured to perform the operations described in this application. In other examples, one or more of the hardware components that perform the operations described in this application are implemented by computing hardware, for example, by one or more processors or computers. A processor or computer may be implemented by one or more processing elements, such as an array of logic gates, a controller and an arithmetic logic module, a digital signal processor, a microcomputer, a programmable logic controller, a field-programmable gate array, a programmable logic array, a microprocessor, or any other device or combination of devices that is configured to respond to and execute instructions in a defined manner to achieve a desired result. In one example, a processor or computer includes, or is connected to, one or more memories storing instructions or software that are executed by the processor or computer. Hardware components implemented by a processor or computer may execute instructions or software, such as an operating system (OS) and one or more software applications that run on the OS, to perform the operations described in this application. The hardware components may also access, manipulate, process, create, and store data in response to execution of the instructions or software. For simplicity, the singular term "processor" or "computer" may be used in the description of the examples described in this application, but in other examples multiple processors or computers may be used, or a processor or computer may include multiple processing elements, or multiple types of processing elements, or both. For example, a single hardware component or two or more hardware components may be implemented by a single processor, or two or more processors, or a processor and a controller. One or more hardware components may be implemented by one or more processors, or a processor and a controller, and one or more other hardware components may be implemented by one or more other processors, or another processor and another controller. One or more processors, or a processor and a controller, may implement a single hardware component, or two or more hardware components. A hardware component may have any one or more of different processing configurations, examples of which include a single processor, independent processors, parallel processors, single-instruction single-data (SISD) multiprocessing, single-instruction multiple-data (SIMD) multiprocessing, multiple-instruction single-data (MISD) multiprocessing, and multiple-instruction multiple-data (MIMD) multiprocessing.

The instructions or software may be written using any programming language based on the block diagrams and the flow charts illustrated in the drawings and the corresponding descriptions used herein, which disclose algorithms for performing the operations that are performed by the hardware components and the methods as described above.

Claim 1:
A computer-implemented neural network data quantizing method, the method comprising:
obtaining (<NUM>) local quantization data (<NUM>; <NUM>, <NUM>) by firstly quantizing, based on a local maximum value for each output channel of a current layer of a neural network, global recovery data (<NUM>; <NUM>, <NUM>, <NUM>, <NUM>; <NUM>, <NUM>) obtained by recovering output data (<NUM>; <NUM>, <NUM>) of an operation of the current layer based on a global maximum value corresponding to a previous layer of the neural network;
storing (<NUM>) the local quantization (<NUM>; <NUM>, <NUM>) data in a memory to perform an operation of a next layer of the neural network;
obtaining (<NUM>) global quantization data (<NUM>; <NUM>, <NUM>) by secondarily quantizing, based on a global maximum value corresponding to the current layer, local recovery data (<NUM>; <NUM>, <NUM>) obtained by recovering the local quantization data (<NUM>, <NUM>; <NUM>, <NUM>) based on the local maximum value for each output channel of the current layer, wherein data recovery means that data is de-quantized into a scale previous to quantization; and
providing (<NUM>) the global quantization data (<NUM>; <NUM>, <NUM>) as input data for the operation of the next layer.