Patent application EP20214065A (EP), filed 2020-12-15
Publication number: EP3926543A1
Title: METHOD, ACCELERATOR, AND ELECTRONIC DEVICE WITH TENSOR PROCESSING
Abstract: [0001]    A processor-implemented tensor processing method includes: receiving a request to process a neural network including a normalization layer by an accelerator; and generating an instruction executable by the accelerator in response to the request, wherein, by executing the instruction, the accelerator is configured to determine an intermediate tensor corresponding to a result of performing a portion of operations included in the normalization layer, by performing, in a channel axis direction, a convolution based on: a target tensor on which the portion of operations is to be performed; and a kernel having a number of input channels and a number of output channels determined based on the target tensor and including elements of scaling values determined based on the target tensor.
IPC codes: G06N   3/    04            A I
G06N   3/   063            A I
G06N   3/  0464            A I
Claims: number: 1
text: A processor-implemented tensor processing method, comprising:
number: 2
text: The method of claim 1, wherein the accelerator is configured to determine the intermediate tensor by extracting diagonal elements from a result tensor determined by the convolution based on the target tensor and the kernel.
number: 3
text: The method of claim 1 or 2, wherein the number of input channels of the kernel is determined based on a number of elements of a normalization unit applied to the target tensor,
preferably,
wherein the number of elements of the normalization unit applied to the target tensor is equal to a number of channels of the input tensor, and the number of input channels of the kernel is equal to the number of channels of the input tensor.
number: 4
text: The method of one of claims 1 to 3, wherein the number of output channels of the kernel is determined based on a width length of the target tensor; and/or
number: 5
text: The method of one of claims 1 to 4, wherein the target tensor is determined based on:
number: 6
text: The method of claim 5, wherein the target tensor is determined by performing, in a channel axis direction, a convolution based on:
number: 7
text: The method of claim 6, wherein
the number of input channels and the number of output channels of the second kernel are equal to the number of elements of the normalization unit, and
the diagonal elements in the second kernel have different scaling values from those of the remaining elements.
number: 8
text: The method of claim 6, wherein the constant value is equal to a square root of the number of elements of a normalization unit applied to the target tensor, and the scaling values of the second kernel are equal to an inverse of the square root.
number: 9
text: The method of one of claims 1 to 8, wherein the normalization layer is configured to perform normalization using either one or both of an average and a variance determined based on values of one or more elements included in the target tensor.
number: 10
text: The method of one of claims 1 to 9, wherein
the convolution is performed between the kernel and an input tensor transformed such that elements included in the same channel are arranged in a line, and
the intermediate tensor is determined by transforming elements determined as a result of the convolution to the same form as the input tensor.
number: 11
text: The method of one of claims 1 to 10, wherein the convolution is performed in the accelerator such that the target tensor is not transmitted outside the accelerator to perform an operation according to the normalization layer; and/or
wherein the accelerator is included in either one of:
number: 12
text: A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, configure the processor to perform the method of one of claims 1 to 11.
number: 13
text: An accelerator, comprising:
one or more processors configured to:
number: 14
text: The accelerator of claim 13, wherein the one or more processors are configured to determine the intermediate tensor by extracting diagonal elements from a result tensor determined by the convolution based on the target tensor and the kernel; and/or
wherein the number of input channels determined of the kernel is based on a number of elements of a normalization unit applied to the target tensor; and/or
wherein a scaling value of each of the elements included in the kernel includes a runtime value corresponding to the target tensor; and/or
wherein a scaling value of each of the elements included in the kernel is equal to a value of a corresponding element in the target tensor.
number: 15
text: An electronic device, comprising:
Office action: search_report, dated 2021-12-22
Prior art cited: ref_id: - XU HAOBO ET AL: "ACG-Engine: An Inference Accelerator for Content Generative Neural Networks", 2019 IEEE/ACM INTERNATIONAL CONFERENCE ON COMPUTER-AIDED DESIGN (ICCAD), IEEE, 4 November 2019 (2019-11-04), pages 1 - 7, XP033678235
ref_type: npl
severity: background
ref_id: - WONKYUNG JUNG ET AL: "Restructuring Batch Normalization to Accelerate CNN Training", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 4 July 2018 (2018-07-04), XP081128616
ref_type: npl
severity: background
ref_id: - XU HAOBO ET AL, "ACG-Engine: An Inference Accelerator for Content Generative Neural Networks", 2019 IEEE/ACM INTERNATIONAL CONFERENCE ON COMPUTER-AIDED DESIGN (ICCAD), IEEE, (20191104), pages 1 - 7, XP033678235 [I] 1-15 * "Motivational analysis" in Section 2; Section 3.1; figure 4 *
ref_type: npl
severity: background
ref_id: - WONKYUNG JUNG ET AL, "Restructuring Batch Normalization to Accelerate CNN Training", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, (20180704), XP081128616 [A] 1-15 * figure 5 *
ref_type: npl
severity: background
Final disposition: pending (2021-12-22)