An information processing device performing deep learning using a first number of processing devices that perform processes in parallel, the deep learning being performed using dynamic fixed-point number, the information processing device includes a processor. The processor configured to allocate, when allocating a propagation operation in a layer of the deep learning to the first number of processing devices, a second number of processing devices for every third number of pieces of input data, the third number being less than a first number, the second number of the processing device acquiring a statistical information used for adjusting decimal point positions of the dynamic fixed-point numbers, and allocate output channels for every third number of pieces of input data while shifting the output channels by a fourth number.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims the benefit of priority of the prior Japanese Patent Application No. 2020-69144, filed on Apr. 7, 2020, the entire contents of which are incorporated herein by reference.

FIELD

The embodiments discussed herein are related to an information processing apparatus and an information processing method.

BACKGROUND

In recent years, in order to improve the recognition performance of a deep neural network (DNN), the number of parameters used for deep learning and the number of pieces of learning data have been increasing. Here, the parameters indude weights between nodes, data held by the nodes, filter elements, and the like. For this reason, the computation load and memory load of a parallel computer used for speeding up the deep learning have grown larger, and the learning time has increased. In re-learning during the service of the DNN, the increase in learning time brings about a heavy burden.

Thus, in order to lighten the DNN, the number of bits used by the parameter to represent data is shrunk. For example, by using an 8-bit fixed-point number instead of a 32-bit floating-point number, the amount of data may be reduced and the amount of computation time may be reduced.

However, using the 8-bit fixed-point number deteriorates the accuracy of operations. In view of this, a dynamic fixed-point number capable of dynamically modifying the fixed-point position of a variable used for learning is used. When the dynamic fixed-point number is used, the parallel computer acquires statistical information on the variable during learning and automatically adjusts the fixed-point position of the variable. Furthermore, the parallel computer may decrease the overhead expected for acquiring the statistical information by providing a statistical information acquisition circuit in respective processing devices that perform operations in parallel.

Japanese Laid-open Patent Publication No. 2018-124681 is disclosed as related art.

SUMMARY

According to an aspect of the embodiments, an information processing apparatus performing deep learning using a first number of processing devices that perform processes in parallel, the deep learning being performed using dynamic fixed-point number, the information processing apparatus includes a memory and a processor coupled to memory and configured to allocate, when allocating a propagation operation in a layer of the deep learning to the first number of processing devices, a second number of processing devices for every third number of pieces of input data, the third number being less than a first number, the second number of the processing device acquiring a statistical information used for adjusting decimal point positions of the dynamic fixed-point numbers, and allocate output channels for every third number of pieces of input data while shifting the output channels by a fourth number.

DESCRIPTION OF EMBODIMENTS

In the related art, if the statistical information acquisition circuits are provided in all the processing devices of the parallel computer, the circuit area of the parallel computer becomes larger. Thus, in order to reduce the circuit area, it is conceivable to provide the statistical information acquisition circuit only in some processing devices. However, if the statistical information is acquired only by some processing devices and thinned out, an error occurs as compared with a case where the statistical information is acquired by all the processing devices, and an appropriate decimal point position may not be set. For this reason, there is a problem that the saturation and rounding of variable values increase during learning, and the learning accuracy deteriorates.

In one aspect, an object of the present embodiments is to suppress the deterioration of learning accuracy when a statistical information acquisition circuit is provided in some processing devices.

Embodiments of an information processing device and an information processing method disclosed by the present application will be described in detail below based on the drawings. Note that the embodiments do not limit the technology disclosed.

Embodiments

First, the information processing device (apparatus) according to an embodiment will be described.FIG. 1is a diagram illustrating a configuration of the information processing device according to the embodiment. As illustrated inFIG. 1, the information processing device1according to the embodiment includes an accelerator board10, a host20, and a hard disk drive (HDD)30.

The accelerator board10is a board equipped with a parallel computer that performs deep learning at high speed. The accelerator board10includes a controller11, a plurality of processing elements (PEs)12, a dynamic random access memory (DRAM)13, and peripheral component interconnect express (PCIe) hardware14. The number of PEs12is, for example, 2,048.

The controller11is a control device that controls the accelerator board10. For example, the controller11instructs each PE12to execute an operation, based on an instruction from the host20. The storage location of data input and output by each PE12is specified by the host20. Note that, although omitted inFIG. 1, the controller11is connected to each PE12.

The PE12executes an operation, based on the instruction from the controller11. The PE12reads out and executes a program stored in the DRAM13. A part of PEs12ainclude a statistical information acquisition circuit and a statistical information storage circuit. The ratio of the part of PEs12ato the number of all PEs12is, for example, 1/16. The number of the part of PEs12ais, for example, a divisor of the number of all PEs12. Note that, in the following, the part of PEs12awill be referred to as information acquisition PEs12a.

The statistical information acquisition circuit acquires statistical information. Note that the statistical information will be described later. The statistical information storage circuit stores the statistical information acquired by the statistical information acquisition circuit. The statistical information stored in the statistical information storage circuit is read out by the controller11and sent to the host20. Note that the statistical information may be stored in the DRAM13so as to be read out from the DRAM13and sent to the host20.

Furthermore, the information acquisition PE12ais not limited to the configuration including the dedicated statistical information acquisition circuit and statistical information storage circuit as long as the information acquisition PE12acan acquire the statistical information and send the acquired statistical information to the host20. For example, a program executed by the PE12described later may include an instruction sequence for acquiring the statistical information. The instruction sequence for acquiring the statistical information is such that, for example, the result of a multiply-add operation is stored in a register #1 as a 32-bit integer, information on the most significant digit position of the result stored in the register #1 is stored in a register #2, and 1 is added to the value in a table indexed by the value in the register #2.

The DRAM13is a volatile storage device that stores a program executed by the PE12, data input by each PE12, and data output by each PE12. An address used by each PE12for data input and output is specified by the host20. The PCIe hardware14is hardware that communicates with the host20by PCI Express (PCIe).

The host20is a device that controls the information processing device1. The host20includes a central processing unit (CPU)21, a DRAM22, and PCIe hardware23.

The CPU21is a central processing unit that reads out a program from the DRAM22and executes the read-out program. The CPU21instructs the accelerator board10to execute parallel operations and performs deep learning by executing a deep learning program. The deep learning program includes an allocation program that allocates operations in deep learning to each PE12. The CPU21implements an allocation unit40by executing the allocation program. Note that the details of the allocation unit40will be described later.

The DRAM22is a volatile storage device that stores programs and data stored in the HDD30, intermediate results of program execution by the CPU21, and the like. The deep learning program is called from the HDD30to the DRAM22and executed by the CPU21.

The PCIe hardware23is hardware that communicates with the accelerator board10by PCI Express.

The HDD30stores the deep learning program, input data used for deep learning, a model generated by deep learning, and the like. The information processing device1may include a solid state drive (SSD) instead of the HDD30.

Next, deep learning according to the embodiment will be described.FIG. 2is a diagram for explaining the deep learning according to the embodiment. As illustrated inFIG. 2, the deep learning according to the embodiment is executed by processes of a convolution layer #1 (Conv_1), a pooling layer #1 (Pool_1), a convolution layer #2 (Conv_2), a pooling layer #2 (Pool_2), a fully connected layer #1 (fc1), and a fully connected layer #2 (fc2). In the deep learning according to the embodiment, the input data is subjected to a forward propagation process in the order of the convolution layer #1, the pooling layer #1, the convolution layer #2, the pooling layer #2, the fully connected layer #1, and the fully connected layer #2. Then, the error is computed based on the output of the fully connected layer #2 and correct data, and a backpropagation process is performed based on the error in the order of the fully connected layer #2, the fully connected layer #1, the pooling layer #2, the convolution layer #2, the pooling layer #1, and the convolution layer #1.

The deep learning according to the embodiment is executed divided into processing units referred to as mini-batches. Here, the mini-batch is a combination of k pieces of data obtained by dividing a collection of input data to be learned {(Ini, Ti), i=1 to N} into plural sets (for example, M sets of k pieces of data, N=k*M). Furthermore, the mini-batch refers to a processing unit of learning that is executed on every such input data set (k pieces of data). Here, Ini is input data (vector) and Ti is correct data (vector). The information processing device1acquires statistical information about some of variables of each layer and updates the decimal point position of each variable of each layer for each mini-batch during the deep learning as follows. Here, a decimal point position e corresponds to an exponent part common to all the elements of a parameter X. When the element of the parameter X is denoted by x and the integer representation is denoted by n, the representation x=n×2ecan hold. Note that the information processing device1may update the decimal point position every time the learning of the mini-batch is ended a predetermined number of times.

The information processing device1, for example, determines the initial decimal point position of each variable by trial (for example, one time on a mini-batch) with a floating-point number or user specification, and starts learning. Then, the information processing device1saves the statistical information about some variables in each layer during learning of one mini-batch (k pieces of data) (t1). If overflow occurs while learning the mini-batch, the information processing device1performs a saturation process and continues learning. Then, the information processing device1updates the decimal point position of the fixed-point number in line with the statistical information after the learning of the mini-batch one time is ended (t2). Thereafter, the information processing device1repeats t1 and t2 until a predetermined learning end condition is satisfied.

FIG. 3is a diagram illustrating an example of the statistical information.FIG. 3illustrates the distribution of position of leftmost set bit for positive number and position of leftmost zero bit for negative number, as an example of the statistical information. Here, the position of leftmost set bit means the position of a leftmost bit where the bit has 1. Furthermore, for negative numbers, the position of leftmost set bit means the position of a leftmost bit that has bit0. The position of leftmost set bit for positive number and position of leftmost zero bit for negative number refers to, for example, the position of a bit with the largest index k among bits[k] different from a sign bit bit[39] when the bits are placed from the most significant bit bit[39] to the least significant bit bit[0]. When the distribution of the position of leftmost set bit for positive number and position of leftmost zero bit for negative number is obtained, the distribution range of the values as absolute values can be grasped.

InFIG. 3, the vertical axis denotes the number of occurrences of the position of leftmost set bit for positive number and position of leftmost zero bit for negative number, and the horizontal axis denotes a value obtained by adding the decimal point position e to a count leading sign (CLS), which is the position of the non-sign most significant bit. An arithmetic operation circuit of the PE12of the information processing device1and a register in the arithmetic operation circuit have a bit width (for example, 40 bits) equal to or greater than the number of bits (for example, 16 bits) of the register specified by an instruction operand. However, the bit width of the arithmetic operation circuit of the PE12and the register in the arithmetic operation circuit is not necessarily limited to 40 bits. Here, the decimal point position e is determined by the decimal point position at the input of an operation. For example, in the case of multiplication, when the decimal point positions of two input vectors are denoted by e1 and e2, e1+e2 obtained by adding e1 and e2 is employed. In addition, the operation result is stored in a register (a register specified by an instruction operand) having a bit width smaller than the bit width of the arithmetic operation circuit, such as a 16-bit register, for example. As a result, the operation result (for example, 40 bits) is shifted by a shift amount specified by the operand, and a bit corresponding to less than bit0is subjected to a predetermined rounding process, while data that exceeds the bit width of the register specified by the operand is subjected to a saturation process. The shift amount is a difference (eo−e) between the decimal point position e and the output decimal point position eo.FIG. 3illustrates a region that can be represented by a 16-bit fixed point, a region that is to be saturated, and a region where underflow occurs, supposing that the shift amount is 15 bits.

Furthermore, the numerical values given to the horizontal axis ofFIG. 3indicate the numerical values that can be represented by a fixed point. For example, when the information processing device1alters the decimal point position eo by −2, the region to be saturated is expanded by 2 bits, and the region in which the underflow occurs is decreased by 2 bits. In addition, for example, when the information processing device1alters the decimal point position eo by +2, the region to be saturated is decreased by 2 bits, and the region in which the underflow occurs is expanded by 2 bits.

The information processing device1may determine an appropriate fixed-point position by obtaining the distribution of the position of leftmost set bit for positive number and position of leftmost zero bit for negative number, during learning execution. For example, the information processing device1can determine the fixed-point position such that the data to be saturated is equal to or less than a specified ratio. This means that, as an example, the information processing device1can determine the fixed-point position prior to the data saturation becoming a predetermined degree rather than the data underflow becoming a predetermined degree.

Note that, as statistical information, instead of the distribution of the position of leftmost set bit for positive number and position of leftmost zero bit for negative number, the information processing device1may use the distribution of the non-sign least significant bit positions, the maximum value at the position of leftmost set bit for positive number and position of leftmost zero bit for negative number, or the minimum value at the non-sign least significant bit position.

Here, the distribution of the non-sign least significant bit positions means the distribution of the positions of the least significant bits where the bits have different values from the signs. For example, when the bits are placed in an array from the most significant bit being bit[39] to the least significant bit being bit[0], the least significant bit position is the position of a bit with the smallest index k among the bits[k] different from the sign bit bit[39]. In the distribution of the non-sign least significant bit positions, a least significant bit induding valid data is grasped.

Furthermore, the maximum value at the position of leftmost set bit for positive number and position of leftmost zero bit for negative number is the maximum value among the values at the most significant bit positions that have values different from the value of the sign bit for one or more fixed-point numbers targeted for instruction execution from the time when the statistical information storage circuit was cleared by a clear instruction to the present time. The information processing device1can use the maximum value at the position of leftmost set bit for positive number and position of leftmost zero bit for negative number to determine an appropriate decimal point position of the dynamic fixed-point number.

The minimum value at the non-sign least significant bit position is the minimum value among the values at the least significant bit positions that have values different from the value of the signs for one or more fixed-point numbers from the time when the statistical information storage circuit was cleared by a clear instruction to the present time. The information processing device1can use the minimum value at the non-sign least significant bit position to determine an appropriate decimal point position of the dynamic fixed-point number.

Next, the allocation unit40will be described. The information processing device1executes all the operations performed in deep learning in parallel as much as possible in order to effectively utilize the PEs12. Here, the information processing device1collectively perform operations of the mini-batches to proceed with the learning.

Taking the operation of the convolution layer as an example, it is assumed that the filter size is 3×3, the number of images in the mini-batch is N, the number of input channels is Cin, the number of output channels is Cout, the height of the image is H, and the width of the image is W. The number of pixels of data to be input is N*Cin*(H+2)*(W+2). Here, “*” indicates multiplication. Furthermore, “2” indicates the number of paddings at two ends in a height direction or a width direction of the image. The number of pixels of the filter to be input is Cin*Cout*3*3. The number of results to be output is N*Cout*H*W. The operation content is indicated by following expression (1).

As illustrated in expression (1), the operation of the convolution layer can be computed independently between each of the image (n), the output channel (co), and the pixel (h, w). In addition, since the input pixel data and filter data are used many times, it is efficient to achieve parallelization in an image direction and an output channel direction in this order, in order to enhance the efficiency of data transfer between the DRAM13and the PEs12.

Thus, as illustrated inFIG. 4, it is conceivable to mechanically allocate the images and output channels to the PEs12. InFIG. 4, the total number of PEs12is N*Cout. Furthermore, when the number of PEs12placed side by side is denoted by X, the thinning rate is 1/X, and the number of information acquisition PEs12ais N*Cout/X.

In this allocation, only the statistical information on a specific image such as an image #0 and a specific output channel such as an output channel #0 is acquired. The statistical information on an image #1, an image #(N−1), and the like, and the output channels such as an output channel #1 and an output channel #(Cout−1) is not acquired. For this reason, the statistical information will be different compared with a case where the statistical information is not thinned out.

FIGS. 5A and 5Bare diagrams illustrating an example of the influence of thinning out on the statistical information. The vertical axis indicates the number of pieces of data. The number of pieces of data is expressed as a percentage to the number of all pieces of data. The series of negative integers on the horizontal axis denotes the values of the exponential parts when the data is expressed in binary.FIGS. 5A(a) and5B(a) illustrate statistical information for four cases: a case without thinning out, a case with image thinning out, a case with output channel thinning out, and a case with image×output channel thinning out. The image thinning rate and the output channel thinning rate are 1/4 each.

Furthermore,FIGS. 5A(b) and5B(b) are diagrams in which the range from −14 to −19 of each series is individually enlarged. InFIGS. 5AB and 5BB, the horizontal lines indicate rmax, which is a threshold value for determining the decimal point position. Here, rmax=0.002% is employed. The vertical lines indicate the upper limit of a representable range that does not exceed rmax.

As illustrated inFIGS. 5A(a) and5B(a), the distribution when thinned out is different from the distribution when not thinned out. Furthermore, as illustrated inFIG. 5A(b), the most significant bit in the representable range is “−18” when not thinned out, but the most significant bit in the representable range is “−15” or “−16” when thinning out is performed. Furthermore, as illustrated inFIG. 5B(b), the most significant bit in the representable range is“−17” when not thinned out, but the most significant bit in the representable range is “−16” or “−18” when thinning out is performed.

In this manner, if the images and output channels are mechanically allocated to the PEs12, the statistical information will be different from a case where the statistical information is not thinned out.

FIG. 6Ais a diagram for explaining the reason why the statistical information is different when the output channels are mechanically allocated to the PEs12as compared with a case where the statistical information is not thinned out. In addition,FIG. 6Bis a diagram for explaining the reason why the statistical information is different when the images are mechanically allocated to the PEs12as compared with a case where the statistical information is not thinned out.

FIG. 6Aillustrates a case where the statistical information is acquired for output channels #0, #4, #8, . . . . As illustrated inFIG. 6A, in deep learning, various filters are applied to the input image. The filter pattern changes with learning, but when a filter (output channel) with a similar pattern is targeted for acquiring the statistical information, the information is biased. Since the filter pattern changes as the learning progresses, it is difficult to control the similarity between the patterns.

InFIG. 6B, the thinning rates of the output channels and the images are ¼. As illustrated inFIG. 6B, when the statistical information is acquired for only one of four images, ¾ of the images are not involved in the decimal point position determination. Therefore, when the images with the solid line frames are targeted for acquiring the statistical information among the images in one mini-batch, the data is biased and the statistical information is biased because the images have similar features (quadrupeds).

In view of this, the allocation unit40allocates the PEs12such that all images and all output channels are targeted for acquiring the statistical information.FIG. 7is a diagram illustrating an allocation example by the allocation unit40. InFIG. 7, the images are not thinned out. Furthermore, the thinning rate of the output channels is 1/16, and N is a multiple of 16.

As illustrated inFIG. 7, the allocation unit40rotates the output channels for each image to allocate the output channels to the PEs12. For example, when the remainder obtained by dividing the image number by 16 is 0, the allocation unit40allocates output channels #0, #16, #32, . . . to the information acquisition PEs12a. Furthermore, when the remainder obtained by dividing the image number by 16 is 1, the allocation unit40allocates output channels #1, #17, #33, . . . to the information acquisition PEs12a. Similarly, in the image #(N−1), the allocation unit40allocates output channels #15, #31, . . . , #(Cout−1) to the information acquisition PEs12a.

In this manner, since the allocation unit40rotates the output channels for each image to allocate the output channels to the PEs12, even when the information acquisition PEs12aare thinned out as a part of the whole PEs12, a bias in the statistical information may be mitigated.

FIG. 8is a diagram illustrating another allocation example by the allocation unit40. InFIG. 8, the thinning rates of the images and the output channels are 1/4. As illustrated inFIG. 8, the allocation unit40allocates the information acquisition PEs12ato 1/4 of the images, and in regard to the images to which the information acquisition PEs12a, allocates the output channels to the PEs12by rotating the output channels for each image.

For example, the allocation unit40allocates the information acquisition PEs12ato images #0, #4, #8, . . . , but does not allocate the information acquisition PEs12ato images #1, #2, #3, #5, #6, #7, . . . . Then, when the remainder obtained by dividing the image number by 16 is 0, the allocation unit40allocates output channels #0, #4, #8, . . . to the information acquisition PEs12a. Furthermore, when the remainder obtained by dividing the image number by 16 is 4, the allocation unit40allocates output channels #1, #5, #9, . . . to the information acquisition PEs12a. Similarly, when the remainder obtained by dividing the image number by 16 is 12, the allocation unit40allocates output channels #3, #7, #11, . . . to the information acquisition PEs12a.

In this manner, since the allocation unit40rotates the output channels for each image to allocate the output channels to the PEs12in regard to the images to which the information acquisition PEs12aare allocated, even when the information acquisition PEs12aare thinned out as a part of the whole PEs12, a bias in the statistical information may be mitigated.

Next, the flow of a learning process by the information processing device1will be described.FIG. 9is a sequence diagram illustrating a flow of the learning process by the information processing device1. As illustrated inFIG. 9, the host20creates a graph representing a neural network and reserves a region (step S1). Here, the graph representing the neural network is, for example, a graph made up of the convolution layer #1, the pooling layer #1, the convolution layer #2, the pooling layer #2, the fully connected layer #1, and the fully connected layer #2 illustrated inFIG. 2. Furthermore, the region is a place to store a parameter. The host20then generates an initial value of the parameter (step S2). Note that the host20may read the initial value from a file instead of generating the initial value.

Then, the host20repeats the processes in steps S3to S11until an end condition for learning is satisfied. The end conditions for learning include, for example, the number of times of learning and the fulfillment of a desired value. As repetitive processes performed on the accelerator board10, the host20loads the learning data (step S3) and calls a layer's forward propagation operation (step S4) in a forward direction of the layers. The propagation operation is a convolution operation in the convolution layer, a pooling operation in the pooling layer, and a fully connected operation in the fully connected layer.

When called by the host20, the accelerator board10executes the forward propagation operation (step S5). Then, the host20calls a layer's backpropagation operation (step S6) on the accelerator board10in a reverse direction of the layers. When called by the host20, the accelerator board10executes the backpropagation operation (step S7).

Then, the host20instructs the accelerator board10to update the parameter (step S8). When instructed by the host20, the accelerator board10executes the parameter update (step S9). Then, the host20determines the decimal point position of the dynamic fixed-point number based on the statistical information, and instructs the accelerator board10to update the decimal point position (step S10). When instructed by the host20, the accelerator board10executes the decimal point position update (step S11).

FIGS. 10A and 10Bare diagrams for explaining calls for the propagation operation.FIG. 10Aillustrates a basic form, andFIG. 10Billustrates a derivative form. As illustrated inFIG. 10A, in the basic form, the host20performs PE allocation (step S21) and calls the propagation operation on the accelerator board10together with PE allocation information, an input data address, and an output data address (step S22). Then, the accelerator board10executes the propagation operation (step S23) and transmits an end notification to the host20.

In this manner, in the basic form, since the host20performs the PE allocation, the host20instructs the accelerator board10to execute the propagation operation together with the PE allocation information.

On the other hand, in the derivative form, the host20calls the propagation operation on the accelerator board10together with the input data address and the output data address (step S26), as illustrated inFIG. 10B. Then, the controller11of the accelerator board10performs PE allocation (step S27) and executes a PE operation call for each PE12(step S28). Subsequently, each PE12executes the operation (step S29). Thereafter, the controller11waits for the end of all the operations (step S30), and when the wait is completed, transmits an end notification to the host20.

In this manner, in the derivative form, since the controller11performs the PE allocation, the host20instructs the accelerator board10to execute the propagation operation without the PE allocation information.

Next, the flow of an allocation process will be described with reference toFIGS. 11 to 16.FIG. 11is a flowchart illustrating the flow of an allocation process when the images and the output channels are mechanically allocated to the PEs12, andFIG. 12is a diagram for explaining the variables illustrated inFIG. 11.

InFIGS. 11 to 16, N denotes the number of images and Cout denotes the number of output channels. An image # expression denotes an image whose identification number is the value of the expression, an output channel # expression denotes an output channel whose identification number is the value of the expression, and PE #p denotes a PE12whose identification number is p. InFIGS. 11, 12, 15, and 16, the thinning rate in the image direction is 1/X, and the thinning rate in the output channel direction is 1/Y. InFIGS. 13 and 14, the thinning rate in the output channel direction is 1/X.

Note that it is assumed that NLis a multiple of X and Cout is a multiple of Y. NLdenotes the number of images allocated at one time. For example, when N is assumed as a multiple of NLand the number of PEs12is denoted by NP, the product of the total number of allocations=NPand the number of times of allocation to all PEs12=NP*(N/NL) holds. Meanwhile, since the total number of allocations=N*Cout holds, NP*(N/Ni)=N*Cout holds. Therefore, NP/NL=Cout holds, and NP/Cout=NLholds. CEIL(x) is a function that rounds up x to an integer.

Furthermore, inFIGS. 11 and 12, i denotes a variable for counting the number of times of allocation to all PEs12, and is incremented by NLfrom 0 within a range not exceeding N−1. The sign p denotes a number that identifies the PE12. The sign n denotes a variable for counting the number of times of image allocation, and is incremented by 1 from 0 to NL−1. The sign c denotes a variable for counting the number of times of allocation of Cout output channels, and is incremented by 1 from 0 to Cout−1. The sign j denotes a variable for counting the number of times of allocation of X images, and is given as the quotient of n divided by X. The sign k denotes a variable for counting the number of image allocations in the allocation of the X images, and is given as a remainder obtained by dividing n by X. The sign l denotes a variable for counting the number of times of allocation of Y output channels in the allocation to one image, and is given as the quotient of c divided by Y. The sign m denotes a variable for counting the number of output channel allocations in the allocation of Y output channels, and is given as a remainder obtained by dividing c by Y.

As illustrated inFIG. 11, the allocation unit40computes CEIL(NP/Cout) and sets CEIL(NP/Cout) in NL(step S31). Here, the allocation unit40mechanically allocates the images and the output channels to the PEs12. Then, the allocation unit40repeats the process of allocating one combination of the image and the output channel to each PE12entirely N/NLtimes.

The allocation unit40increments n by 1 from 0 to NL−1, and allocates the output channels of an image #n to the PEs12. The allocation unit40computes the variables j and k, and sets k*Y+j*Cout in a variable p0 that represents the top PE number to which the image #n is allocated (step S32). The allocation unit40increments c by 1 from 0 to Cout−1, and repeats the process of allocating the output channel #c of the image #n to the PE12Cout times.

In one process of allocating one combination of the image and the output channel to each PE12entirely, the allocation unit40computes the variables l and m to set m+l*X*Y in a variable p1 that represents the relative value of the PE number to which the channel #c is allocated (step S33), and allocates an image #(n+i*NL) and the output channel #c to PE #(p0+p1) (step S34). The allocation unit40increments c by 1 from 0 to Cout−1, and repeats steps S33and S34.

FIG. 13is a flowchart illustrating the flow of the allocation process by the allocation unit40, andFIG. 14is a diagram for explaining the variables illustrated inFIG. 13.

Furthermore, inFIGS. 13 and 14, i denotes a variable for counting the number of times of allocation to all PEs12, and is incremented by NLfrom 0 within a range not exceeding N−1. The sign n denotes a variable for counting the number of image allocations in the allocation to all PEs12, and is incremented by 1 from 0 to NL−1. The sign c denotes a variable for counting the number of times of allocation of Cout output channels, and is incremented by 1 from 0 to Cout−1.

As illustrated inFIG. 13, the allocation unit40computes CEIL(NP/Cout) and sets CEIL(NP/Cout) in NL(step S41). Then, the process of allocating one combination of the image and the output channel to each PE12entirely is repeated N/NLtimes. Then, the allocation unit40increments n by 1 from 0 to NL−1, and allocates the output channels of the image #n to the PEs12.

The allocation unit40sets n Cout in the variable p0 that represents the top PE number to which the image #n is allocated (step S42). The allocation unit40increments c by 1 from 0 to Cout−1, and repeats the process of allocating the output channel #c of the image #n to the PE12Cout times.

In one process of allocating one combination of the image and the output channel to each PE12entirely, the allocation unit40sets (c−n+Cout) % Cout in a variable c′ for the channel #c to set c′ in the variable p1 that represents the relative value of the PE number to which the channel #n is allocated (step S43), and allocates the image #(n+i*NL) and the output channel #c to PE #(p0+p1) (step S44). For example, the allocation unit40shifts the output channels using n in step S43. The allocation unit40increments c by 1 from 0 to Cout, and repeats steps S43and S44.

In this manner, when allocating the combination of the images and the output channels to the PEs12, the allocation unit40shifts the output channels using n, which means to rotate the output channels for each image, such that a bias in the statistical information may be mitigated.

FIG. 15is a flowchart illustrating the flow of a process for the another allocation illustrated inFIG. 8by the allocation unit40, andFIG. 16is a diagram for explaining the variables illustrated inFIG. 15. ComparingFIGS. 11 and 15andFIGS. 12 and 16, the process in step S53is different from the process in step S33inFIG. 15. For example, (c−j+Cout) % Cout is set in the variable c′, and the variables l and m are set using the variable c′ instead of the variable c. The allocation unit40performs n+i*NL, which means to shift the output channels using j.

In this manner, when allocating the combination of the images and the output channels to the PEs12, the allocation unit40shifts the output channels using j, which means to rotate the output channels for each allocation of X images, such that a bias in the statistical information may be mitigated.

Next, the effect of allocation by the allocation unit40will be described.FIGS. 17A and 17Bare diagrams for explaining the effect of allocation by the allocation unit40. As illustrated inFIGS. 17A(a) and17B(a), the distribution when the allocation according to the embodiment is performed is similar to the distribution when no thinning out is performed, as compared with the other cases where thinning out is performed. Furthermore, as illustrated inFIG. 17A(b), the most significant bit in the representable range is “−18”, which is the same as the case where no thinning out is performed, even when thinning out is performed. In addition, as illustrated inFIG. 17B(b), the most significant bit in the representable range is “−17”, which is the same as the case where no thinning out is performed, even when thinning out is performed.

As described above, in the embodiment, the accelerator board10includes the information acquisition PEs12aas a part of the whole PEs12. Furthermore, when allocating the layer's propagation operation of deep learning to the PEs12, the allocation unit40of the host20evenly allocates the information acquisition PEs12afor every certain number of images, and rotates the output channels for every certain number of images to allocate the output channels to the PEs12. Therefore, the information processing device1may suppress a bias in the statistical information and may suppress the deterioration of the learning accuracy.

Furthermore, in the embodiment, the allocation unit40evenly allocates the information acquisition PEs12afor each image, and rotates the output channels for each image to allocate the output channels to the PEs12, such that a bias in the statistical information may be suppressed.

In addition, in the embodiment, when allocating the propagation operation in the convolution layer of deep learning to the PEs12, the allocation unit40evenly allocates the information acquisition PEs12afor every certain number of images, and rotates the output channels for every certain number of images to allocate the output channels to the PEs12. Therefore, the information processing device1may suppress a bias in the statistical information acquired in the propagation operation in the convolution layer.

Besides, in the embodiment, the controller11of the accelerator board10may perform the allocation process instead of the allocation unit40, such that the load on the host20may be lowered.

Additionally, in the embodiment, the case of learning images has been described, but the information processing device1may learn other data.