Patent Description:
Patent Document <NUM> discloses a recognition device having a hierarchical network including a plurality of layers of an input layer, a middle layer, and an output layer. The recognition device performs recognition using a network structure and weight data trained by a training device. An example for a software accelerator is disclosed in "<NPL>). Another custom hardware accelerator is disclosed in "<NPL>.

In general, a recognition device may be equipment such as a mobile terminal or a camera which is reduced in size but lower in a processing capability as compared with a training device. As an image sensor is enhanced in performance and a capacity for communication data is increased, a processing time for input data and a resource required for processing tend to increase in the recognition device.

In this technical field, as for a device performing calculation for processing input data using a network structure and weight data, there is a demand for a calculation scheme decision system, a calculation scheme decision device, a processing device, a calculation scheme decision method, a processing method, a calculation scheme decision program, and a processing program which are capable of deciding a calculation scheme taking into account a calculation cost.

An aspect of the present invention is a calculation scheme decision system including a pre-calculation unit configured to perform, in an execution environment in which calculation for processing input data is performed using a network structure and weight data, calculation for each of respective layers of the network structure on predetermined data using at least one of calculation schemes prepared in advance for the respective layers of the network structure, a cost acquisition unit configured to acquire a calculation cost of the at least one calculation scheme for each of the respective layers of the network structure based on a result of the calculation by the pre-calculation unit, a decision unit configured to select one calculation scheme for each of the respective layers of the network structure based on the calculation cost from among the at least one of the calculation schemes prepared in advance for the respective layers to associate the layer of the network structure with the selected one calculation scheme, and a calculation unit configured to perform the calculation for each of the respective layers of the network structure on the input data in the execution environment using the calculation scheme associated with each layer of the network structure by the decision unit.

In the calculation scheme decision system, one calculation scheme is selected for each layer, from among the calculation schemes prepared in advance, based on the calculation cost obtained by actually operating those calculation schemes in the execution environment. In this way, the calculation scheme decision system can take into account the calculation cost to decide an optimal calculation scheme for each layer. Therefore, the calculation scheme decision system can take into account the calculation cost to decide the calculation scheme.

In an embodiment, the at least one calculation scheme may include a plurality of algorithms executable in the execution environment and exerting the same function using arithmetic operations different from each other. In this case, the calculation scheme decision system can actually operate, in the execution environment, a plurality of algorithms which output the same results but are different from each other in the arithmetic operation method, and decide an optimal algorithm based on the obtained calculation costs.

In an embodiment, at least one calculation scheme may include a plurality of algorithms executable in the execution environment and performing the same arithmetic operation using resources different from each other. In this case, the calculation scheme decision system can actually operate, in the execution environment, a plurality of algorithms which use the same arithmetic operation and output the same results but are different from each other in the usage resource, and decide an optimal algorithm based on the obtained calculation costs.

In an embodiment, the calculation scheme decision system may further include a candidate decision unit configured to prepare the at least one calculation scheme for each of the respective layers of the network structure based on the execution environment. In this case, the calculation scheme decision system can prepare the calculation scheme taking into account the resources in the execution environment.

In an embodiment, the calculation scheme decision system may include a conversion unit configured to acquire the network structure and the weight data from an external device and convert the acquired network structure and weight data into a predetermined format. In this case, the calculation scheme decision system can deal with the training result of the external device in the predetermined format.

In an embodiment, the calculation scheme decision system may further include a first adjustment unit configured to change a data order of the weight data based on the execution environment, wherein the pre-calculation unit and the calculation unit perform calculation based on the weight data adjusted by the first adjustment unit. In this case, the calculation scheme decision system can change the data array of the weight data such that the data becomes a state of being easy to be placed on a cache.

In an embodiment, the calculation scheme decision system may further include a second adjustment unit configured to perform a preprocess related to the calculation of the calculation unit on the weight data, wherein the pre-calculation unit and the calculation unit perform calculation based on the weight data adjusted by the second adjustment unit. In this case, the calculation scheme decision system can reduce a calculation load on the calculation unit.

In an embodiment, the pre-calculation unit and the calculation unit are included in a first terminal, and the calculation scheme decision system may further include a provision unit configured to provide the calculation scheme decided by the decision unit to a second terminal having an execution environment the same as an execution environment of the first terminal. In this case, the calculation scheme decision system can, without calculating in a terminal in the same environment, apply the calculation scheme decided in another terminal having the same environment.

Another aspect of the present invention is a calculation scheme decision computer-implemented method including performing, in an execution environment in which calculation for processing input data is performed using a network structure and weight data, calculation for each of respective layers of the network structure on predetermined data using at least one of calculation schemes prepared in advance for the respective layers of the network structure, acquiring a calculation cost of the at least one calculation scheme for each of the respective layers of the network structure based on a result of the calculation for each of the respective layers of the network structure, selecting one calculation scheme for each of the respective layers of the network structure based on the calculation cost from among the at least one of the calculation schemes prepared in advance for the respective layers to associate the layer of the network structure with the selected one calculation scheme, and performing the calculation for each of the respective layers of the network structure on the input data in the execution environment using the calculation scheme associated with each layer of the network structure.

The calculation scheme decision method exerts the same effect as the calculation scheme decision system described above.

Another aspect of the present invention is a calculation scheme decision program causing a computer to operate. The calculation scheme decision program causes the computer to function as a pre-calculation unit configured to perform, in an execution environment in which calculation for processing input data is performed using a network structure and weight data, calculation for each of respective layers of the network structure on predetermined data using at least one of calculation schemes prepared in advance for the respective layers of the network structure, a cost acquisition unit configured to acquire a calculation cost of the at least one calculation scheme for each of the respective layers of the network structure based on a result of the calculation by the pre-calculation unit, a decision unit configured to select one calculation scheme for each of the respective layers of the network structure based on the calculation cost from among the at least one of the calculation schemes prepared in advance for the respective layers to associate the layer of the network structure with the selected one calculation scheme, and a calculation unit configured to perform the calculation for each of the respective layers of the network structure on the input data in the execution environment using the calculation scheme associated with each layer of the network structure by the decision unit.

The calculation scheme decision program exerts the same effect as the calculation scheme decision system described above.

According to various aspects of the present invention, it is possible to change a calculation scheme taking into account a calculation cost in a device performing calculation for processing input data using a network structure and weight data.

Hereinafter, a description is given of embodiments with reference to the drawings. Note that in description of the drawings the same components are designated by the same reference signs, and the duplicated description is not repeated.

A calculation scheme decision system <NUM> (see <FIG>) according to an embodiment is a system which decides an optimal calculation scheme for a processing device having a hierarchical network. The hierarchical network is a network having a hierarchical structure, and examples of the hierarchical network include a neural network. The neural network is defined using a network structure, weight data, and the like. The neural network is described later in detail.

The processing device has an execution environment capable of executing various programs. In the execution environment, calculation for processing input data is performed using the network structure and weight data. The input data is data processed to achieve an aim of the hierarchical network. For example, the input data is recognition target data when the aim of the hierarchical network is recognition. The processing device is not specifically limited so long as it is a device having a hierarchical network. The processing device may be a terminal device labeling content of an image, a monitoring camera identifying an object position within an image (for example, a position of a person, or the like), or a general-purpose personal computer, as an example. In the following description, as an example of the processing device, a terminal device including a recognition unit <NUM> which recognizes content of recognition target data using a neural network is described (see <FIG> and <FIG>, an example of a calculation unit). The recognition target data is data to be recognized by a computer, and examples of the recognition target data include image data, voice data, and text data.

A calculation scheme for the recognition unit <NUM> decided by the calculation scheme decision system <NUM> is a scheme for performing an arithmetic operation on an input and outputting a result. The calculation scheme decision system <NUM>, on an assumption that the same result is output for an input with a predetermined accuracy performance, decides an optimal algorithm from among a plurality of kinds of executable algorithms, or an optimal algorithm from among a plurality of identical algorithms different in a usage amount or usage aspect of resources. Examples of the resource include a hardware resource and a software resource. Examples of the hardware resource include a CPU or a cache for performing the arithmetic operation. Examples of the software resource include a library. A resource used by an algorithm may be defined by parameters of the algorithm, for example.

The calculation scheme decision system <NUM> takes into account a calculation cost to decide the calculation scheme for the recognition unit <NUM>. The calculation cost is evaluated by a time required for calculation as an example. In this case, the longer the time required for calculation, the more the calculation cost. The calculation cost may be evaluated by the usage amount of resources. In this case, the more the usage amount of resources, the more the calculation cost. The calculation cost may be evaluated by two of the time required for calculation and the usage amount of resources.

The calculation scheme decision system <NUM> decides a calculation scheme having the lowest calculation cost from among the calculation schemes executable in the execution environment for the recognition unit <NUM>. The calculation scheme decision system <NUM> may select the calculation scheme having the calculation cost lower than an average of the calculation costs of the calculation schemes executable. In this way, the calculation scheme decision system <NUM> optimizes a calculation speed of the recognition unit <NUM>, or optimizes the resource usage amount used by the recognition unit <NUM>.

The calculation scheme decision system <NUM> exhaustively performs the calculation schemes executable in the execution environment for the recognition unit <NUM> in deciding the calculation scheme, and evaluates the calculation schemes based on results of the performing to decide an optimal calculation scheme.

As a specific example, the calculation scheme decision system <NUM> is configured to be able to execute performing calculation for each of respective layers of the network structure on predetermined data using at least one of calculation schemes prepared in advance for the respective layers of the network structure in the execution environment for the recognition unit <NUM>, acquiring the calculation cost of at least one calculation schemes for the respective layers of the network structure based on a result of the calculation for each of the respective layers of the network structure, selecting one calculation scheme for each of the respective layers of the network structure, based on the calculation cost, from among at least one of the calculation schemes prepared in advance for the respective layers to associate the layer of the network structure with the selected one calculation scheme, and performing calculation for each of the respective layers of the network structure on the input data in the execution environment using the calculation scheme associated with each layer of the network structure.

First, a description is given of the recognition unit <NUM> for which the calculation scheme decision system decides the calculation scheme. Hereinafter, a description is given, as an example, of a case that the recognition target data is the image data, and a target to be recognized is content of an image (person, animal, object, scenery, room, etc.).

<FIG> is a diagram illustrating the recognition unit <NUM>. As shown in <FIG>, the recognition unit <NUM> is included in a terminal device <NUM>. The recognition unit <NUM> inputs recognition target data G1 that is image data, and outputs a recognition result. As shown in (A) of <FIG>, the recognition target data G1 is the image data in which a dog is drawn. The recognition unit <NUM> inputs the image data (to be more specific, pixel value), and outputs a label representing content of the image by use of a training result (for example, network structure and weight data). The label, which is used to classify the content of the recognition target data, is information for identifying a category set in advance by a system user. In the example in (A) of <FIG>, the recognition unit <NUM> outputs a label of "dog". The label is given to the recognition target data G1 by the recognition unit <NUM>. The term giving means making an association, and, for example, only a relationship between the recognition target data G1 and the label may be recorded by means of an association table or the like, or the label may be incorporated into the recognition target data G1 itself. In general, the label being given to the recognition target data is referred to as annotation. The recognition unit <NUM>, because of being capable of inputting the image data to give the label, can automatically classify the image data, or search a web for a desired image.

When there is a plurality of labels set in advance by the system user, there are a single-labeling process of giving the most possible label among the labels to the recognition target data, and a multi-labeling process of giving all labels having a certain possibility to the recognition target data. As shown in (B) of <FIG>, recognition target data G2 is image data in which a person and flowers are drawn. In a case that the recognition unit <NUM> performs the single-labeling process, a label of "person" is given to the recognition target data G2 by the recognition unit <NUM>. In a case that the recognition unit <NUM> performs the multi-labeling process, two labels, the label of "person" and a label of "flower" are given to the recognition target data G2 by the recognition unit <NUM>.

<FIG> is a diagram illustrating a neural network in the recognition unit <NUM>. The recognition unit <NUM> recognizes a label corresponding to the image data by use of the neural network. The neural network is an information processing system which models a cerebral nervous system. As shown in <FIG>, the neural network in the recognition unit <NUM> is a so-called hierarchical neural network, in which many artificial neurons illustrated by circles form a hierarchy and are coupled to each other. The hierarchical neural network includes artificial neurons for input, artificial neurons for processing, and artificial neurons for output.

The artificial neurons for input acquire the recognition target data to distribute to the artificial neurons for processing. Hereinafter, a signal itself communicated in the neural network is referred to as a score. The score is a numerical value. The artificial neurons for input are arranged in parallel to form an input layer <NUM>.

The artificial neurons for processing which are connected with the artificial neurons for input process inputs in accordance with functions of the artificial neurons to deliver outputs to other neurons. The artificial neurons for processing are arranged in parallel to form an intermediate layer <NUM>. The intermediate layer <NUM> may be a plurality of layers. Note that three or more hierarchies of neural networks including the intermediate layer <NUM> are called a deep neural network.

The artificial neurons for output externally output the recognition score. The number of artificial neurons for output to be prepared is the same as the number of labels. In other words, the recognition score is output for each label in the neural network. In the example in <FIG>, three artificial neurons are prepared correspondingly to three labels "dog", "person", and "flower". The artificial neurons for output outputs a recognition score B1 corresponding to the label of "dog", a recognition score B2 corresponding to the label of "person", and a recognition score B3 corresponding to the label of "flower". The recognition score is a score representing a recognition result, and in a case of training assuming that the positive evaluation is "<NUM>" and the negative evaluation "<NUM>", the higher the recognition score of the label, the higher a possibility that the label represents the content of the image. The artificial neurons for output are arranged in parallel to form an output layer <NUM>.

The recognition unit <NUM> uses the recognition score output by the output layer <NUM> to decide a label to be given. For example, the recognition unit <NUM> gives the label corresponding to the recognition score equal to or more than a predetermined value to the recognition target data. This allows the recognition target data to be automatically given the label representing the recognition target data. Note that in the case of the single-labeling process, the recognition unit <NUM> gives the label corresponding to the highest recognition score to the recognition target data.

<FIG> is a diagram illustrating the artificial neurons shown in <FIG>. The artificial neurons shown in (A) of <FIG> input, x<NUM>, x<NUM>, and x<NUM>, and multiply these by corresponding weight coefficients w<NUM>, w<NUM>, and w<NUM>, respectively. The artificial neurons compute a sum of an integrated value (x<NUM>·w<NUM>·x<NUM>·w<NUM>·x<NUM>·w<NUM>) and a bias value b. The sum is substituted into an activating function to obtain an output of the artificial neurons.

To be more specific, the output of the targeted artificial neurons is as the following mathematical expression. <MAT> Here, g represents an activating function, for example, a sigmoid function.

(B) of <FIG> is a diagram illustrating the artificial neurons of N hierarchies (N = <NUM>). As shown in (B) of <FIG>, in a case of three hierarchies, outputs h<NUM>(<NUM>), h<NUM>(<NUM>), and h<NUM>(<NUM>) of the artificial neurons in the second hierarchy are as the following mathematical expressions <NUM> to <NUM>, respectively. Here, n represents the number of artificial neurons in the targeted hierarchy, w1j(<NUM>) represents a weight coefficient for the first artificial neuron in the second hierarchy corresponding to an output of the j-th output in the first hierarchy, and b<NUM>(<NUM>) represents a bias value for the first hierarchy. <MAT> <MAT> <MAT> where, w2j(<NUM>) represents a weight coefficient for the second artificial neuron in the second hierarchy corresponding to an output of the j-th output in the first hierarchy, w3j(<NUM>) represents a weight coefficient for the third artificial neuron in the second hierarchy corresponding to an output of the j-th output in the first hierarchy, b<NUM>(<NUM>) represents the second bias value in the first hierarchy, and b<NUM>(<NUM>) represents the third bias value in the first hierarchy. Accordingly, an output h<NUM>(<NUM>) of the artificial neuron in the third hierarchy is represented by the following mathematical expression <NUM>. <MAT> Note that the bias value b is not necessarily needed, and only the integrated value of the output of the artificial neuron and weight coefficient in the previous hierarchy may be used to calculate the output.

The artificial neurons are not limited to the above, and may be those generalized. A general expression for a function of the i-th intermediate layer <NUM> is as the following mathematical expression <NUM>. <MAT> Here, x(i) represents an input vector to the intermediate layer <NUM>, w(i) represents a weight parameter vector for the intermediate layer <NUM>, and b(i) represent a bias vector, and v(i) represents an output vector of the intermediate layer <NUM>. Examples of the intermediate layer <NUM> generally used in image recognition include a fully connected layer and a convolutional layer. The output of the fully connected layer illustrated in <FIG> is generally as the following mathematical expression <NUM>. <MAT> Here, xp(i) represents the p-th component of the input of the i-th intermediate layer <NUM>, vq(i) represents the q-th component of the output of the intermediate layer <NUM>, and wp, q(i) represents p, q components of the weight coefficient for the intermediate layer <NUM>. The output of the convolutional layer is as the following mathematical expression <NUM>. <MAT> Here, xp, (r, s)(i) represents a (r, s) component on the p-th channel of the input of the i-th intermediate layer <NUM>, vq, (r, s)(i) represents a (r, s) component on the q-th channel of the output of the intermediate layer <NUM>, and wp, q, (r', s')(i) represents a weight coefficient for a convolutional filter of the intermediate layer <NUM>. r' and s' vary from <NUM> to values (width - <NUM>), and (height - <NUM>) of the convolutional filter. By repeating the calculation of the intermediate layer <NUM> and activating function g(i) as above, the output of the intermediate layer immediately before the output layer <NUM> is as the following mathematical expression <NUM>.

The network structure, weight coefficient (weight data), and bias value described above are those trained by a training device <NUM> described later, and delivered as the training results to the recognition unit <NUM>. In other words, the training device <NUM> is a device training the network structure, weight coefficient, and bias value which make a feature amount of the recognition target data correspond to the label indicating the content thereof. Note that when the recognition unit <NUM> does not use the bias value b, the training device <NUM> trains only the network structure and weight coefficient.

<FIG> is a functional block diagram of the calculation scheme decision system <NUM> according to the embodiment. As shown in <FIG>, the calculation scheme decision system <NUM> includes the terminal device <NUM> and a conversion device <NUM>, and is connected to the training device <NUM>. The training device <NUM> collects and is trained with the image data. The training result of the training device <NUM> is provided via the conversion device <NUM> to the terminal device <NUM>.

First, a description is given of hardware of the terminal device <NUM>, the training device <NUM>, and th conversion device <NUM>. <FIG> is s block diagram showing a hardware configuration of the device shown in <FIG>. As shown in <FIG>, the terminal device <NUM> is configured physically as an ordinary computer system including a CPU (Central Processing Unit) <NUM>, a main memory such as a RAM (Random Access Memory) <NUM> and a ROM (Read Only Memory) <NUM>, an input device <NUM> such as a touch panel and a keyboard, an output device <NUM> such as a display, an auxiliary storage device <NUM> such as a hard disk, and the like. Functions of the terminal device <NUM> are implemented by that the CPU <NUM> reads predetermined computer software into the hardware such as RAM <NUM> and ROM <NUM> to cause the input device <NUM> and output device <NUM> to operate under control of the CPU <NUM>, and to read and write data from and into the main memory or the auxiliary storage device <NUM>. Note that the terminal device <NUM> may include hardware other than those described above. For example, the terminal device <NUM> may include a GPU (Graphics Processing Unit), a FPGA (Field-Programmable Gate Array), a DSP (Digital Signal Processor), or the like.

The hardware of the training device <NUM> and conversion device <NUM> can be also configured by the hardware the same as the terminal device <NUM>. Specifically, the training device <NUM> is configured physically as an ordinary computer system including a CPU <NUM>, a main memory such as a RAM <NUM> and a ROM <NUM>, an input device <NUM>, an output device <NUM>, an auxiliary storage device <NUM>, and the like. The conversion device <NUM> is configured physically as an ordinary computer system including a CPU <NUM>, a main memory such as a RAM <NUM> and a ROM <NUM>, an input device <NUM>, an output device <NUM>, an auxiliary storage device <NUM>, and the like. Functions of the conversion device <NUM> are implemented by that the CPU <NUM> reads predetermined computer software into the hardware such as RAM <NUM> and ROM <NUM> to cause the input device <NUM> and output device <NUM> to operate under control of the CPU <NUM>, and to read and write data from and into the main memory or the auxiliary storage device <NUM>. Note that the training device <NUM> and the conversion device <NUM> are not necessarily accommodated as the hardware in one housing, and may be separated into some devices.

First, a description is given of a functional configuration of the training device <NUM>. The training device <NUM> collects and is trained with the image data. The training device <NUM>, which is connected with a database <NUM> storing the image data, a camera <NUM> generating the image data, a website <NUM> from which the image data can be downloaded, and the like, can acquire the image data that is the input data for training. Of course, the training device <NUM> may be connected to an external storage medium to acquire the image data, or receive the image data via communication, and is not limited to the aspect of image data acquisition.

The training device <NUM> includes a platform for establishing a neural network through machine learning. The platform is an operating environment for machine learning of a training model base on training data. The training model is a network structure, weight data, and the like, for example, and training data is image data, for example. The platform includes framework that is a software group performing the machine learning. For the platform and the framework, there are a plurality of formats for a training result depending on a difference in a language used, a difference in a design concept, or the like.

As an example, the training device <NUM> includes a first platform <NUM> and a second platform <NUM>. The first platform <NUM> and the second platform <NUM> are platforms different from each other. Specifically, the training device <NUM> outputs the trained models different from each other in the format as the training results. The training result includes the network structure and the weight data.

The conversion device <NUM> converts the training results different in the format acquired from the training device <NUM> into those having a uniform format, and provides the converted results to the terminal device <NUM>. This allows the terminal device <NUM> to use the training result trained by any platform.

The conversion device <NUM> includes a conversion unit <NUM>. The conversion unit <NUM> acquires the network structure and the weight data from the training device <NUM> (an example of an external device) and converts the acquired network structure and weight data into a training result M1 having a uniform format (an example of a predetermined format).

The conversion device <NUM> not only converts the format of the training result but also performs a predetermined process on the training result M1 such that an operation of the recognition unit <NUM> in the terminal device <NUM> is efficient. Specifically, the conversion device <NUM> is configured to be able to execute acquiring, from the terminal device <NUM> (an example of the processing device) in which calculation for processing the input data is performed using the network structure and the weight data, a calculation cost in a case of calculating, for each of respective layers of the network structure, in at least one of calculation schemes prepared in advance for the respective layers of the network structure.

For such a process, the conversion device <NUM> includes a data acquisition unit <NUM>. The data acquisition unit <NUM> in the conversion device <NUM> acquires, for respective layers of the network structure, the calculation cost in the case of calculating in the calculation scheme prepared in advance for each of the respective layers of the network structure, from the terminal device <NUM>. Acquiring the calculation cost for each of the respective layers of the network structure means acquiring the calculation cost for each of the respective layers of the neural network described above, and as an example, the data acquisition unit <NUM> acquires the calculation cost for each intermediate layer <NUM>. As a specific example, the data acquisition unit <NUM> acquires a calculation cost, from a result of actually performing the process in calculation schemes prepared for the respective layers, such as a calculation cost for each of a plurality of algorithms prepared for a first convolutional layer, a calculation cost for each of a plurality of algorithms prepared for a first fully connected layer, and a calculation cost of one algorithm prepared for a second convolutional layer. As a more specific example, the data acquisition unit <NUM> acquires a results list TD2 in which a layer, a calculation scheme, and the calculation cost are associated with each other from the terminal device <NUM>. Hereinafter, the results list TD2 is also referred to as a profile.

The conversion device <NUM> includes a link unit <NUM> which links the results list TD2 acquired by the data acquisition unit <NUM> with the training result M1. As an example, the link unit <NUM> stores information of the selected algorithm in an attribute data area of data of the training result. By doing so, the link unit <NUM> generates a training result M2 associated with the results list TD2. The training result M2 is data output to the terminal device <NUM>.

The terminal device <NUM> includes not only the recognition unit <NUM> but also a candidate decision unit <NUM>, a pre-calculation unit <NUM>, and a cost acquisition unit <NUM> so that the data acquisition unit <NUM> described above can acquire the results list TD2.

The pre-calculation unit <NUM> acquires the training result M1 from the conversion device <NUM>. Then, the pre-calculation unit <NUM> outputs information of layer included in the training result M1 to the candidate decision unit <NUM>.

The candidate decision unit <NUM> prepares at least one calculation scheme for each of the respective layers of the network structure based on the execution environment. As an example, the candidate decision unit <NUM> decides a calculation scheme TD1 executable for each layer based on the information of layer included in the training result M1, referring to a hardware configuration profile, a software library and the like in the execution environment for the terminal device <NUM>. This allows the calculation scheme to be prepared for each layer before the performance of the pre-calculation unit <NUM>.

The calculation scheme TD1 includes a plurality of algorithms which are executable in the execution environment and exert the same function using arithmetic operations different from each other, for example. Examples of the calculation scheme TD1 include an algorithm in which an order of a loop hierarchy is different, an algorithm using a CPU extension function, and an algorithm not using the CPU extension function. In the case of the algorithms exerting the same function using arithmetic operations different from each other, the calculation scheme TD1 may use the same resource or different resources. The calculation scheme TD1 may include a plurality of algorithms which are executable in the execution environment and performs the same arithmetic operation using resources different from each other, for example. Examples of the calculation scheme TD1 include the same algorithms different in parameters (such as a thread to be used, a cache size to be used), an algorithm performing an arithmetic operation using a CPU and an algorithm performing an arithmetic operation using a GPU, and an algorithm using a hardware acceleration and an algorithm not using a hardware acceleration.

The pre-calculation unit <NUM> performs the calculation for each of the respective layers of the network structure on predetermined data using the calculation scheme TD1 prepared in advance for each of the respective layers of the network structure, in the terminal device <NUM>. The predetermined data may be any data so long as it is data which is processable in the neural network and predefined, and may be test data prepared in advance. The pre-calculation unit <NUM> exhaustively performs the calculation scheme TD1. For example, in a case where three calculation schemes are prepared for a convolutional layer, the pre-calculation unit <NUM> performs the calculation of the convolutional layer in each of the calculation schemes one or more times.

The cost acquisition unit <NUM> acquires the calculation cost of at least one calculation scheme for each of the respective layers of the network structure based on a result of the calculation by the pre-calculation unit <NUM>. The cost acquisition unit <NUM> generates the results list TD2 in which a layer, a calculation scheme, a calculation cost are associated with each other. As an example, the cost acquisition unit <NUM> generates the results list TD2 in which a layer, a calculation scheme, and an arithmetic operation speed. The results list TD2 is output to the conversion device <NUM> and acquired by the data acquisition unit <NUM> as described above.

The terminal device <NUM> further includes an acquisition unit <NUM> and a decision unit <NUM>. The acquisition unit <NUM> acquires the training result M2 associated with the results list TD2 from the conversion device <NUM>.

The decision unit <NUM> selects one calculation scheme for each of the respective layers of the network structure based on the calculation cost from among the calculation schemes prepared in advance for the respective layers based on the training result M2 to associate the layer of network structure with the selected one calculation scheme. The decision unit <NUM> selects the calculation scheme of which the calculation cost is the lowest, for example. As a specific example, the decision unit <NUM> selects an algorithm of which a calculation cost is the lowest from among a plurality of algorithms prepared for the first convolutional layer. Then, the decision unit <NUM> associates the first convolutional layer with the selected algorithm. The term "associate" means that a specific algorithm is made to be able to be called for calculating a specific layer.

The recognition unit <NUM> refers to the training result M2, and performs calculation for each of the respective layers of the network structure on the input data in the execution environment using the calculation scheme associated with each layer of the network structure by the decision unit <NUM>. This improves recognition process efficiency.

The conversion device <NUM> may change a data array (data order) of the training result. The training result is data in which the weight data is very large volume. For this reason, a data array of the weight data may be changed to be easy to refer to by the terminal device <NUM>. The conversion device <NUM> may include a first adjustment unit <NUM> for such a process.

The first adjustment unit <NUM> may change a data array of the weight data for the training result M1, or may change a data array of the weight data for the training result M2. Hereinafter, a description is given as an example of a case where the first adjustment unit <NUM> changes the data array of the weight data for the training result M1.

The first adjustment unit <NUM> changes the data array of the weight data for the training result M1 based on the execution environment for the terminal device <NUM>. For example, the first adjustment unit <NUM> changes the data array of the weight data such that the data becomes a state of being easy to be placed on a cache depending on a cache size of the terminal device <NUM> or the like. This allows the pre-calculation unit <NUM> and the recognition unit <NUM> to perform calculation based on the weight data adjusted by the first adjustment unit.

Alternatively, the conversion device <NUM> may perform a preprocess related to the calculation of the recognition unit <NUM> on the weight data. The conversion device <NUM> may include a second adjustment unit <NUM> for such a process. The recognition unit <NUM>, in a case of a process on the convolutional layer, for example, performs frequency conversion on the weight data, and then, performs the arithmetic operation. For this reason, the second adjustment unit <NUM> perform a preprocess of the frequency conversion on the weight data used for the convolutional layer. This eliminates the need for the frequency conversion on the terminal device <NUM> side, and therefore, the recognition process efficiency improves. Alternatively, in a case where compiling OpenCL (Open Computing Language) (registered trademark) or the like is needed in the calculation of the recognition unit <NUM>, the second adjustment unit <NUM> may perform a preprocess of offline compiling.

The link unit <NUM> may encrypt the training result M2 for security improvement.

Hereinafter, a description is given of operations of the conversion device <NUM> and the terminal device <NUM>.

<FIG> is a flowchart of a converting process. The flowchart shown in <FIG> is performed by the conversion unit <NUM> in the conversion device <NUM>.

As shown in <FIG>, the conversion unit <NUM> in the conversion device <NUM> acquires a training model (network structure and weight data) from the training device <NUM>, as a model acquiring process (S10). Subsequently, the conversion unit <NUM> converts the training model acquired in the model acquiring process (S10) into a uniform format, as a format converting process (S12). Subsequently, the conversion unit <NUM> outputs the training result M1 in the uniform format, as a model outputting process (S14). When the model outputting process (S14) is completed, the flowchart in <FIG> ends.

The flowchart shown in <FIG> is performed so that the training model is acquired and converted into the uniform format.

<FIG> is a flowchart of a profile acquiring process. The flowchart shown in <FIG> is performed by the terminal device <NUM>.

As shown in <FIG>, the pre-calculation unit <NUM> in the terminal device <NUM> acquires the training result M1, as a training result acquiring process (S20). Subsequently, the pre-calculation unit <NUM> acquires the calculation scheme TD1 prepared in advance, as a calculation scheme acquiring process (S22). The pre-calculation unit <NUM> refers to the calculation scheme TD1 stored by the candidate decision unit <NUM>. Subsequently, the pre-calculation unit <NUM> inputs the test data to the training result M1 and exhaustively performs the prepared calculation schemes, as a pre-calculating process (S24), At this time, a processing speed or usage resource for each calculation scheme is measured as an arithmetic operation cost. The cost acquisition unit <NUM> in the terminal device <NUM> generates the results list TD2 in which a layer, a calculation scheme, and an arithmetic operation are associated with each other based on a time and the usage resource required for calculation by the pre-calculation unit <NUM>, as a cost acquiring process (S26). When the cost acquiring process (S26) is completed, the flowchart in <FIG> ends. Note that an execution order of the training result acquiring process (S20) and the calculation scheme acquiring process (S22) may be reversed. The pre-calculating process S24 and the cost acquiring process may be alternately performed every layer.

The flowchart shown in <FIG> is performed so that an operation result of the operating environment can be acquired.

<FIG> is a flowchart of an information giving process. The flowchart shown in <FIG> is performed by the conversion device <NUM>.

As shown in <FIG>, the data acquisition unit <NUM> in the conversion device <NUM> acquires the results list TD2 from the terminal device <NUM>, as a results list acquiring process (S30). Subsequently, the link unit <NUM> links the training result M1 with the results list TD2 (profile) to generate the training result M2, as a linking process (S34). When the linking process (S34) is completed, the flowchart shown in <FIG> ends.

The flowchart shown in <FIG> is performed so that the training result M2 to which a result of the calculation cost is given is generated.

<FIG> is a flowchart of a calculating process. The flowchart shown in <FIG> is performed by the terminal device <NUM>.

As shown in <FIG>, the terminal device <NUM> secures a memory required for the process, as a first initializing process (S40). Subsequently, the recognition unit <NUM> in the terminal device <NUM> reads the training result M2 acquired from the conversion device <NUM>, as a training result reading process (S42). Similarly, the recognition unit <NUM> in the terminal device <NUM> reads the results list TD2 given to the training result M2 acquired from the conversion device <NUM>, as a profile reading process (S44). Subsequently, the terminal device <NUM> secures a memory required for the process, as a second initializing process (S46). Subsequently, the recognition unit <NUM> inputs data to be identified, as an input data reading process (S48). Subsequently, the recognition unit <NUM> performs a calculating process for recognition in each layer, as a calculating process (S50). At this time, the results list TD2 read in the profile reading process (S44) is used. The calculating process (S50) is described later in detail (<FIG>). Finally, the recognition unit <NUM> outputs a result of recognition, as a result outputting process (S52), When the result outputting process (S52) is completed, the flowchart shown in <FIG> ends.

<FIG> is a flowchart of a calculating process and illustrates the calculating process (S50) in <FIG> in detail. As shown in <FIG>, the recognition unit <NUM> inputs all pixel values corresponding to one image to the input layer, as an inputting process (S60). Subsequently, the recognition unit <NUM> inputs information of a layer to be processed, as a layer information reading process (S62). In the information of layer, included is information identifying which layer of the neural network is the layer to be processed. Subsequently, the recognition unit <NUM> branches the process based on the identification information acquired in the layer information reading process (S62), as a layer determining process (S64).

In a case where the layer to be processed is a convolutional layer, the decision unit <NUM> refers to the results list TD2 and specifies a calculation scheme of which a calculation cost is the smallest in a convolutional layer process, as an association determining process (S66). In a case where the calculation cost is the smallest in a convolutional layer process A, the decision unit <NUM> associates the convolutional layer process A with the layer to be processed. Then, the recognition unit <NUM> performs the convolutional layer process A, as a performing process (S68). On the other hand, in a case where the calculation cost is the smallest in a convolutional layer process B, the decision unit <NUM> associates the convolutional layer process B with the layer to be processed. The convolutional layer process B is a process in which the function is the same as the above but implementation is different. Then, the recognition unit <NUM> performs the convolutional layer process B, as a performing process (S70). In the above example, two cases that the calculation scheme is the convolutional layer process A and that the calculation scheme is the convolutional layer process B are described, but the number of the processes may be three or more. In this way, the decision unit <NUM> branches the process depending on the results list TD2.

On the other hand, in a case where the layer to be processed is a fully connected layer, the decision unit <NUM> refers to the results list TD2 and specifies a calculation scheme of which a calculation cost is the smallest in a fully connected layer process, as an association determining process (S72). In a case where the calculation cost is the smallest in a fully connected layer process A, the decision unit <NUM> associates the fully connected layer process A with the layer to be processed. Then, the recognition unit <NUM> performs the fully connected layer process A, as a performing process (S74). On the other hand, in a case where the calculation cost is the smallest in a fully connected layer process B, the decision unit <NUM> associates the fully connected layer process B with the layer to be processed. The fully connected layer process B is a process in which the function is the same as the above but implementation is different. Then, the recognition unit <NUM> performs the fully connected layer process B, as a performing process (S76). In the above example, two cases that the calculation scheme is the fully connected layer process A and that the calculation scheme is the fully connected layer process B are described, but the number of the processes may be three or more. In this way, the decision unit <NUM> branches the process depending on the results list TD2.

In a case where the layer to be processed is a further another layer, for example, a pooling layer, an activation function layer, or the like, the same processes as for the convolutional layer and the fully connected layer are performed.

When the process corresponding to each layer is completed, the recognition unit <NUM> determines whether or not the process is completed for all layers of the neural network, as a determining process (S80). In a case where the process is determined to be not completed for all layers, the process goes to the layer information reading process (S62) to continue the process on the next layer. In a case that the process is determined to be completed for all layers, the flowchart shown in <FIG> ends.

Next, a description is given of a calculation scheme decision program causing to function as the calculation scheme decision system <NUM>. The calculation scheme decision program includes a main module, a pre-calculation module, a cost acquisition module, a decision module, and a calculation module. The main module is a part for collectively controlling the device. The functions realized by performing the pre-calculation module, the cost acquisition module, the decision module, and the calculation module are respectively similar to the functions of the above described pre-calculation unit <NUM>, cost acquisition unit <NUM>, decision unit <NUM>, and recognition unit <NUM>.

The program is provided by way of a non-transitory record such as the ROM or a semiconductor memory medium, for example. The program may be provided through a communication of a network or the like.

In the calculation scheme decision system <NUM>, one calculation scheme is selected for each layer, from among the calculation schemes prepared in advance, based on the calculation cost obtained by actually operating those calculation schemes in the execution environment. In this way, the calculation scheme decision system <NUM> can take into account the calculation cost to decide an optimal calculation scheme for each layer. Therefore, the calculation scheme decision system <NUM> can take into account the calculation cost to decide the calculation scheme.

The calculation scheme decision system <NUM> can actually operate, in the execution environment, a plurality of algorithms which output the same results but are different from each other in the arithmetic operation method, and decide an optimal algorithm based on the obtained calculation costs.

The calculation scheme decision system <NUM>, because of including the candidate decision unit <NUM>, can prepare the calculation scheme taking into account the resources in the execution environment.

The calculation scheme decision system <NUM>, because of including the conversion unit <NUM>, can deal with the training result of the training device <NUM> in a predetermined format.

The calculation scheme decision system <NUM>, because of including the first adjustment unit <NUM>, can change the data array of the weight data such that the data becomes a state of being easy to be placed on a cache.

The calculation scheme decision system <NUM>, because of including the second adjustment unit <NUM>, can reduce a calculation load on the recognition unit <NUM>.

A calculation scheme decision system 100A according to a second embodiment is different in that the function of the decision unit <NUM> is on the conversion device <NUM> side, and the same in other aspects, as compared with the calculation scheme decision system <NUM> according to the first embodiment. In the second embodiment, the description the same as in the first embodiment is not repeated.

<FIG> is a functional block diagram of a calculation scheme decision system 100A according to the second embodiment. As shown in <FIG>, the calculation scheme decision system 100A includes a conversion device 40A (an example of the calculation scheme decision device) and a terminal device 10A.

The conversion device 40A is different in including a decision unit 17A as compared with the conversion device <NUM>. The decision unit 17A is different in an input source and an output destination and the same as in other configurations as compared with the decision unit <NUM>. In other words, the decision unit 17A selects one calculation scheme for each of the respective layers of the network structure based on the calculation cost from among the calculation schemes prepared in advance for the respective layers based on the results list TD2 to associate the layer of network structure with the selected one calculation scheme.

A link unit 43A gives the correspondence relation associated by the decision unit 17A to the training result M1. In other words, the link unit 43A does not give the results list TD2 as it is to training result M1, but gives the result derived from the results list TD2 to the training result M1. By doing so, a training result M3 is generated.

An acquisition unit 16A is different in acquiring the training result M3 and the same as in other configurations as compared with the acquisition unit <NUM>. In other words, the acquisition unit 16A acquires the network structure, the weight data, and a correspondence relation between the layer of the network structure and the calculation scheme. This allows the recognition unit <NUM> to use the correspondence relation associated by the decision unit 17A to perform the calculation for each of the respective layers of the network structure on the input data.

Other configurations of the calculation scheme decision system 100A are the same as those of the calculation scheme decision system <NUM>.

An operation of the calculation scheme decision system 100A is different in the information giving process (<FIG>) and the calculating process (<FIG> and <FIG>), and the same as in other operations as compared with the operation of the calculation scheme decision system <NUM>.

<FIG> is a flowchart of an information giving process. The flowchart shown in <FIG> is performed by the conversion device 40A.

As shown in <FIG>, the data acquisition unit <NUM> in the conversion device 40A acquires the results list TD2 from the terminal device <NUM>, as a results list acquiring process (S300). Subsequently, the decision unit 17A in the conversion device <NUM> selects one calculation scheme for each of the respective layers of the network structure based on the calculation cost acquired by the data acquisition unit <NUM>, from among the calculation schemes prepared in advance for the respective layers, as a deciding process (S320). Subsequently, the link unit <NUM> links the training result M1 with one calculation scheme selected for each layer to generate the training result M3, as a linking process (S340). When the linking process (S340) is completed, the flowchart shown in <FIG> ends.

The flowchart shown in <FIG> is performed so that the training result M3 to which a result of the calculation cost is given is generate.

<FIG> is a flowchart of a calculating process. The flowchart shown in <FIG> is performed by the terminal device 10A.

As shown in <FIG>, the terminal device 10A secures a memory required for the process, as a first initializing process (S400). Subsequently, the recognition unit <NUM> in the terminal device 10A reads the training result M3 acquired from the conversion device 40A, as a training result reading process (S420). Subsequently, the terminal device 10A secures a memory required for the process, as a second initializing process (S460). Subsequently, the recognition unit <NUM> inputs data to be identified, as an input data reading process (S480). Subsequently, the recognition unit <NUM> performs a calculating process for recognition in each layer, as a calculating process (S500). The calculating process (S500) is described later in detail (<FIG>). Finally, the recognition unit <NUM> outputs a result of recognition, as a result outputting process (S520), When the result outputting process (S520) is completed, the flowchart shown in <FIG> ends.

As shown in <FIG>, the recognition unit <NUM> inputs all pixel values corresponding to one image to the input layer, as an inputting process (S600). Subsequently, the recognition unit <NUM> inputs information of a layer to be processed, as a layer information reading process (S620). In the information of layer, included is information identifying which layer of the neural network is the layer to be processed. Subsequently, the recognition unit <NUM> reads the correspondence relation given to the training result M3 to specify the calculation scheme associated with the layer, as a calculation scheme specifying process (S622). Subsequently, the recognition unit <NUM> performs the calculating process for recognition in the calculation scheme specified in the correspondence relation, as a calculating process (S624).

When the process corresponding to each layer is completed, the recognition unit <NUM> determines whether or not the process is completed for all layers of the neural network, as a determining process (S800). In a case where the process is determined to be not completed for all layers, the process goes to the layer information reading process (S620) to continue the process on the next layer. In a case that the process is determined to be completed for all layers, the flowchart shown in <FIG> ends.

A processing program and a calculation scheme decision program causing to function as the terminal device 10A and the conversion device 40A include modules corresponding to function units of the devices similar to the program described in the first embodiment.

In the calculation scheme decision system 100A, one calculation scheme is selected for each layer, from among the calculation schemes prepared in advance, based on the calculation cost obtained by actually operating those calculation schemes in the execution environment. In this way, the calculation scheme decision system 100A can take into account the calculation cost to decide an optimal calculation scheme for each layer. Therefore, the calculation scheme decision system 100A can take into account the calculation cost to decide the calculation scheme.

Note that the present disclosure is not limited to the above embodiments.

The calculation scheme decision system <NUM> is illustrated to include the terminal device <NUM> and the conversion device <NUM> as the hardware, but is not limited thereto. For example, the calculation scheme decision system <NUM> may be configured as a collection of devices prepared for respective functions connected to each other through a communication network, such as a device for the pre-calculation unit <NUM>, a device for the cost acquisition unit <NUM>, a device for the decision unit <NUM>, and a device for the recognition unit <NUM>. Alternatively, the calculation scheme decision system <NUM> may be constituted by single hardware capable of exerting the all functions. The calculation scheme decision system 100A may similarly be constituted by various hardware or single hardware.

Since the calculation scheme decision system <NUM> decides an optimal calculation scheme for each layer for the recognition unit <NUM>, one set of optimal calculation schemes is provided to one recognition unit <NUM>. However, the number of sets of optimal calculation schemes provided is not limited to one, and two or more sets may be provided. In this case, the recognition unit <NUM> may perform each of a plurality of sets provided to select a set in which the process can be performed the fastest, for example.

The calculation scheme decision system <NUM> may develop an optimal calculation scheme for another terminal having the same environment. For example, assume that there is a terminal device 10X (an example of a second terminal) having an environment the same as the terminal device <NUM> (an example of a first terminal). The terminal device 10X is a model the same as the terminal device <NUM>, for example. The calculation scheme decision system <NUM> may further include a provision unit providing the calculation scheme decided by the decision unit <NUM> to the terminal device 10X. In this case, the calculation scheme decision system <NUM> can, without calculating in a terminal in the same environment, apply the calculation scheme decided in another terminal having the same environment.

Claim 1:
A calculation scheme decision system comprising:
a pre-calculation unit configured to perform, in an execution environment in which calculation for processing input data is performed using a network structure and weight data, calculation for each of respective layers of the network structure on predetermined data using at least one of calculation schemes prepared in advance for the respective layers of the network structure;
a cost acquisition unit configured to acquire a calculation cost of the at least one calculation scheme for each of the respective layers of the network structure based on a result of the calculation by the pre-calculation unit;
a decision unit configured to select one calculation scheme for each of the respective layers of the network structure based on the calculation cost from among the at least one of the calculation schemes prepared in advance for the respective layers to associate the layer of the network structure with the selected one calculation scheme; and
a calculation unit configured to perform the calculation for each of the respective layers of the network structure on the input data in the execution environment using the calculation scheme associated with each layer of the network structure by the decision unit.