Patent Description:
In construction of a classifier performing classification of a target indicated in data, use of a variable well representing a feature of the target leads to improvement of precision of classification.

As a method of deriving a variable well representing a feature of a target from input data, a method of using an autoencoder is well known.

A typical autoencoder includes an input layer, an intermediate layer, and an output layer. The typical autoencoder determines optimum values of a weight and a bias used for encoding (that is, conversion from data in the input layer to data in the intermediate layer), and a weight and a bias used for decoding (that is, conversion from data in the intermediate layer to data in the output layer), based on a comparison between data input to the input layer and data output from the output layer.

Data output in the intermediate layer by encoding using the weight and the bias determined as a result of learning by the autoencoder can be considered information well representing a feature of a target. The data output in the intermediate layer are generally referred to as a "feature value vector," a "feature vector," or simply a "feature value" or a "feature. " The data output in the intermediate layer are herein also referred to as a "set of latent variable values" or a "latent variable vector.

PTL <NUM> is a literature describing a technology related to the present invention. PTL <NUM> discloses an image processing device converting (in another word, normalizing) a size, a rotation angle, a position, and/or the like of a target in an image to a state suitable for identification. Magnitude of the conversion for normalization is determined by a coefficient determined based on a relation between a vector (mapping vector) in a case of mapping data of a coarse-grained image onto a space F by nonlinear transformation and a subspace including a basis vector representing a feature of a learning sample. PTL <NUM> describes that when an autoencoder is used in the technology, an output of the output layer of the autoencoder corresponds to a mapping by the aforementioned nonlinear transformation, and an output of the intermediate layer corresponds to a projection of the mapping vector on the subspace.

The document <NPL>, discloses to learn an embedding space that has generative capabilities to construct 3D structures, while being predictable from RGB images. The document <CIT>, discloses automated object recognition of 3D objects using statistical shape information.

A feature value vector derived by a neural network optimized by a common autoencoder is not necessarily represented in such a way that different forms taken by the same target interrelate with each other. Then, for example, it is assumed that, by use of a feature value vector derived by a neural network optimized by a common autoencoder, a classifier for classifying a chair appearing in an image as a chair is generated by learning using only images of the chair photographed in a direction indicated by <FIG> as training data. In such a case, the generated classifier may not be able to identify a chair photographed in a direction indicated by <FIG> or from an angle indicated by <FIG> as a chair. The reason is that feature value vectors unrelated to each other may be derived from data recorded in forms (a direction and an angle in the example described above) different from each other even when the data are related to the same target.

In order to correctly identify a target object that may take various forms, use of training data completely covering the various forms in learning by a classifier is generally required. However, preparation of training data completely covering the various forms is not necessarily easy.

The technology described in PTL <NUM> is a technology of improving identification performance on a target that may take various forms, by normalizing an image. However, a function for performing the normalization needs to be derived by learning having images in which various forms of a target appear as training data. A pattern identification unit <NUM> identifying a target takes a normalized image as an identification target and therefore does not guarantee correct identification of a target taking a form not included in the training data.

An object of the present invention is to provide a learning device capable of generating an identifier capable of identifying a target in various forms even when the number of samples of data in which the target is recorded is small.

A learning device according to claim <NUM>.

A learning method according to claim <NUM>.

A storage medium according to an aspect of the present invention stores a program according to claim <NUM>.

The present invention can generate an identifier capable of identifying a target in various forms even when the number of samples of data in which the target is recorded is small.

Example embodiments of the present invention will be described in detail below referring to drawings.

First, a first example embodiment of the present invention will be described. <FIG> is a block diagram illustrating a configuration of a learning device <NUM> according to the first example embodiment.

The learning device <NUM> performs two types of learning being variable derivation learning and classification learning. A unit related to the variable derivation learning is herein referred to as a variable derivation unit <NUM>, and a unit performing the classification learning is referred to as a classification learning unit <NUM>.

First, a configuration and an operation of the variable derivation unit <NUM> will be described below.

The variable derivation unit <NUM> includes a data acquisition unit <NUM>, an encoder <NUM>, a conversion unit <NUM>, a decoder <NUM>, a parameter updating unit <NUM>, and a parameter storage unit <NUM>.

For example, the data acquisition unit <NUM>, the encoder <NUM>, the conversion unit <NUM>, the decoder <NUM>, and the parameter updating unit <NUM> are provided by one or a plurality of central processing units (CPUs) executing a program.

For example, the parameter storage unit <NUM> is a memory. The parameter storage unit <NUM> may be an auxiliary storage device such as a hard disk. According to another example embodiment, the parameter storage unit <NUM> may be configured to be external to the learning device <NUM> and be communicable with the learning device <NUM> in a wired or wireless manner. The parameter storage unit <NUM> stores parameters used in a conversion performed by the encoder <NUM> and parameters used in a conversion performed by the decoder <NUM>.

The variable derivation unit <NUM> may include a storage device transitorily or non-transitorily storing data, aside from the parameter storage unit <NUM>.

The data acquisition unit <NUM> acquires data used by the variable derivation unit <NUM>. Data used by the variable derivation unit <NUM> are input data, correct answer data, and difference information indicating a relation between the input data and the correct answer data.

Input data are data in which a target of learning by the variable derivation unit <NUM> is recorded. For ease of understanding, an optical image is assumed as an example of input data in the description of the present example embodiment. Examples of input data other than an optical image will be described in the item "Supplement.

When input data represent an optical image, the input data represent an image in which a target (for example, an object and a person) appears. For example, input data represent a vector having a pixel value of each pixel of an image as a component. When a gray-scale image having <NUM> pixels in a longitudinal direction and <NUM> pixels in a lateral direction is assumed as input data, the number of components of the input data is <NUM> × <NUM> = <NUM>.

An image may have any size. A pixel value may be an integer value ranging from <NUM> to <NUM>, a binary value being <NUM> or <NUM>, or a floating-point number. There may be one type or two or more types of colors. When there are a plurality of color types, the number of components of input data increases in proportion to the number of the types. Examples of input data include an RGB image, a multispectral image, and a hyperspectral image.

For example, the data acquisition unit <NUM> acquires input data by receiving the input data from a storage device internal or external to the learning device <NUM>. The learning device <NUM> may include a device capable of acquiring input data, such as a camera, and the data acquisition unit <NUM> may receive input data from the device.

Correct answer data are data used in the variable derivation learning and specifically in an update of parameter values by the parameter updating unit <NUM> to be described later.

Correct answer data are data in which a target indicated by input data is recorded. At least one piece of correct answer data is data in which a target indicated by input data is recorded in a form different from a form in the input data. When input data and correct answer data represent images, a form may be reworded as a "way to be photographed" or a "way to be viewed. " Examples of a form in an image include a direction, an angle, a position, a size, a degree of distortion, a hue, and clarity. A form that may differ between input data and correct answer data is predefined. In other words, the variable derivation unit <NUM> handles a set of input data and correct answer data between which at least one specific form differs. The learning device <NUM> may handle input data as one type of correct answer data.

For example, the data acquisition unit <NUM> acquires correct answer data by receiving the correct answer data from a storage device internal or external to the learning device <NUM>. The learning device <NUM> may include a device capable of acquiring correct answer data, such as a camera, and the data acquisition unit <NUM> may receive correct answer data from the device.

Alternatively, the data acquisition unit <NUM> may generate correct answer data by processing input data. For example, the data acquisition unit <NUM> may generate correct answer data by processing input data, by using a process of changing a rotation angle of a target or a known technology of changing a hue or clarity.

Difference information is information indicating a relation between input data and correct answer data. Specifically, difference information indicates the difference between a form of a target indicated by input data and a form of the target indicated by correct answer data. For example, difference information may be represented by a parameter indicating existence of a difference or a degree of difference.

As a simple example, it is assumed that input data represent an image in which a chair appears, and correct answer data represent an image of the chair photographed in a direction different from the direction in the input data. Examples of a set of input data and correct answer data include a set of an image in <FIG> and an image in <FIG>, and a set of the image in <FIG> and an image in <FIG>. An example of difference information indicating a relation between the image in <FIG> and the image in <FIG> is a value [such as "+<NUM> (degrees)"] indicating a rotation angle. An example of difference information indicating a relation between the image in <FIG> and the image in <FIG> is a value [such as "-<NUM> (degrees)"] indicating a change in an azimuth angle.

For example, when input data represent an optical image, examples of a difference indicated by difference information include a rotation angle with a direction perpendicular to a display surface of an image as an axis, a difference in an angle (a direction of a target relative to an imaging device), an amount of increase (or an amount of decrease) in brightness, a difference in contrast, a difference in a level of noise (noise originating in existence of rain, fog, or the like, or low resolution), and a difference in existence of an obstacle, an attachment, or an ornament, in comparison with input data. When a target is an object streaming in the wind, such as hair or a flag, difference information may be information indicating an intensity of the wind. A parameter closely related to the examples cited above may be employed as difference information. When input data and correct answer data are separately acquired, a form being a target indicated by employed difference information does not need to be a form a change of which can be represented by processing the input data.

Difference information may be a quantitative parameter or may be a parameter having a plurality of steps. As an example, when difference information is a parameter indicating an intensity of rain, the parameter may be represented by four types of values being "no rain," "light," "moderately heavy," and "heavy. " Difference information may be a parameter taking only two values (for example, "existence" and "nonexistence").

For example, the data acquisition unit <NUM> acquires difference information by receiving the difference information from a storage device internal or external to the learning device <NUM>. The data acquisition unit <NUM> may receive input of difference information from a person or a device grasping a relation between input data and correct answer data, and acquire the input difference information. The data acquisition unit <NUM> may acquire difference information by specifying the difference by comparison between input data and correct answer data.

The encoder <NUM> derives a set of latent variable values from input data. For example, by using a neural network, the encoder <NUM> inputs input data to the input layer of the neural network and derives n values as an output. Note that n denotes the number of units in the output layer of the neural network. The set of n values is herein referred to as a set of latent variable values or a latent variable vector. While the term "vector" is used in the present example embodiment, a latent variable vector is not limited to a one-dimensional array of a plurality of values. The number of output values may be one. Alternatively, a latent variable vector may be a two-or-more-dimensional array. A latent variable vector may be held in the learning device <NUM> in a format other than an array format. Derivation of a latent variable vector by a neural network is also referred to as encoding.

A structure of a neural network used by the encoder <NUM> may be freely designed. For example, there is no limit on the number of layers, the number of components in each layer, and a connection method between components. As an example, the encoder <NUM> may use a convolutional neural network including an input layer with the number of components being <NUM>, an intermediate layer with the number of components being <NUM>, and an output layer with the number of components being <NUM>. The number of values output by the encoder <NUM> (that is, the number of components of a latent variable vector) is typically configured to be less than the number of components in input data. However, the number of values output by the encoder <NUM> may be configured to be equal to or more than the number of components in input data.

An activation function used in a neural network used by the encoder <NUM> may be any activation function. Examples of an activation function include an identity function, a sigmoid function, a rectified linear unit (ReLU) function, and a hyperbolic tangent function.

The encoder <NUM> reads parameters (typically a weight and a bias) in a neural network to be used from the parameter storage unit <NUM> and performs encoding of input data.

The conversion unit <NUM> converts a latent variable vector output by the encoder <NUM> to another latent variable vector. Conversion of a latent variable vector by the conversion unit <NUM> is herein referred to as variable conversion.

The conversion unit <NUM> converts a latent variable vector by use of a conversion function. The conversion unit <NUM> uses different conversion functions according to the aforementioned difference information.

The conversion unit <NUM> uses a conversion function using a conversion parameter taking a value that may vary according to difference information. After determining a conversion parameter according to difference information, the conversion unit <NUM> may convert a latent variable vector by use of a conversion function using the determined conversion parameter.

The conversion function include a function changing an arrangement of components of a latent variable vector. The conversion function is a function shifting an arrangement of components of a latent variable vector. An amount of shift may be determined by a conversion parameter. A manipulation of shifting k components in an arrangement of components of a vector with the number of components being n is a manipulation of shifting the first to (n-k)-th components of the vector to the (k+<NUM>)-th to n-th components and shifting the (n-k)-th to n-th components to the first to k-th components.

The conversion function is a function shifting an arrangement of components of a latent variable vector with the number of components being <NUM>, based on a value of a conversion parameter p. It is assumed that difference information acquired by the data acquisition unit <NUM> is a rotation angle θ where θ takes a value being a multiple of <NUM> out of integers equal to or more than <NUM> and equal to or less than <NUM>. In such a case, a value acquired by dividing θ by <NUM> may be defined as the conversion parameter p. Then, p is a parameter that may take an integer value in a range from <NUM> to <NUM>. Then, the conversion function may be defined in such a way that a value twice the value of p corresponds to an amount of shift of the arrangement of the components of the latent variable vector. For example, a value of the conversion parameter p corresponding to a rotation of <NUM> degrees is <NUM> and is related to shifting <NUM> components in the arrangement of the components of the latent variable vector.

The conversion function shifting an arrangement of components of a latent variable vector may be represented as a multiplication of a conversion matrix representing a shift. When a latent variable vector is denoted as Z<NUM>, the number of components of the latent variable vector is denoted as n, a value of a conversion parameter is denoted as k, and a conversion matrix representing a shift is denoted as Sk, Sk is an n × n matrix, and the aforementioned conversion function is represented by the following equation.

The matrix Sk is a matrix illustrated in <FIG>.

Specifically, the matrix Sk is a matrix in which a numerical value of the i-th row and the (kr+i)-th column is <NUM> for i where <NUM> ≤ i ≤ n-kr, a numerical value of the (n-kr+j)-th row and the j-th column is <NUM> for j where <NUM> ≤ j ≤ kr, and every remaining numerical value is <NUM>. Note that kr is a value determined by k × n/N(k) when the number of value that may be taken by k is denoted as N(k).

By the conversion by the conversion unit <NUM>, a new latent variable vector with the number of components being n is generated.

A generation method of a function and a matrix for variable conversion is not limited to the above. For example, the conversion unit <NUM> may use a matrix generated by applying a Gaussian filter to the aforementioned matrix Sk in place of the matrix Sk.

The type of variable conversion is not limited to the shift manipulation described above. For example, variable conversion may be subtraction processing on a component value by which an amount of subtraction increases according to magnitude of a difference indicated by difference information. Variable conversion may be smoothing processing executed a number of times based on magnitude of a difference indicated by difference information. Variable conversion is an operation on a predetermined component, and details of the operation or the number of components undergoing the operation may depend on magnitude of a difference indicated by difference information.

Variable conversion performed by the conversion unit <NUM> may include identity transformation. Variable conversion in a case of difference information indicating nonexistence of a difference in particular may be identity transformation.

When there are two or more types of forms that may be different between input data and correct answer data, the conversion unit <NUM> may perform a variable conversion, based on difference information related to each form. As an example, when difference information is denoted by two parameters (α, β) each indicating a change in a three-dimensional direction, the conversion unit <NUM> may generate a new latent variable vector by applying a conversion function dependent on α to a latent variable vector and then applying a conversion function dependent on β. The conversion function dependent on α and the conversion function dependent on β may be applied in parallel. Alternatively, the conversion unit <NUM> may determine one conversion function, based on difference information about each of the differences between two or more types of forms and execute a variable conversion by use of the conversion function.

The decoder <NUM> generates output data from a latent variable vector after conversion by the conversion unit <NUM>. For example, by using a neural network (different from the neural network used by the encoder <NUM>), the decoder <NUM> inputs a latent variable vector to the input layer of the neural network and generates output data composed of m components as an output. Note that m is the number of units in the output layer of the neural network used by the decoder <NUM>. The value m is set to the same value as the number of components of correct answer data. When input data and correct answer data are data represented in the same format, m matches the number of components of the input data, that is, the number of units in the input layer of the encoder <NUM>. Generation of output data from a latent variable vector by a neural network is also referred to as decoding.

A structure of a neural network used by the decoder <NUM> may be freely designed. For example, there is no limit on the number of layers, the number of components in an intermediate layer (in a case of a multilayer neural network), and a connection method between components. As an example, the decoder <NUM> may use a neural network including an input layer with the number of components being <NUM>, an intermediate layer with the number of components being <NUM>, and an output layer with the number of components being <NUM>.

An activation function used in a neural network used by the decoder <NUM> may be any activation function. Examples of an activation function include an identity function, a sigmoid function, a ReLU function, and a hyperbolic tangent function.

The decoder <NUM> reads values of parameters (typically a weight and a bias) in a neural network to be used from the parameter storage unit <NUM> and performs decoding of a latent variable vector.

The parameter updating unit <NUM> updates parameter values of neural networks used by the encoder <NUM> and the decoder <NUM>, based on a comparison between output data generated by the decoder <NUM> and correct answer data acquired by the data acquisition unit <NUM>.

A specific example of a parameter value updating procedure will be described. First, for each of one or more sets of correct answer data and output data, the parameter updating unit <NUM> calculates an error of the output data with respect to the correct answer data. For example, the parameter updating unit <NUM> may use a mean square error as an error function for determining an error. Then, the parameter updating unit <NUM> determines new parameter values in such a way as to reduce the calculated error. A method known as a parameter value optimization method employed in a common autoencoder may be used as a technique for determining new parameter values. As an example, the parameter updating unit <NUM> may calculate a gradient by use of error back propagation and determine parameter values by use of stochastic gradient decent (SGD). Other employable techniques include "RMSprop," "Adagrad," "Adadelta," and "Adam.

Then, the parameter updating unit <NUM> records the determined new parameter values into the parameter storage unit <NUM>. The encoder <NUM> and the decoder <NUM> thereafter use the new parameter values. The above concludes the specific updating procedure.

Target parameter values to be updated by the parameter updating unit <NUM> are a weight and a bias of a neural network used by the encoder <NUM>, and a weight and a bias of a neural network used by the decoder <NUM>. A conversion parameter used in a variable conversion is not included in the target parameters to be updated by the parameter updating unit <NUM>.

The parameter updating unit <NUM> may repeatedly update parameter values a predetermined number of times. For example, the predetermined number of times may be determined as a value received as an input of a numerical value indicating the predetermined number of times from a user of the learning device <NUM> through an input interface.

An error function used by the parameter updating unit <NUM> for determining an error may be freely designed. The parameter updating unit <NUM> may use an error function considering values of an average and a variance of a latent variable vector, such as an error function used in a variational autoencoder (VAE).

An outline of processing related to the variable derivation learning by the variable derivation unit <NUM> will be described referring to <FIG>.

First, a latent variable vector having n components (z<NUM>, z<NUM>,. , zn) are derived from input data having m data values (x<NUM>, x<NUM>,. , xm) as components by a neural network of the encoder <NUM>. The latent variable vector is converted to another latent variable vector having n components (z'<NUM>, z'<NUM>,. , z'n) by a variable conversion by the conversion unit <NUM>. Output data having m components (y'<NUM>, y'<NUM>,. , y'm) are generated from the another latent variable vector by a neural network of the decoder <NUM>.

A set of the thus generated output data and correct answer data having m components (y<NUM>, y<NUM>,. , ym) and being in such a relation with the input data that forms of the target are different is used for learning as a training data set.

A processing flow related to the variable derivation learning by the variable derivation unit <NUM> will be described referring to a flowchart in <FIG>. When each type of processing included in the processing related to the variable derivation learning is executed by a device executing a program, the each type of processing may be executed according to the order of instructions in the program. When each type of processing is executed by a separate device, processing may be executed by a device completing the previous processing giving notification to a device executing the processing. For example, each unit performing processing records data generated by each type of processing into a storage area included in the learning device <NUM> or an external storage device. Each unit performing processing may receive data required for each type of processing from a unit generating the data or read the data from the aforementioned storage area included in the learning device <NUM> or the aforementioned external storage device.

First, the data acquisition unit <NUM> acquires input data, correct answer data, and difference information (Step S11). Timings at which the pieces of data are acquired may not be the same. A timing at which data are acquired may be any time before processing in a step in which the data are used is performed.

Next, the encoder <NUM> converts the input data to a latent variable vector (Step S12).

Next, the conversion unit <NUM> converts the latent variable vector by use of conversion parameter values based on a difference indicated by the difference information (Step S13).

Next, the decoder <NUM> converts the converted latent variable vector to output data (Step S14).

Next, the parameter updating unit <NUM> determines whether to end updating of parameter values used in the encoder <NUM> and the decoder <NUM>.

For example, a case of ending updating is a case of the number of times the parameter updating unit <NUM> updates the parameter values reaching a predetermined number of times.

As another example, a case of ending updating may be a case of an error of the output data with respect to the correct answer data being sufficiently small. For example, the parameter updating unit <NUM> may determine that the error is sufficiently small in the following cases and determine to end updating.

Alternatively, the parameter updating unit <NUM> may determine to end updating when an average value or a maximum value of an absolute amount of change in each parameter value (that is, an absolute value of an amount of change in a parameter value when updating is performed) or an average value or a maximum value of a rate of change (that is, a ratio of the absolute amount of change to the current value) falls below a predetermined reference value.

When not ending updating (NO in Step S16), the parameter updating unit <NUM> updates the parameter values (Step S17), and the variable derivation unit <NUM> performs the processing in Steps S12 to Step S14 again. In the processing in Step S12 and Step S14 from the second time onward, the encoder <NUM> and the decoder <NUM> perform the processing by use of the updated parameter values. The parameter updating unit <NUM> compares output data newly generated by the processing in Step S14 with the correct answer data again (Step S15) and determines whether to end updating of the parameter values. Thus, the variable derivation unit <NUM> repeats updating of the parameter values and generation of output data using the updated parameter values until updating of the parameters is determined to be ended. Processing of updating the parameter values through such repetition is the variable derivation learning. The parameter updating unit <NUM> updates parameter values by, in a sense, learning with a set of output data and correct answer data as a training data set. Making parameter values more suitable values by repeatedly performing updates is also referred to as optimization.

When updating of parameter values is determined to be ended (YES in Step S16), the processing of the variable derivation learning ends.

For the same target, the variable derivation unit <NUM> can derive interrelated latent variable vectors respectively representing features of different forms of the target.

Based on the aforementioned specific example, an example of an effect provided by the variable derivation unit <NUM> is as follows.

The encoder <NUM>, the conversion unit <NUM>, and the decoder <NUM> in the variable derivation unit <NUM> after completion of learning can generate a plurality of images representing different forms of a target, according to a conversion parameter. Accordingly, even when a form of the target in an image changes, a latent variable vector output by the encoder <NUM> can represent the change by a conversion. In other words, a combination of the encoder <NUM> and the conversion unit <NUM> can generate interrelated latent variable vectors respectively representing features of the different forms of the target.

When a difference between forms is a difference that may be represented quantitatively, a set of the conversion unit <NUM> and the decoder <NUM> may generate data in which a form not included in correct answer data is recorded. For example, it is assumed in the variable derivation learning that data in which a target in a certain form (denoted as a "form SA") is recorded and data in which the target in another form (denoted as a "form SC") is recorded are respectively used as correct answer data. The conversion unit <NUM> can generate a latent variable vector representing the target in a form (denoted as a "form SB") corresponding to a form between the form SA and the form SC from a latent variable vector representing the target in the form SA, by a variable conversion using a half value of a value of a conversion parameter corresponding to a change from the form SA to the form SC. By generating output data from the latent variable vector by the decoder <NUM>, output data in which the target in the form SB is recorded may be generated.

Even when a difference between forms is a difference not represented quantitatively, the set of the conversion unit <NUM> and the decoder <NUM> may generate data in which a form not included in correct answer data is recorded. For example, it is assumed in the variable derivation learning that data in which a certain target (denoted as a "target TA") in the form SA is recorded, data in which the target TA in the form SB is recorded, and data in which another target (denoted as a "target TB") in the form SA is recorded are respectively used as correct answer data. By the learning, the set of the conversion unit <NUM> and the decoder <NUM> can generate data in which the target TA in the form SA is recorded and data in which the target TA in the form SB is recorded from a latent variable vector. Accordingly, the conversion unit <NUM> is considered to be able to derive a latent variable vector representing the target TB in the form SB by converting the latent variable vector representing the target TB in the form SA. Then, it is expected that, by decoding, the converted latent variable vector can generate data in which the target TB in the form SB is recorded.

When a difference between forms is a difference that may be represented quantitatively, the encoder <NUM> may be able to derive a latent variable vector representing a target in a form not included in input data. For example, it is assumed in the variable derivation learning that data in which a target in the form SA is recorded and data in which the target in the form SC is recorded are respectively used as input data. When data in which the target in the form SB corresponding to a form between the form SA and the form SC is recorded are input to the encoder <NUM> after optimization of parameter values, a derived latent variable vector may be similar to (or match) a latent variable vector that can be generated from a latent variable vector representing the target in the form SA by performing a variable conversion. In other words, from the target in a form not used in the learning, the encoder <NUM> may be able to derive a latent variable vector that can be converted to a latent variable vector representing a form other than the form.

Even when a difference between forms is a difference not represented quantitatively, the encoder <NUM> may derive a latent variable vector representing the target in a form not included in input data. For example, it is assumed in the variable derivation learning that data in which the target TA in the form SA is recorded, data in which the target TA in the form SB is recorded, and data in which the target TB in the form SA is recorded are respectively used as input data. By the learning, the encoder <NUM> can derive a latent variable vector representing the target TA in the form SB. Accordingly, the encoder <NUM> is considered to be also able to derive a latent variable vector representing the target TB in the form SB from data in which the target TB in the form SB is recorded. Then, it is expected that, by a variable conversion, a latent variable vector representing the target TB in the form SA can be converted from the derived latent variable vector.

As described above, by the variable derivation learning, the encoder <NUM> may be able to derive interconvertible latent variable vectors by a conversion using a conversion parameter for the same target in a different form.

The learning device <NUM> may handle any type of data, any target, and any difference in forms, as long as two or more pieces of data in which forms of a target are different and information (difference information) indicating the differences between the data can be acquired.

Input data are not limited to an optical image.

An example of input data is SAR data. SAR data are sensing data acquired by a synthetic aperture radar (SAR). Examples of a target recorded by SAR data include a topography, a structure, a vehicle, an aircraft, and a ship. Examples of a changeable form include an azimuth angle and a depression angle when SAR data are acquired. In other words, a difference resulting from a condition when sensing is performed by a SAR may be employed as a difference handled by the learning device <NUM>.

For example, input data may be sound data. Sound data are data in which a sound is recorded. When input data are sound data, the input data may be represented specifically by an amplitude per unit time, an intensity of a spectrogram per time window, or the like.

When input data are sound data, examples of a target include a human voice, a speech content, an acoustic event, and music. An acoustic event refers to a sound indicating occurrence of some event, such as a scream or a glass shattering sound. When input data are sound data, examples of a variable form include a frequency (a pitch of a sound), a recording place, an echo level, a tone, a reproduction speed (a tempo) of data, a noise level, a type of object generating a sound, and a person generating a sound or an emotional state of the person.

A configuration and an operation of the classification learning unit <NUM> will be described.

Referring to <FIG>, the classification learning unit <NUM> includes a data acquisition unit <NUM>, a conversion unit <NUM>, a classification unit <NUM>, a parameter updating unit <NUM>, an output unit <NUM>, and a parameter storage unit <NUM>.

For example, the data acquisition unit <NUM>, the conversion unit <NUM>, the classification unit <NUM>, the parameter updating unit <NUM>, and the output unit <NUM> are provided by one or a plurality of CPUs executing a program.

For example, the parameter storage unit <NUM> is a memory. The parameter storage unit <NUM> may be an auxiliary storage device such as a hard disk. According to another example embodiment, the parameter storage unit <NUM> may be configured to be external to the learning device <NUM> and be communicable with the learning device <NUM> in a wired or wireless manner. The parameter storage unit <NUM> stores parameters used in classification performed by the classification unit <NUM>.

The learning device <NUM> may include a storage device transitorily or non-transitorily storing data, aside from the parameter storage unit <NUM>.

The data acquisition unit <NUM> acquires data used by the classification learning unit <NUM>. Data used by the classification learning unit <NUM> are a latent variable vector derived by the encoder <NUM> and correct answer information.

Correct answer information is information considered desirable as information to be output as a classification result by the classification unit <NUM> to be described later. Correct answer information is given as a set with input data. Correct answer information is information to be output when a target indicated in input data associated with the correct answer information is correctly identified.

For example, when classification performed by the classification unit <NUM> is multi-class classification identifying which of L (where L is any integer equal to or more than <NUM>) classes a target belongs to, correct answer information may be an L-dimensional vector in which a value of any one component is "<NUM>," and values of the other components are "<NUM>. " Such a vector is also referred to as one-hot data. In the one-hot data, each component is associated with a class. In other words, the one-hot data indicates that the target is classified as a class associated with the component with the value "<NUM>.

For example, when classification performed by the classification unit <NUM> is binary classification identifying whether a target is a specific object, correct answer information may be information taking a value of "<NUM>" or "<NUM>.

Correct answer information is compared with a classification result by the classification unit <NUM> in updating of parameter values by the parameter updating unit <NUM> to be described later.

The data acquisition unit <NUM> may acquire a latent variable vector derived by the encoder <NUM> by reading the vector from the latent variable storage unit <NUM>.

The conversion unit <NUM> converts a latent variable vector derived by the encoder <NUM> to another latent variable vector. The conversion unit <NUM> performs a variable conversion using a conversion function, similarly to the conversion unit <NUM>.

The conversion function used by the conversion unit <NUM> is the same type of conversion function as the conversion unit <NUM>, differing only in a value of a conversion parameter at most.

The conversion unit <NUM> may generate a plurality of separate latent variable vectors by a plurality of variable conversions using various values of the conversion parameter.

The classification unit <NUM> performs classification on a latent variable vector output by the conversion unit <NUM>.

For example, by use of a neural network, the classification unit <NUM> inputs a latent variable vector to the input layer of the neural network and generates information indicating a classification result as an output.

For example, when the classification unit <NUM> is used as a multi-class classifier, information indicating a classification result is a multidimensional vector indicating a distribution of a probability (may also be referred to as a likelihood) that a target belongs to a classification class. The number of components of the multidimensional vector in such a case is the number of classification classes. When the classification unit <NUM> is used as a binary classifier, information indicating a classification result may be a numerical value indicating a probability that a target is a predetermined recognition target. In any event, information indicating a classification result is data represented by a format comparable with correct answer information.

A structure of a neural network used by the classification unit <NUM> may be freely designed. For example, there is no limit on the number of layers, the number of components in an intermediate layer (in a case of a multilayer neural network), and a connection method between components. An activation function used in a neural network used by the classification unit <NUM> may be any activation function.

The classification unit <NUM> reads values of parameters (typically a weight and a bias) in a neural network to be used from the parameter storage unit <NUM> and performs classification.

The parameter updating unit <NUM> updates parameter values of a neural network used by the classification unit <NUM>, based on a comparison between information indicating a classification result by the classification unit <NUM> and correct answer information acquired by the data acquisition unit <NUM>.

A specific example of an updating procedure of parameter values will be described. First, for each of one or more sets of information indicating a classification result and correct answer information, the parameter updating unit <NUM> calculates an error of the information indicating the classification result with respect to the correct answer information. For example, the parameter updating unit <NUM> may use cross entropy as an error function for determining an error. Then, the parameter updating unit <NUM> determines new parameter values in such a way as to reduce the calculated error. A method known as an optimization method of parameter values employed in learning by a common classifier may be used as a technique for determining new parameter values. As an example, the parameter updating unit <NUM> may calculate a gradient by use of error back propagation and determine parameter values by use of SGD. Other employable techniques include "RMSprop," "Adagrad," "Adadelta," and "Adam.

Then, the parameter updating unit <NUM> records the determined new parameter values into the parameter storage unit <NUM>. The classification unit <NUM> thereafter uses the new parameter values. The above concludes the specific updating procedure.

The parameter updating unit <NUM> may repeatedly perform parameter value updating a predetermined number of times. For example, the predetermined number of times may be determined as a value received as an input of a numerical value indicating the predetermined number of times from a user of the learning device <NUM> through an input interface.

The output unit <NUM> outputs information about parameter values updated by the parameter updating unit <NUM>. For example, the output unit <NUM> outputs parameter values optimized by repeatedly updating the parameter values by the parameter updating unit <NUM>. Examples of an output destination of an output by the output unit <NUM> include a display device, a storage device, and a communication network. When the output unit <NUM> outputs information to a display device, the output unit <NUM> may convert information in such a way that the display device can display the information. The aforementioned display device and storage device may be devices external to the learning device <NUM> or components included in the learning device <NUM>.

A processing flow related to the classification learning by the classification learning unit <NUM> will be described referring to a flowchart in <FIG>. When each type of processing related to the classification learning is executed by a device executing a program, the each type of processing included in the processing may be executed according to the order of instructions in the program. When each type of processing is executed by a separate device, processing may be executed by a device completing the previous processing giving notification to a device executing the processing. For example, each unit performing processing records data generated by each type of processing into a storage area included in the learning device <NUM> or an external storage device. Each unit performing processing may receive data required for each type of processing from a unit generating the data or read the data from the aforementioned storage area included in the learning device <NUM> or the aforementioned external storage device.

First, the encoder <NUM> derives a latent variable vector from input data by use of parameter values optimized by the variable derivation learning (Step S31). The encoder <NUM> records the derived latent variable vector into the latent variable storage unit <NUM>.

Next, the data acquisition unit <NUM> acquires the latent variable vector derived by the encoder <NUM> and correct answer information (Step S32). The correct answer information is input to the learning device <NUM> as a set with the input data. In other words, the correct answer information is associated with the input data and the latent variable vector derived from the input data.

Next, the conversion unit <NUM> converts the latent variable vector to another latent variable vector (Step S33).

Next, the classification unit <NUM> performs classification on the aforementioned another latent variable vector (Step S34).

Next, the parameter updating unit <NUM> determines whether to end updating of values of parameters used by the encoder <NUM> and the decoder <NUM>.

As another example, a case of ending updating may be a case of an error of output data with respect to correct answer data being sufficiently small. For example, the parameter updating unit <NUM> may determine that the error is sufficiently small in the following cases and determine to end updating.

When not ending updating (NO in Step S36), the parameter updating unit <NUM> updates the parameter values (Step S37), and the classification learning unit <NUM> performs the processing in Step S34 and Step S35 again. In the processing in Step S34 from the second time onward, the classification unit <NUM> performs classification by use of the updated parameter values. The parameter updating unit <NUM> compares a classification result newly generated by the processing in Step S34 with the correct answer information again (Step S35) and determines whether to end updating of the parameter values. Thus, the classification learning unit <NUM> repeats updating of the parameter values and classification using the updated parameter values until updating of the parameters is determined to be ended. Processing of updating parameter values through such repetition is the classification learning. The parameter updating unit <NUM> updates parameter values by, in a sense, learning with a set of a classification result and correct answer information as a training data set.

When updating of the parameter values is determined to be ended (YES in Step S36), the output unit <NUM> outputs the parameter values (Step S38).

As a result of the classification learning described above, the classification unit <NUM> using updated parameter values can output a correct classification result from each of latent vectors representing various forms of a target. Accordingly, by combining the encoder <NUM> and the classification unit <NUM>, an identifier capable of identifying a target in various forms can be generated.

In learning for generating the encoder <NUM>, preparation of data in which a target takes every form is not necessarily required, as already described. In other words, the learning device <NUM> can generate an identifier capable of identifying a target in various forms even when the number of samples of data in which the target is recorded is small.

A learning device may not include a variable derivation unit <NUM>. A learning device has only to be configured in such a way as to be able to acquire a latent variable vector derived by an encoder configured to derive interconvertible latent variable vectors by variable conversion for the same target in different forms.

<FIG> is a block diagram illustrating a configuration of a learning device <NUM> according to a second example embodiment of the present invention. The learning device <NUM> includes the configuration included in the classification learning unit <NUM> according to the first example embodiment, that is, a data acquisition unit <NUM>, a conversion unit <NUM>, a classification unit <NUM>, a parameter updating unit <NUM>, an output unit <NUM>, and a parameter storage unit <NUM>. The learning device <NUM> is communicably connected to an encoder <NUM> in a wired or wireless manner.

For example, the encoder <NUM> is the encoder <NUM> according to the first example embodiment. The encoder <NUM> is configured to derive a latent variable vector by use of a neural network using the parameter values optimized by the variable derivation learning described in the description of the first example embodiment.

The learning device <NUM> can also generate an identifier capable of identifying a target in various forms. The reason is the same as the reason described in the description of the first example embodiment.

The encoder <NUM> does not need to be the encoder <NUM> according to the first example embodiment. Another method for configuring the encoder <NUM> having a desired function (that is, a function of deriving interconvertible latent variable vectors for the same target in different forms by variable conversion) will be described below.

For example, the encoder <NUM> may be generated by performing learning with a target in various forms as correct answer data and interconvertible latent variable vectors as correct answers. In the learning, output data generated by the decoder <NUM> according to the first example embodiment may be employed as correct answer data, and latent variable vectors output by the conversion unit <NUM> according to the first example embodiment may be employed as latent variable vectors to be correct answer.

As an example, one of methods of generating the encoder <NUM> having the desired function is the following method. First, a learning device <NUM> including a variable derivation unit <NUM> as illustrated in <FIG> is prepared. The learning device <NUM> performs the variable derivation learning described in the first example embodiment by using data in which various forms of a target TA are respectively recorded as input data. Consequently, output data in which various forms of the target TA are respectively recorded can be output by a combination of an encoder <NUM>, a conversion unit <NUM>, and a decoder <NUM>. Next, the learning device <NUM> derives, by the encoder <NUM>, a latent variable vector from data in which a target TB in a certain form is recorded. Then, by converting the latent variable vector by a variable conversion and generating output data, the learning device <NUM> acquires a set of output data in which the target TB in an unlearned form is recorded and a latent variable vector.

By use of the aforementioned set, the encoder <NUM> performs learning for deriving a correct latent variable vector from the data in which the target TB in the unlearned form is recorded. Consequently, the encoder <NUM> can derive a latent variable vector that can be converted from the data in which the target TB in the unlearned form is recorded to a latent variable vector representing the target TB in a learned form.

Data that need to be prepared in the aforementioned method are data in which various forms of the target TA are respectively recorded and data in which the target TB in a certain form is recorded. Data in which the target TB in an unlearned form is recorded do not need to be prepared.

A learning device <NUM> according to one example embodiment of the present invention will be described. <FIG> is a block diagram illustrating a configuration of the learning device <NUM>. The learning device <NUM> includes a data acquisition unit <NUM>, a conversion unit <NUM>, and a parameter updating unit <NUM>.

The data acquisition unit <NUM> acquires a first feature value derived from data in which an identification target is recorded. The first feature value is a feature value derived by an encoder configured to derive interconvertible feature values from data in which different forms of the same target are respectively recorded, by a conversion using a conversion parameter taking a value based on the difference between the forms. A method of implementing the aforementioned encoder is as already described.

A feature value according to the present example embodiment refers to a set of values derived from input data by the encoder. A feature value may also be referred to as information representing a target, a data representation, or the like. Derivation of a feature value may also be referred to as "extracting a feature value. " A "latent variable vector" according to each of the aforementioned example embodiments corresponds to a "feature value" according to the present example embodiment. A form in which a feature value held in the learning device <NUM> is not considered relevant. For example, a feature value may be held in an array format or may be held as values of variables assigned with names, respectively.

The conversion unit <NUM> generates a second feature value by performing a conversion using a conversion parameter on a first feature value acquired by the data acquisition unit <NUM>.

The parameter updating unit <NUM> updates values of parameters (hereinafter also referred to as "classification parameters") used in classification by a classifier (unillustrated). The classifier is a module configured to perform classification with a feature value as an input. The classification unit <NUM> according to each of the aforementioned example embodiments corresponds to the classifier. The classifier may or may not be included in the learning device <NUM>. The learning device <NUM> and a device having the classifier function may be communicably connected to each other. The classification parameter may be stored by the learning device or may be stored by the device having the classifier function. For example, the classification parameters are a weight and a bias that are generally used in a neural network.

The parameter updating unit <NUM> updates values of classification parameters in such a way that the classifier outputs a result indicating a class associated with an identification target as a classification when a second feature value is determined as an input. Specifically, the learning device <NUM> performs learning with a set of a second feature value and a result indicating a class associated with an identification target as a classification as training data.

For example, updating classification parameter values refers to recording new values of classification parameters into a storage unit storing the classification parameters. The parameter updating unit <NUM> may output the new values of the classification parameters to a device (for example, a storage device, a display device, or an information processing device using the classifier) external to the learning device <NUM>.

Referring to a flowchart in <FIG>, an example of a processing flow by the learning device <NUM> will be described. First, the data acquisition unit <NUM> acquires a first feature value (Step S301). Next, the conversion unit <NUM> generates a second feature value by performing a conversion using a conversion parameter on the first feature value (Step S302). Then, the parameter updating unit <NUM> updates classification parameter values in such a way that the classifier outputs a result indicating a class associated with an identification target as a classification when the second feature value is determined as an input (Step S303).

The learning device <NUM> can generate an identifier capable of identifying a target in various forms even when the number of samples of data in which the target is recorded is small. The reason is that when a classifier uses updated classification parameter values, data in which an identification target is recorded that may be represented by a second feature value are classified correctly (in other words, as a class associated with the identification target) even when the data are not used in learning.

A block indicating each component in each device according to each example embodiment of the present invention described above is described on a functional basis. However, a block indicating a component does not necessarily mean that each component is configured with a separate module.

For example, processing by each component may be provided by a computer system reading and executing a program causing the computer system to execute the processing, the program being stored by a computer-readable storage medium. For example, a "computer-readable storage medium" includes a portable medium such as an optical disk, a magnetic disk, a magneto-optical disk, and a nonvolatile semiconductor memory, and a storage device such as a read only memory (ROM) built into the computer system and a hard disk. A "computer-readable storage medium" also includes a medium capable of transitorily holding the program such as a volatile memory inside the computer system and a medium transmitting the program, such as a network and a communication line such as a telephone line. The aforementioned program may be a program for providing part of the aforementioned functions and may further be a program capable of providing the aforementioned functions in combination with a program already stored in the computer system.

As an example, a "computer system" is a system including a computer <NUM> as illustrated in <FIG>. The computer <NUM> includes a configuration as follows.

For example, each component in each device according to each example embodiment is provided by the CPU <NUM> loading the program 904A providing the function of the component into the RAM <NUM> and executing the program. For example, the program 904A providing the function of each component in each device is previously stored in the storage device <NUM> and/or the ROM <NUM>. Then, the CPU <NUM> reads the program 904A as needed. For example, the storage device <NUM> is a hard disk. The program 904A may be supplied to the CPU <NUM> through the communication network <NUM>, or may be previously stored in the storage medium <NUM>, be read into the drive device <NUM>, and be supplied to the CPU <NUM>. For example, the storage medium <NUM> is a portable medium such as an optical disk, a magnetic disk, a magneto-optical disk, and a nonvolatile semiconductor memory.

There are various modified examples of a method of providing each device. For example, each device may be provided by a separate, practicable combination of a computer <NUM> and a program for each component. A plurality of components included in each device may be provided by one practicable combination of a computer <NUM> and a program.

A part or the whole of each component of each device may be provided by another general-purpose or dedicated circuit, computer, and/or the like, or a combination thereof. The above may be configured with a single chip or a plurality of chips connected through a bus.

Claim 1:
A learning device comprising:
acquisition means for acquiring input data in which an identification target is recorded,
in the input data, different forms of the same target being respectively recorded,
the input data being an optical image of the target, an image generated from sensing data by a synthetic aperture radar, SAR, or sound data,
the forms being an direction, an angle, a position, a size, a degree of distortion, a hue, clarity, an azimuth angle, a depression angle, when the input data is an image;
and being a frequency, a recording place, an echo level, a tone, a reproduction speed of data, a noise level, a type of object generating sound, a person generating a sound, when the input data is sound data;
encode means for encoding the input data to a first feature value which is a latent variable vector by using a neural network;
conversion means for converting the first feature value into a second feature value which is a latent variable by using aa conversion function and a conversion parameter, the conversion function being a function shifting an arrangement of components of the first feature value based on the conversion parameter having a value according to a difference in the forms;
classification means for performing classification on the second feature value; and
parameter updating means for updating a value of a classification parameter used in classification by the classification means, in such a way that the classification means outputs a result indicating a class associated with the identification target as a classification to reduce an error of output data from the classification means with respect to correct answer data in which the target indicated by the input data is recorded in a form different from the forms in the input data, by comparing the output data and the correct answer data;
wherein correct answer information is information to be output when a target indicated in input data associated with the correct answer information is correctly identified.