Patent Publication Number: US-10783218-B2

Title: Variable group calculation apparatus, variable group calculation method, variable group calculation program, and data structure

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
     This application is based upon and claims the benefit of priority from Japanese patent application No. 2017-107122, filed on May 30, 2017, the disclosure of which is incorporated herein in its entirety by reference. 
     BACKGROUND 
     The present disclosure relates to a variable group calculation apparatus, a variable group calculation method, a variable group calculation program, and a data structure. 
     In application of machine learning, high dimensional data such as image data and voice data has a problem in which a result of the learning is affected by unnecessary information included in the data, and thus overfitting is likely to occur. 
     An effective method for avoiding such a problem is “sparse learning”. The sparse learning is a method of learning while deleting information irrelevant to a purpose using low dimensionality (sparseness) of intrinsically significant information included in the data. 
     In the sparse learning, when a desired high-dimensional observation data group (y) is represented by a linear combination of a dictionary matrix (A), column vectors of the dictionary matrix (A) are in a linear dependence relationship, and unnecessary information is often included in the dictionary matrix (A). Thus, an explanatory variable group functioning to indicate intrinsically significant column components in the dictionary matrix (A) is used as an undetermined variable group (x), which is a main factor for obtaining the observation data group (y). 
     In order to calculate the undetermined variable group (x) in the sparse learning, it is necessary to solve a minimization problem for calculating the undetermined variable group (x) that minimizes a difference value between an added composite value (Ax), which is obtained by adding and combining the undetermined variable group (x) and the dictionary matrix (A), and the observation data group (y). However, the minimization problem needs further improvement in the calculation accuracy. A representative example of the minimization problem includes a least squares method of calculating the undetermined variable group (x) that minimizes |y−Ax| 2 . 
     It is known that in the sparse learning, a constraint (a regularization term |x| p , where p is a constant) that does not increase the undetermined variable group (x) is added to the above minimization problem, and x, which simultaneously decreases (minimizes) the above-mentioned difference value and a data value including the difference value and the regularization term, is calculated, so that the undetermined variable group (x) is accurately calculated. 
     In the sparse learning, for example, the following technique is known as a technique for adding the regularization term and calculating the undetermined variable group (x). 
     Japanese Patent No. 6080783 discloses a calculation apparatus that performs the above-described calculation with the regularization term of L0 norm where p=0. 
     Japanese Unexamined Patent Application Publication No. 2017-033172 discloses a calculation apparatus that performs the above-described calculation with the regularization term of an L1 norm (|x|) where p=1 or an L2 norm (|x| 2 ) where p=2. 
     Ryota Tomioka discloses, in “Machine Learning with Sparsity Inducing Regulations (machine learning professional series)”, Kodansha Ltd., December 2015, a calculation apparatus that performs the above-described calculation with the regularization term of an L2 norm (|x| 2 ) where p=2. 
     SUMMARY 
     However, when the regularization term has p=0, it is practically a problem of combining the undetermined variable groups in the minimization problem. Thus, a large amount of data pieces in the observation data groups leads to a problem of insufficient calculation speed. 
     When the regularization term has p=1, the regularization term includes many non-consecutive parts that cannot be differentiated. For this reason, complicated branching conditions need to be included in the calculation process of the minimization problem, leading to a problem that the calculation of the minimization problem needs further improvement. 
     When the regularization term has p=2 or greater, the regularization term is a continuous function, which is easy to differentiate, hence it is expected that the calculation speed can be faster than when p=0 and p=1. However, such a case has a trade-off problem in which the undetermined variable to be a main factor is difficult to be identified because the number of undetermined variable groups obtained as results of the calculation tends to be large. 
     In light of this problem, in the sparse learning, a variable group calculation apparatus that achieves a high calculation speed and can easily identify the undetermined variable group is desired as the variable group calculation apparatus that adds the regularization term and calculates the undetermined variable group. 
     The present disclosure has been made in view of the above problem. The present disclosure provides a variable group calculation apparatus, a variable group calculation method, a variable group calculation program, and a data structure that achieve a high calculation speed and can easily identify the undetermined variable group as the main factor. 
     An example aspect of the present disclosure is a variable group calculation apparatus for calculating an undetermined variable group that simultaneously minimizes a difference value and a data value. The difference value is a difference between an added composite value, which is obtained by adding and combining the undetermined variable group and a dictionary data group, and an observation data group. The data value includes the difference value and a regularization term of the undetermined variable group. The variable group calculation apparatus includes: 
     a convolution unit configured to convert the regularization term to a convolution value for an L1 norm using the undetermined variable group and a mollifier function; and 
     a calculation unit configured to perform the calculation using the regularization term, which is converted to the convolution value by the convolution unit. 
     Another example aspect of the present disclosure is a variable group calculation method by a variable group calculation apparatus for calculating an undetermined variable group that simultaneously minimizes a difference value and a data value. The difference value is a difference between an added composite value, which is obtained by adding and combining the undetermined variable group and a dictionary data group, and an observation data group. The data value includes the difference value and a regularization term of the undetermined variable group. The variable group calculation method includes: 
     converting the regularization term to a convolution value for an L1 norm using the undetermined variable group and a mollifier function; and 
     performing the calculation using the regularization term, which is converted to the convolution value. 
     Another example aspect of the present disclosure is a variable group calculation program for causing a computer, which calculates an undetermined variable group that simultaneously minimizes a difference value and a data value, the difference value being a difference between an added composite value, which is obtained by adding and combining the undetermined variable group and a dictionary data group, and an observation data group, and the data value including the difference value and a regularization term of the undetermined variable group, to execute: 
     converting the regularization term to a convolution value for an L1 norm using the undetermined variable group and a mollifier function; and 
     performing the calculation using the regularization term, which is converted to the convolution value. 
     Another example aspect of the present disclosure is a data structure used by a variable group calculation apparatus. The data structure includes: 
     a difference value between an added composite value, which is obtained by adding and combining an undetermined variable group and a dictionary data group, and an observation data group; and 
     a regularization term of the undetermined variable group, the regularization term being a convolution value for an L1 norm using the undetermined variable group and a mollifier function. 
     The data structure is used by the variable group calculation apparatus in order to calculate the undetermined variable group that simultaneously minimizes the difference value and a data value including the difference value and the regularization term. 
     According to the respective example aspects of the present disclosure, it is possible to provide a variable group calculation apparatus, a variable group calculation method, a variable group calculation program, and a data structure that achieve a high calculation speed and can easily identify an undetermined variable group as a main factor. 
     The above and other objects, features and advantages of the present disclosure will become more fully understood from the detailed description given hereinbelow and the accompanying drawings which are given by way of illustration only, and thus are not to be considered as limiting the present disclosure. 
    
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         FIG. 1  is a diagram showing an example of an observation data group; 
         FIG. 2  is a block diagram showing a configuration example of a variable group calculation apparatus according to an embodiment; 
         FIG. 3  is a flowchart describing an example of a flow of a variable group calculation method performed by the variable group calculation apparatus according to the embodiment; 
         FIG. 4  is a diagram describing a specific example of processing of a smoothing unit setting unit according to the embodiment; 
         FIG. 5  is a diagram showing an example of a section of a mollifier function; 
         FIG. 6  is a diagram describing a specific example of processing of a convolution unit according to the embodiment; 
         FIGS. 7A to 7D  are diagrams showing an example of the mollifier function in a form of graphs; and 
         FIG. 8  is a diagram describing an example of a method of generating the mollifier function shown in  FIGS. 7A to 7D . 
     
    
    
     DESCRIPTION OF EMBODIMENTS 
     Hereinafter, a specific embodiment will be described in detail with reference to the drawings. The same or corresponding elements are denoted by the same signs throughout the drawings, and repeated descriptions will be omitted as necessary for the sake of clarity. 
     Outline of this Embodiment 
     First, an outline of this embodiment will be described. 
       FIG. 1  is a diagram showing an example of an observation data group (y) according to this embodiment. As shown in  FIG. 1 , in this embodiment, image data is handled as the observation data group (y). The image data shown in  FIG. 1  is an example, and the image data is not limited to this. 
     The observation data group (y) is expressed by the following equation (1), where β represents an explanatory variable group, which is an example of the undetermined variable group to be a main factor for obtaining the observation data group (y), and A represents a dictionary matrix, which is an example of a dictionary data group.
 
 y=A·β   (1)
 
     Here, A=(A 1 , . . . , A m ), and β=(β 1 , . . . , β q ). Further, m and q are constants. 
     Incidentally, the actual observation data group (y) relatively often has a structure (a sparse structure) in which the explanatory variable group (β) actually affecting the observation data group (y) is sparsely present. For example, in the image data, colors of adjacent pixels are mostly similar colors. When the pixels having similar colors are put together well, the information of the image data can be largely compressed. 
     In the sparse learning, although the observation data group (y) itself is large, it is used when there are only a few explanatory variables (β j ) (j is a positive integer where 1≤j≤q) actually affecting the observation data group (y). In the sparse learning, a huge number of explanatory variable groups (β) is prepared, and the explanatory variable (β j ) not affecting the observation data group (y) within the explanatory variable groups (β) is estimated to be “0”. For example, in  FIG. 1 , when the explanatory variable (β 1 ) constituting the explanatory variable group (β) is a variable not affecting the observation data group (y), the explanatory variable (β 1 ) is estimated to be “0”. 
     In the sparse learning, the regularization term of the explanatory variable group (β) is added to the minimization problem for minimizing the difference value between the added composite value (Ax), which is obtained by adding and combining the undetermined variable group (x) and the dictionary matrix (A), and the observation data group (y). Further, the explanatory variable group (β) for simultaneously minimizing the above difference value and the data value including the above difference value and the regularization term is calculated. It is known that by doing so, the explanatory variable group can be accurately calculated. 
     In the sparse learning, a cost function R(β) when the regularization term is added and the explanatory variable group (β) is calculated is expressed by, for example, the following equation (2).
 
 R (β)= f (β)+φ(β)  (2)
 
     In this equation, f(β) represents a loss function, and Ψ(β) represents a regularization term. In this embodiment, the loss function f(β) is expressed by, for example, the following equation (3).
 
 f (β)=∥ y−A·β∥ 2  (3)
 
     In addition, the regularization term Ψ(β) is typically expressed as an L1 norm, for example, by the following equation (4).
 
φ(β)=λΣ j−1   q |β j |  (4)
 
     In this equation, λ represents a regularization variable. The regularization term Ψ(β) expressed by the equation (4) is a product of the sum of the absolute values of the explanatory variables (β j ) and the regularization variable λ. The sparse learning of the type in which the regularization term Ψ(β) is expressed by the equation (4) is referred to as LASSO (Least Absolute Shrinkage and Selection Operator). 
     In the minimization problem expressed as above, the explanatory variable group (β) that simultaneously minimizes the loss function f(β) and the cost function R(β) is calculated. The loss function f(β) corresponds to the difference value between the added composite value (Ax), which is obtained by adding and combining the explanatory variable group (β) and the dictionary matrix (A), and the observation data group (y). The cost function R(β) corresponds to the data value including the loss function f(β) and the regularization term Ψ(β). 
     Here, when the regularization term Ψ(β) is a regularization term of the L1 norm like the equation (4) (hereinafter, the regularization term of the L1 norm is appropriately referred to as an L1 regularization term), it is easy to narrow down the explanatory variable group (β), which is an advantage. 
     Furthermore, if Newton&#39;s method or its variations can be applied to the calculation of the explanatory variable group (β), it is expected the calculation speed could be improved because Newton&#39;s method is quadratic convergence and has a higher speed of convergence to a solution compared with the linear convergence and the superlinear convergence. 
     However, when the waveform of the L1 regularization term Ψ(β) is sharp, Ψ(β) cannot be differentiated, hence Newton&#39;s method or its variations cannot be applied to the calculation of the above explanatory variable group (β). Thus, the calculation speed cannot be improved. 
     On the other hand, the waveform of the loss function f(β) is often smooth. 
     Thus, in this embodiment, the L1 regularization term Ψ(β) becomes a smoothed convex function, so that the entire cost function R(β) becomes a smoothed convex function. By doing so, Newton&#39;s method or its variations can be applied while making full use of the characteristics of the L1 regularization term that enables the explanatory variable group (β) to be easily narrowed down. This improves the calculation speed of the explanatory variable group (β). 
     Configuration of this Embodiment 
     Next, a configuration of this embodiment will be described. 
       FIG. 2  is a block diagram showing a configuration example of the variable group calculation apparatus  1  according to this embodiment. As shown in  FIG. 2 , a hardware configuration of the variable group calculation apparatus  1  according to this embodiment includes a processor  10 , a memory  20 , and an interface (I/F) unit  30 . The processor  10  includes a smoothing unit setting unit  11 , a convolution unit  12 , and a calculation unit  13 . 
     The memory  20  stores a program (a calculation program) including instructions to be executed by the processor  10 . The memory  20  is, for example, a volatile memory, a non-volatile memory, or a combination thereof. 
     The interface (I/F) unit  30  inputs and outputs various information items from and to the outside. 
     The processor  10  reads the program including the instructions from the memory  20  and executes it to thereby achieve the functions of the smoothing unit setting unit  11 , the convolution unit  12 , and the calculation unit  13 . The smoothing unit setting unit  11 , the convolution unit  12 , and the calculation unit  13  will be described later in detail. The processor  10  is a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a microprocessor, a combination thereof, or the like. 
     The above-mentioned program can be stored and provided to a computer using any type of non-transitory computer readable media. Non-transitory computer readable media include any type of tangible storage media. Examples of non-transitory computer readable media include magnetic storage media (such as floppy disks, magnetic tapes, hard disk drives, etc.), optical magnetic storage media (e.g., magneto-optical disks), CD-ROM (Compact Disk-Read Only Memory), CD-R (CD-recordable), CD-R/W (CD-rewritable), and semiconductor memories (such as mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory), etc.). 
     The program may be provided to a computer using any type of transitory computer readable media. Examples of transitory computer readable media include electric signals, optical signals, and electromagnetic waves. Transitory computer readable media can provide the program to a computer via a wired communication line (e.g. electric wires, and optical fibers) or a wireless communication line. 
     Operation of this Embodiment 
     Next, an operation of this embodiment will be described. 
       FIG. 3  is a flowchart illustrating an example of a flow of a variable group calculation method performed by the variable group calculation apparatus  1  according to this embodiment. 
     As shown in  FIG. 3 , firstly, the observation data group (y), the dictionary matrix (A), and information about the necessary number of differentiation of the L1 regularization term are input from outside to the interface (I/F)  30  (Step S 1 ). Note that the observation data group (y) is the image data shown in  FIG. 1 . The dictionary matrix (A) has a dictionary matrix corresponding to the observation data group (y). The necessary number of differentiations of the L1 regularization term is two or more. 
     Next, the smoothing unit setting unit  11  normalizes each of the explanatory variables (β j ) so that a variance (σ 2   j ) becomes 1 and a mean (μ j ) becomes 1, and sets smoothing units. The smoothing unit indicates a range in which a function value of each of the normalized explanatory variables (β j ) is smoothed. The smoothing unit setting unit  11  sets a plurality of the smoothing units for each explanatory variable (β j ) (Step S 2 ). 
     Next, the convolution unit  12  determines a mollifier function (a polynomial) according to the necessary number of differentiations of the L1 regularization term (Step S 3 ). As described later, the mollifier function of an order n is a function that can be differentiated n−1 times. Thus, when the necessary number of differentiations is n d  (n d  is an integer of two or greater), a mollifier function having an order not less than (n d +1) is determined. 
     Next, the convolution unit  12  selects one of the plurality of smoothing units set in Step S 2  (Step S 4 ). After that, the minimization problem is solved for the smoothing unit selected in Step S 4 . 
     Next, the convolution unit  12  converts, for each explanatory variable (β j ), the function value of each explanatory variable (β j ) within the range indicated by the smoothing unit selected in Step S 4  to a convolution value to be convoluted with the mollifier function determined in Step S 3  so as to smooth the function value (Step S 5 ). 
     As expressed by the equation (4), the L1 regularization term Ψ(β) is represented by the product of the sum of the absolute values of the explanatory variables (β j ) and the regularization variable λ. For each smoothing unit selected in Step S 4 , the function value of each explanatory variable (β j ) is smoothed, to thereby smooth the L1 regularization term Ψ(β). Then, the L1 regularization term Ψ(β) becomes a convex function that can be differentiated for the necessary number of differentiations. As a result, Newton&#39;s method or its variations can be applied to the calculation of the explanatory variable group (β), as the entire cost function R(β) becomes a smoothed convex function. 
     Thus, the calculation unit  13  applies Newton&#39;s method or its variations to calculate the explanatory variable group (β) that simultaneously minimizes the loss function f(β) and the cost function R(β) (Step S 6 ). This consequently improves the calculation speed of the explanatory variable group (β). 
     Next, the calculation unit  13  estimates the explanatory variable (β j ) that does not affect the observation data group (y) within the explanatory variable group (β) calculated in Step S 6  to be “0” (Step S 7 ). 
     The calculation of the minimization problem is thus complete for the smoothing unit selected in Step S 4 . 
     Next, the calculation unit  13  decides whether there is an unselected smoothing unit yet to be selected out of the plurality of smoothing units set in Step S 2  (Step S 8 ). If there is an unselected smoothing unit (YES in Step S 8 ), the process returns to Step S 4  where the convolution unit  12  selects the unselected smoothing unit, and solves the minimization problem for the selected smoothing unit likewise. 
     On the other hand, if there is no unselected smoothing unit (NO in Step S 8 ), the calculation unit  13  outputs, for each of the plurality of smoothing units set in Step S 2 , the explanatory variable group (β) estimated in Step S 7  to the outside via the interface (I/F) unit  30  (Step S 9 ). 
     Accordingly, an external apparatus, which has received the explanatory variable group (β) for each of the plurality of smoothing units, can obtain, for each of the plurality of smoothing units, the main explanatory variables (β j ) that affect the observation data group (y) and the number of the main explanatory variables (β j ). 
     In  FIG. 3 , although the explanatory variable group (β) for each of the plurality of smoothing units is collectively output to the outside in Step S 9 , it is not limited to this. For example, in Step S 7 , when the calculation of the minimization problem for the smoothing unit selected in Step S 4  is completed, the explanatory variable group (β) of this smoothing unit may be output. 
     Hereinafter, the processing of the smoothing unit setting unit  11 , the convolution unit  12 , and the calculation unit  13  will be described in detail using a specific example. 
     First, a specific example of the processing of the smoothing unit setting unit  11  will be described. 
       FIG. 4  is a diagram for describing a specific example of the processing of the smoothing unit setting unit  11  when q−2, where q represents the number of explanatory variables (β j ).  FIG. 5  is a diagram showing an example of a section of the mollifier function. 
     As shown in  FIG. 4 , a range in which data pieces are distributed differs for each explanatory variable (β j ). Therefore, for each explanatory variable (β j ), the smoothing unit setting unit  11  sets the smoothing unit using statistics of explanatory variable (β j ) such as a standard deviation (σ j ), the variance (σ 2   j ), and the mean (μ j ). 
     To be more specific, firstly the smoothing unit setting unit  11  statistically performs, for each explanatory variable (β j ), the normalization processing using the following equation (5) so that the variance (σ 2   j ) becomes 1 and the average (μ j ) becomes 1.
 
{tilde over (β)} j (β j −μ j )/σ j   (5)
 
In this equation,
 
{tilde over (β)} j  
 
represents the normalized explanatory variable (β j ).
 
     As shown in  FIG. 5 , the section of the mollifier function is, for example, [−a n /2, a n /2]. The smoothing unit setting unit  11  sets, for each explanatory variable (β j ), a plurality of smoothing units by setting a n , for example, as follows. The following “a n ” is an example, and it is not limited to this.
 
 a   n =0.1 σ j ,0.2 σ j , . . . ,0.9 σ j ,1 σ j ,2 σ j ,3 σ j , . . .
 
     That is, firstly, the smoothing unit setting unit  11  sets a n =0.1 σ j , and sets, for each explanatory variable (β j ), the section of the normalized explanatory variable (β j ) of [−0.05 σ j , 0.05 σ j ] as the smoothing unit. 
     Next, the smoothing unit setting unit  11  sets a n =0.2 σ j  and sets, for each explanatory variable (β j ), the section of the normalized explanatory variable (β j ) of [−0.1 σ j , 0.1 σ j ] as the smoothing unit. 
     In this way, the smoothing unit setting unit  11  sets the plurality of smoothing units for each explanatory variable (β j ). 
     Hereinafter, a specific example of the processing of the convolution unit  12  will be described. 
       FIG. 6  is a diagram for describing a specific example of the processing of the convolution unit  12  when q=2, where q represents the number of explanatory variables (β j ). 
     The convolution unit  12  selects one of the plurality of smoothing units set by the smoothing unit setting unit  11  and, for each explanatory variable (β j ), converts each function value of the explanatory variable (β j ) within the range indicated by the selected smoothing unit to the convolution value to be convoluted with the mollifier function so as to smooth the function value. 
     For example, when the smoothing unit with a n =0.2 σ j  is selected, as shown in  FIG. 6 , the convolution unit  12  matches, for each explanatory variable (β j ), a section [−a n /2, a n /2] of the mollifier function with the section [−0.1 σ j , 0.1 σ j ] of the function value of the normalized explanatory variable (β j ), and calculates a composite product of the two sections to smooth the function value. In  FIG. 6 , the solid lines are waveforms before the smoothing and the broken lines are the waveforms after the smoothing. 
     The above-mentioned mollifier function will be described in detail. 
       FIGS. 7A to 7D  are diagrams showing an example of the mollifier function T n (x) in a form of graphs. 
     The mollifier function is, for example, a Ternary Polynomial function. The ternary polynomial function is a polynomial function that has three regions of a function value increasing region, constant region, and decreasing region. The increasing region and the decreasing region are symmetrical with the constant region interposed therebetween. Details of the ternary polynomial are disclosed in Japanese Unexamined Patent Application Publication No. 2009-053926 already filed by the applicant of the present disclosure and shall be incorporated herein by reference. 
     The mollifier function T n (x) (n is 0 or a positive integer) is a function that can be differentiated n−1 times, and has constants a 0 , a 1 , . . . a n , and C. Here, d is defined as d=(a 0 , a 1 , . . . , a n , C). Further, C is adjusted in advance so that an integrated value of the mollifier function T n (x) becomes 1. 
       FIG. 7A  shows a mollifier function T 0 (x) of an order 0 with a rectangular wave profile.  FIG. 7B  shows a mollifier function T 1 (x) of an order 1 with a trapezoidal profile.  FIG. 7C  shows a mollifier function T 2 (x) of an order 2 with a smoothly changing trapezoidal profile.  FIG. 7D  shows a mollifier function T 3 (x) of an order 3 with is a smoothly changing trapezoidal profile. 
       FIG. 8  is a diagram for describing an example of a method of generating the mollifier function T n (x) shown in  FIGS. 7A to 7D . 
     As shown in  FIG. 8 , the mollifier function T n (x) distributes the waveform of T n (x) symmetrically about the origin of the coordinate system of T n (x)−x to generate T′ n+1 (x). This T′ n|1 (x) is integrated to generate T n|1 (x) with an order higher by one than that of T n (x). 
     For example, when T 2 (x) is generated from T 0 (x), T 0 (x) may be symmetrically divided (symmetrically distributed) and integrated to generate T 1 (x), and T 1 (x) may be symmetrically divided and integrated to generate T 2 (x). Alternatively, after T 0 (x) is symmetrically divided, it may be further symmetrically divided to generate T″ 2 (x), and T″ 2 (x) may be integrated twice to generate T 2 (x). 
     Next, a method of calculating a composite product of the function value of the explanatory variable (β j ) and the mollifier function T n (x) will be described in detail. 
     The convolution unit  12  calculates a composite product (Ψ j ) d (x) using the following equation (6) based on Ψ j (β j ) representing the function value of the explanatory variable (β j ) and the mollifier function T n (x) of the above-mentioned polynomial to obtain the convolution value.
 
(Ψ j ) d ( x )=∫ −∞   +∞   T   n ( x−β   j )Ψ j )β j ) dβ   j )  (6)
 
     In this equation, (Ψ j ) d (x) represents a function that can be differentiated n−1 times and uniformly converge to (Ψ j ) d →Ψ j . The composite product of the absolute value function and the mollifier function (polynomial) like the one above can be calculated by algebraic processing with a small processing load. This achieves high-speed processing. 
     Next, a specific example of the processing of the calculation unit  13  will be described. 
     As described above, the convolution unit  12  converts the function value of each explanatory variable (β j ) to the convolution value to be convolved with the mollifier function so as to smooth the function value. Thus, the L1 regularization term Ψ(β) is smoothed and becomes the convex function that can be differentiated for the necessary number of differentiations. 
     That is, before the smoothing of the L1 regularization term Ψ(β), the minimization problem is expressed as a minimization problem of LASSO as in the following equation (7).
 
 R (β)∥ y−A·β∥   2 +φ(β)  (7)
 
     On the other hand, after the L1 regularization term Ψ(β) is smoothed, the minimization problem is expressed as the minimization problem of the composite product LASSO (Convolutional LASSO) as shown in the following equation (8).
 
 R (β)=∥ y−A·β∥   2 +{tilde over (φ)}(β)  (8)
 
In this equation,
 
{tilde over (φ)}(β)
 
represents the smoothed L1 regularization term Ψ(β).
 
     The composite product LASSO is a convex function, and thus Newton&#39;s method or its variations may be applied to calculate a global solution. 
     Accordingly, the calculation unit  13  applies Newton&#39;s method or its variations to calculate the explanatory variable group (β) that simultaneously minimizes ∥y−A−β∥ 2  and the cost function R(β). 
     Next, the calculation unit  13  compares each of the explanatory variables (β j ) constituting the above-mentioned calculated explanatory variable group (β) with a corresponding ε j . That is, the calculation unit  13  compares each of β=(β 1 , β 2 ) with corresponding one of (ε 1 , ε 2 ). Here, ε j  is, for example, 0.1 σ j . Note that this ε j  is an example, and it is not limited to this. 
     Then, if β j &lt;ε j , the calculation unit  13  decides that the explanatory variable (β j ) does not affect the observation data group (y), and estimates the explanatory variable (β j ) to be “0”. Further, when β j ≥ε j , the calculation unit  13  decides that the explanatory variable (β j ) affects the observation data group (y), and leaves the explanatory variable (β j ) as it is. For example, if β 1 &lt;ε 1  and β 2 ≥ε 2 , the calculation unit  13  estimates the explanatory variable (β 1 ) to be “0” and sets the explanatory variable group (β)=(0, β 2 ). In this case, the main explanatory variable is β 2 , and the number of main explanatory variables is one. 
     The calculation unit  13  outputs the estimated explanatory variable group (β) for each of the plurality of smoothing units, which is calculated as described above, to the outside via the interface (I/F) unit  30 . 
     Effect of this Embodiment 
     Next, an effect of this embodiment will be described. In the sparse learning, the variable group calculation apparatus  1  according to this embodiment, when adding the regularization term and calculating the explanatory variable group (β), firstly converts the regularization term Ψ(β) to the convolution value for the L1 regularization term, which is the regularization term of the L1 norm, and the mollifier function so as to smooth the regularization term Ψ(β). After that, the smoothed regularization term Ψ(β) is used as the convolution value to calculate the explanatory variable group (β). 
     This makes it possible to differentiate the regularization term Ψ(β) two or more times while making full use of the characteristics of the L1 norm that enables the explanatory variable group (β) to be easily narrowed down as the main factor, and Newton&#39;s method (the quadratic convergence) or its variations can be applied to the calculation of the explanatory variable group (β). This guarantees the convergence order of the quadratic convergence or greater order convergence, and thus the calculation speed of the explanatory variable group (β) can be improved. Therefore, the explanatory variable group (β) can be easily identified, and the calculation speed can be improved. 
     Further, the variable group calculation apparatus  1  of this embodiment determines the mollifier function according to the necessary number of differentiations of the L1 regularization term Ψ(β). At this time, when the order of the mollifier function is increased, the acceleration method of the required convergence order can be applied. 
     Note that the present disclosure is not limited to the above-described embodiment, and can be appropriately changed without departing from the spirit of the present disclosure. 
     For example, in the above embodiment, the L1 norm is used, but the present disclosure can be applied to when the L1 norm is changed to the Lp norm (p≠2). 
     In the above embodiment, the image data is used as the observation data group. However, the observation data group is not limited to this, and any large data group (big data) may be used. Examples of the observation data include voice data (conversation data), biometric data, astronomical data, natural language processing data. 
     From the disclosure thus described, it will be obvious that the embodiments of the disclosure may be varied in many ways. Such variations are not to be regarded as a departure from the spirit and scope of the disclosure, and all such modifications as would be obvious to one skilled in the art are intended for inclusion within the scope of the following claims.