Embedded Feature Selection Using Probabilistic Model-Based
Optimization
Shota Saito

Shinichi Shirakawa

Youhei Akimoto

Yokohama National University
Kanagawa, Japansaito-shota-bt@ynu.jp

Yokohama National University
Kanagawa, Japanshirakawa-shinichi-bg@ynu.ac.jp

University of Tsukuba
Ibaraki, Japanakimoto@cs.tsukuba.ac.jp

ABSTRACT1

In machine learning, feature selection is a commonly used technique for improving the predictive performance and interpretabilityof a trained model. Feature selection techniques are classified into
three approaches: the filter, wrapper, and embedded approaches.
The embedded approach performs the feature selection processduring the model training and achieves a good balance between
performance and computational cost in general. In the paper, we
propose a novel embedded feature selection method using probabilistic model-based evolutionary optimization. We introduce themultivariate Bernoulli distribution, which determines the selectionof features, and we optimize its parameters during the training.
The distribution parameter update rule is the same as that of the
population-based incremental learning (PBIL), but we simultaneously update the parameters of the machine learning model using an
ordinary gradient descent method. This method can be easily implemented into non-linear models, such as neural networks. Moreover,
we incorporate the penalty term into the objective function to control the number of selected feature. We apply the proposed methodwith the neural network model to the feature selection of threeclassification problems. The proposed method achieves competitive
performance and reasonable computational cost compared withconventional feature selection methods.

In machine learning, feature selection is useful for improving thepredictive performance and identifying which features are important. Feature selection techniques are generally classified into three
approaches: the filter, wrapper, and embedded approaches [4]. Evolutionary computation has been mainly applied to the wrapper
and filter approaches [21]. Whereas, the embedded approach isconsidered as having a good balance between computational costand accuracy because it performs the selection of features during
the model training. The typical method of the embedded approach
is LASSO [18] and similar methods [12, 22], which use the L1 regularization for the weight coefficients in a linear model. LASSO canbe only applied to the linear model in general and is difficult tobe applied to the complex models such as neural networks as thefeature selection method.
In this paper, we propose a novel method called the probabilistic
model-based embedded feature selection (PEFS) that can easily incorporate non-linear models such as neural networks. In the PEFS,
we define the multivariate Bernoulli distribution of the binary vector and use the vector for feature selection. The key technique torealize the embedded feature selection is to optimize the parametersof the distribution during the model training introduced in [17]. We
formulate the objective function to be minimized as the expectationof loss function under the distribution for the feature selection
and show that such formulation relates to the information geometric optimization (IGO) [14] and population-based incremental
learning (PBIL) [2]. Different from the IGO and PBIL, we simultaneously optimize the parameters of machine learning model (e.g., theweight parameters of neural network) using an ordinary gradientdescent method. Furthermore, we introduce the penalty term into
the objective function to control the number of selected features.
To evaluate the performance of the PEFS, we apply the PEFS witha neural network model to three classification problems. From theexperimental results, the PEFS shows the competitive performanceand reasonable computational cost compared with some existing
feature selection methods.
The contribution of this paper is as follows: (1) applying the idea
introduced in [17], which simultaneously optimizes both of theweight parameters of neural network and the Bernoulli parameters
based on the gradient descent1 , to the feature selection problem,and (2) incorporating the penalty term into the objective function
to control the number of selected features.

CCS CONCEPTS• Computing methodologies → Feature selection; Machinelearning; • Theory of computation → Evolutionary algorithms;

KEYWORDS
Feature Selection, Embedded Approach, Natural Gradient, Information Geometric Optimization, Neural Network
ACM Reference Format:
Shota Saito, Shinichi Shirakawa, and Youhei Akimoto. 2018. Embedded
Feature Selection Using Probabilistic Model-Based Optimization. In GECCO’18 Companion: Genetic and Evolutionary Computation Conference Companion, July 15–19, 2018, Kyoto, Japan. ACM, New York, NY, USA, 4 pages.
https://doi.org/10.1145/3205651.3208227

GECCO ’18 Companion, July 15–19, 2018, Kyoto, Japan© 2018 Association for Computing Machinery.
This is the author’s version of the work. It is posted here for your personal use.
Not for redistribution. The definitive Version of Record was published in GECCO ’18
Companion: Genetic and Evolutionary Computation Conference Companion, July 15–19,
2018, Kyoto, Japan, https://doi.org/10.1145/3205651.3208227.

INTRODUCTION

1 The framework proposed in [17] uses the Bernoulli distribution to represent theneural network structures such as the selection of layers or units and optimize theneural network structures during the training.

GECCO ’18 Companion, July 15–19, 2018, Kyoto, Japan2

Shota Saito, Shinichi Shirakawa, and Youhei Akimoto

PEFS: THE PROPOSED METHOD

We consider the general problem setting of supervised learningwith a given training dataset D = {𝑋, 𝑌 }. Let 𝑋 = {𝑥 1, . . . , 𝑥 𝑁 } bethe set of input vectors, 𝑥𝑖 ∈ R𝑑 , and 𝑌 = {𝑦1, . . . , 𝑦𝑁 } be the set oftarget variables. In the classification problem, 𝑌 refers to the labels,
whereas it refers to continuous values in the regression problem.
In the PEFS, the features to be used are selected by a random variable consisting of 𝑑-dimensional binary vector 𝑀 = (𝑚 1, . . . , 𝑚𝑑 ) > ,𝑚𝑖 ∈ {0, 1}. Given a binary vector 𝑀 and an input vector 𝑥, the inputof the prediction model is represented by diag(𝑀)𝑥, where diag represents a diagonal matrix whose diagonal elements are zero or one
depending on the corresponding element of 𝑀. This means that thevalues of the unused features become zero. We consider the probability distribution for the random variable 𝑀 as the multivariateÎBernoulli distribution defined by 𝑝 (𝑀 | 𝜃 ) = 𝑑𝑖=1 𝜃𝑖𝑚𝑖 (1 − 𝜃𝑖 ) 1−𝑚𝑖 ,>where 𝜃 = (𝜃 1, . . . , 𝜃𝑑 ) , 𝜃𝑖 ∈ [0, 1] refers to the parameters of the
distribution. The parameter 𝜃𝑖 corresponds to the probability thatthe 𝑖-th feature is selected, i.e., 𝑚𝑖 = 1.
Let us denote the model to predict the target variable from the
input vector as 𝜙 (𝑋,𝑊 , 𝑀), and the loss function to be minimized as
L (𝑌 , 𝑋,𝑊 , 𝑀). We often use the mean squared errors for regression
and the softmax cross-entropy losses for classification. We notethat the filter approach in feature selection optimizes the model
parameter 𝑊 using the pre-selected binary vector 𝑀, whereas thewrapper approach searches the better binary vector 𝑀 based on thevalidation error of the optimized model parameter 𝑊 . Therefore,
the wrapper approach generally requires a much computationalcost. Since we treat the binary vector 𝑀 as the random variable,
we consider minimizing the expected loss function as G(𝑊 , 𝜃 ) =Í𝑀 ∈M L (𝑌 , 𝑋,𝑊 , 𝑀)𝑝 (𝑀 | 𝜃 ).
Furthermore, to reduce and control the number of selected features, we add the term with respect to (w.r.t.) the number of selectedÍfeatures 𝑚𝑘 as the penalty. Considering the loss function withthe penalty of the number of selected features, its expectation under𝑝 (𝑀 |𝜃 ) is given by"#𝑑ÕG(𝑊 , 𝜃 ) = E L (𝑌, 𝑋,𝑊 , 𝑀) + 𝜖
(1)
𝑚𝑘
𝑘=1

=

Õ
𝑀 ∈M

L (𝑌, 𝑋,𝑊 , 𝑀)𝑝 (𝑀 | 𝜃 ) + 𝜖

𝑑
Õ

𝜃𝑘 ,

(2)

𝑘=1

where 𝜖 is the coefficient to control the influence of the penalty.
In the case of 𝜖 = 0, the minimization of G(𝑊 , 𝜃 ) w.r.t. 𝜃 canbe viewed as the same formulation in the information geometric
optimization (IGO) [14] with the objective function L (𝑌, 𝑋,𝑊 , 𝑀).
The IGO updates the parameters of the distribution toward thenatural gradient direction [1], which is the steepest direction of 𝜃
w.r.t. the KL-divergence. The natural gradient ∇˜ 𝜃 G(𝑊 , 𝜃 ) is givenby∇˜ 𝜃 G(𝑊 , 𝜃 ) =Õ𝑀 ∈M

L (𝑌, 𝑋,𝑊 , 𝑀) ∇˜ 𝜃 ln 𝑝 (𝑀 | 𝜃 ) + 𝜖 ∇˜ 𝜃

𝑑
Õ

𝜃𝑘 ,

𝑘=1

(3)
where the natural gradients of the log-likelihood and the penalty areÍgiven by ∇˜ 𝜃 ln 𝑝 (𝑀 | 𝜃 ) = F−1 (𝜃 )∇𝜃 ln 𝑝 (𝑀 | 𝜃 ) and ∇˜ 𝜃 𝑑𝑘=1 𝜃𝑘 =

ÍF−1 (𝜃 )∇𝜃 𝑑𝑘=1 𝜃𝑘 , respectively, and F−1 (𝜃 ) denotes the Fisher information matrix of 𝑝 (𝑀 | 𝜃 ). Since we consider the Bernoulli
distribution as 𝑝 (𝑀 | 𝜃 ), the natural gradients can be obtainedÍ
analytically as ∇˜ 𝜃 ln 𝑝 (𝑀 | 𝜃 ) = 𝑀 − 𝜃 and ∇˜ 𝜃 𝑑𝑘=1 𝜃𝑘 = 𝜃 (1 − 𝜃 ).
Unlike the IGO and PBIL, minimizing G(𝑊 , 𝜃 ) requires optimization not only w.r.t. the distribution parameters 𝜃 but also w.r.t. themodel parameters 𝑊 . The gradient w.r.t. 𝑊 is given byÕ∇𝑊 G(𝑊 , 𝜃 ) =∇𝑊 L (𝑌, 𝑋,𝑊 , 𝑀) .
(4)
𝑀 ∈M

We note that the gradient ∇𝑊 L (𝑌, 𝑋,𝑊 , 𝑀) can be computed by
the back-propagation when the neural network model is considered.
We realize the embedded feature selection by simultaneously
optimizing 𝜃 and 𝑊 using (3) and (4), respectively. In practice, thegradients (3) and (4) are approximated by Monte-Carlo method
using 𝜆 samples of 𝑀 drawn from 𝑝 (𝑀 | 𝜃 ).
In most practical cases, the loss L (𝑌, 𝑋,𝑊 , 𝑀𝑖 ) is approximatedusing mini-batch samples Z = {(𝑥 1, 𝑦1 ), . . . , (𝑥 𝑁¯ , 𝑦𝑁¯ )}. Referring
to [17], we use the same mini-batch between different 𝑀𝑖 to obtain
the accurate rankings of losses. The approximated loss function
Íis given by L̄ (𝑌, 𝑋,𝑊 , 𝑀𝑖 ) = 𝑁¯ −1 (𝑦,𝑥) ∈Z 𝑙 (𝑦, 𝑥,𝑊 , 𝑀𝑖 ), where𝑙 (𝑦, 𝑥,𝑊 , 𝑀) represents the loss of a datum, and 𝑁¯ indicates thenumber of the mini-batch samples. Also, we transform the loss
value into the ranking-based utility as done in [17] as follows:
1(best d𝜆/4e samples)L̄ (𝑌, 𝑋,𝑊 , 𝑀𝑖 ) ↦→ 𝑢𝑖 = −1 (worst d𝜆/4e samples) . (5)0(otherwise)With the transformation, the 𝜃 update using 𝜆 samples drawn from𝑝 (𝑀 | 𝜃 ) reads!
𝜆Õ𝑢𝑖(𝑡 +1)
(𝑡 )
(𝑡 )
(𝑡 )
(𝑡 )
𝜃
= 𝜃 + 𝜂𝜃(𝑀𝑖 − 𝜃 ) − 𝜖𝜃 (1 − 𝜃 ) , (6)𝜆𝑖=1where 𝜂𝜃 is the learning rate for 𝜃 . If 𝜖 is zero, the update rule of 𝜃
recovers the PBIL [2] which is one of the estimation of distribution
algorithms. Moreover, we restrict the range of 𝜃 within [1/𝑑, 1−1/𝑑]in order to keep the possibility of generating any binary vector.
As proposed in [17], there are two options to predict a new datausing optimized 𝜃 and 𝑊 . First, the binary vectors are sampledfrom 𝑝 (𝑀 | 𝜃 ) and the prediction results are averaged. While thisprediction method can reach an accurate prediction, it is not adesirable approach from the feature selection perspective because
it may use all features. Another way is to deterministically decide
the binary vector 𝑀 such that 𝑚𝑖 = 1 if 𝜃𝑖 ≥ 0.5; otherwise, 𝑚𝑖 = 0.
In the experiment, we use this deterministic prediction and report
the results.

3

EXPERIMENT AND RESULTIn this section, we evaluate the PEFS with the neural networkmodel on three classification problems. The datasets used in theexperiment are summarized in Table 1.

3.1

Experimental Setting

For all datasets, the neural network consisting of three fully connected hidden layers with 200 units and the softmax output layer
is used. Each hidden unit performs the rectified linear unit (ReLU)[13] as the activation, followed by the batch normalization [8]. The

Embedded Feature Selection Using Probabilistic Model-Based Optimization
Table 1: The summary of datasets used in the experiment.

Datasetpcmac [11]usps [7]adult [10]No. of data
(training / test)
1555 / 388
7439 / 1859
36178 / 9044

No. offeatures
3289
256
104

No. ofclasses
2
10
2

Data
domainArticleImage
Incomenumber of units in the input and output layers depends on the number of features and classes. The weight parameters are initialized
by He’s initialization [6]. We use the cross-entropy loss function
as L. The weight parameters are optimized using Adam [9], an
accelerated stochastic gradient method, with the default parametersetting, and we use the weight decay of 10−4 .
In the PEFS, we initialize the distribution parameters by 𝜃 init =0.5 and use the learning rate of 𝜂𝜃 = 1/𝑑, where 𝑑 is the numberof features. The mini-batch size 𝑁¯ is set to 𝑁¯ = 128 in the PEFS.
The sample size 𝜆 is set to two2 . We vary the coefficient 𝜖 from0 to 1 in 0.1 increments to check the effect. The natural gradientcorresponding to L is bounded in [−0.5, 0.5]𝑑 because of the utility
transformation defined in (5), and the one corresponding to thepenalty term is within [0, 𝜖/4]𝑑 .Therefore, it is reasonable to set 𝜖in [0, 1] independently of dataset s. The parameter update in the
training is terminated at 100,000 iterations for all datasets.
We compare the performance of the PEFS with those of existing
feature selection methods. For the comparison, the minimum redundancy and maximum relevance (mRMR) [15] and ReliefF [16]were used as the filter approach; the wrapper method using particle
swarm optimization (PSO), which is called ErFS in [20], was usedas the wrapper approach; and LASSO with a linear model was usedas the embedded approach. Filter methods select the features basedon the feature ranking and train the above-mentioned neural network model using high-ranked feature subset. The fitness used in
ErFS is based on the error of the randomly selected 30% validation
data from the training data. We set the numbers of individuals andgenerations to 10 and 25, respectively, in ErFS. This setting uses
smaller values than those used in [20] to reduce the computational
cost because the above-mentioned neural network model is morecomputationally expensive than the model used in [20]. In these
existing methods, the mini-batch size of the neural network training is set to 𝜆 𝑁¯ , which is the same number of data samples to be
used at one iteration in the PEFS, for a fair comparison. In LASSO,
the linear model is optimized by the SAGA [3] with 500 iterations.
We employ the different regularization factors from 1 to 10 in 1 increments. The features are selected if the trained model parameters
are greater than 10−5 in LASSO.

3.2

Result and Discussion

Prediction Performance and Number of Features. Figure 1 shows
the relation between the percentile of selected features and thetest error. The result of the different numbers of selected features,from 10% to 100% in 10% increments, are reported in the filter
methods. The plotted values are the average values of test errors of
2 The 𝜃 update in (6) with 𝜆 = 2 and 𝜖 = 0 recovers the update rule of the compact

genetic algorithm (cGA) [5].

GECCO ’18 Companion, July 15–19, 2018, Kyoto, Japan

Table 2: Computational time (hour) of a typical single run.
We report the total time of the feature selection and neuralnetwork training. Note that LASSO uses the linear model.

Dataset
pcmac
usps
adult

Without
FS
0.675
0.451
0.458

Embedded
PEFS / LASSO
0.748 / 0.010
0.659 / 0.027
0.601 / 0.009

FilterReliefF / mRMR
0.857 / 11.1
0.549 / 1.05
1.27 / 0.682

Wrapper
PSO
169
114
113

10 different neural network trainings. In the PEFS, we also report
the average test errors and average percentile of selected features of
10 different trainings for each setting. The results of the PSO-based
wrapper method and LASSO are reported for a single run.
Comparing the PEFS and filter methods, the PEFS outperforms
the filter methods for small feature subset around from 10% to 40% in
the usps and adult datasets. For the pcmac dataset, the performancesof the PEFS outperforms the filter method for selected features ofaround 50%.
Comparing the PEFS and LASSO as an embedded approach, thePEFS outperforms LASSO in the usps dataset on the same percentileof selected features. However, LASSO outperforms the PEFS andfilter methods in the pcmac dataset. Interestingly, the PEFS reaches
the equivalent performance with LASSO when the fewer number offeatures are selected in the adult dataset. We consider that this performance difference is caused by the model difference; i.e., LASSO
uses the linear model. These results demonstrate that the PEFS cantake good advantage of the neural network model as the embedded approach. In addition, the percentile of the selected features in
LASSO is not distributed compared to the PEFS. This is because theappropriate scale of the regularization factor in LASSO depends onthe dataset. In contrast, the PEFS does not require to tune the scaleof 𝜖 by the utility transformation.
Next, we compare the test error with ErFS, the PSO-based wrapper approach. ErFS outperforms the filter methods except for the
pcmac dataset. In general, the wrapper method is superior to thefilter method, but it is computationally expensive when the complicated model is used. In our setting, the numbers of individuals andgenerations may not be sufficient to reach the fewer test errors. The
performances of the PEFS and ErFS are comparable on the same
percentile of selected features.
Computational Costs. We compare the total computational timesof the feature selection and neural network trainings. Table 2 shows
the computational times of a typical single run for each method andthe case of using all features (i.e., without feature selection (FS)). For
three datasets, the computational time of the PEFS was about 1.2 to
1.5 times as long as without the feature selection, while ReliefF andmRMR require more computational time as the number of training
data or feature increase. The computational complexity of ReliefF
and mRMR is 𝑂 (𝑑𝑁 ) [16] and 𝑂 (𝑑 2 ) [19], respectively. LASSO isfast because it uses the linear model, unlike the other methods.
Obviously, the PSO-based wrapper method requires significant
computational cost because it needs the cycle of the model training
to evaluate the feature subset.

Test Error (%)

GECCO ’18 Companion, July 15–19, 2018, Kyoto, Japan
1.0

pcmac

usps (log-scale)

102

40
0.5

Shota Saito, Shinichi Shirakawa, and Youhei Akimoto

adult
25

ReliefF
mRMR
LASSO
Proposed Method
PSO

20

101

20
0.0
0.00

50

0.2

100

0 0.4

50

0.6 100

15

0

0.8

50

100
1.0

Percentile of Selected Features (%)
Figure 1: The relations between the percentile of selected features and the test error for each dataset. The vertical and horizontal axes represent the test error rate and the percentile of the selected features, respectively. For the mRMR, ReliefF andPEFS (proposed method), the average value and standard deviation of 10 different model trainings are plotted.
In the PEFS, the additional computational time for the feature
selection does not have a significant impact because the additional
computation is the computation regarding the sampling and updating of 𝑝 (𝑀 | 𝜃 ). The experimental result shows that the PEFS canselect features in a reasonable computational time even when usingneural network models. Moreover, the coefficient of the penalty did
not affect the computational time in our implementation.

4

CONCLUSION

In this paper, we have proposed a novel embedded feature selection
method using probabilistic model-based optimization, called thePEFS. The distribution for selecting features is updated by the natural gradient like IGO, while the model parameters are optimized
by a usual gradient descent method simultaneously. Moreover, wehave introduced the penalty based on the number of selected features. Although evolutionary algorithms have often been used forthe wrapper approach, our method has shown the potential of evolutionary algorithms for embedded feature selection. We evaluatedthe PEFS with the neural network on three classification problems
and compared the performance of our method with those of existing feature selection methods. We confirmed that introducing
the penalty can control the number of selected features. The PEFS
achieved the competitive test error and reasonable computationaltime and particularly outperformed the filter methods in the small
number of selected features.
A future work can formulate the penalty under the prior distribution. Another future work can combine the PEFS with the dynamic
model structure optimization technique introduced in [17], which
is based on the same framework used in this paper. We can simply
combine these methods and simultaneously optimize the model
structure, the selected features, and the model parameters.

ACKNOWLEDGMENT
This work is partially supported by the SECOM Science and Technology Foundation.

REFERENCES[1] Shun-ichi Amari. 1998. Natural Gradient Works Efficiently in Learning. NeuralComputation 10, 2 (1998), 251–276.
[2] Shumeet Baluja. 1994. Population based incremental learning: A method for
integrating genetic search based function optimization and competitve learning.
CMU-CS-94-163, Carnegie-Mellon Univ Pittsburgh Pa Dept Of Computer Science(1994).
[3] Aaron Defazio, Francis Bach, and Simon Lacoste-Julien. 2014. SAGA: A Fast
Incremental Gradient Method With Support for Non-Strongly Convex Composite

Objectives. In Proceedings of Advances In Neural Information Processing Systems(NIPS) 27.
[4] Isabelle Guyon and André Elisseeff. 2003. An Introduction to Variable and Feature
Selection. Journal of Machine Learning Research 3 (2003), 1157–1182.
[5] Georges R. Harik, Fernando G. Lobo, and David E. Goldberg. 1999. The compact
genetic algorithm. IEEE Transactions on Evolutionary Computation 3, 4 (1999),
287–297.
[6] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Delving Deep into
Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. In2015 IEEE International Conference on Computer Vision (ICCV). IEEE, 1026–1034.
[7] Jonathan J. Hull. 1994. A Database for Handwritten Text Recognition Research.
IEEE Transactions on Pattern Analysis and Machine Intelligence 16, 5 (1994), 550–
554.
[8] Sergey Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating
Deep Network Training by Reducing Internal Covariate Shift. In Proceedingsof the 32nd International Conference on Machine Learning (ICML), Vol. PMLR37.
448–456.
[9] Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. In Proceedings of the 3rd International Conference on Learning Representations(ICLR).
[10] Ron Kohavi. 1996. Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decisiontree Hybrid. In Proceedings of the Second International Conference on Knowledge
Discovery and Data Mining (KDD’96). AAAI Press, 202–207.
[11] Ken Lang. 1995. NewsWeeder: Learning to Filter Netnews. In Machine Learning
Proceedings 1995, Armand Prieditis and Stuart Russell (Eds.). Morgan Kaufmann,
San Francisco (CA), 331–339.
[12] Fan Li, Yiming Yang, and Eric P Xing. 2006. From Lasso regression to Feature
vector machine. Advances In Neural Information Processing Systems (NIPS) 18(2006), 779–786.
[13] Vinod Nair and Geoffrey E Hinton. 2010. Rectified Linear Units Improve Restricted Boltzmann Machines. In Proceedings of the 27th International Conference
on Machine Learning (ICML). 807–814.
[14] Yann Ollivier, Ludovic Arnold, Anne Auger, and Nikolaus Hansen. 2017.
Information-Geometric Optimization Algorithms: A Unifying Picture via Invariance Principles. Journal of Machine Learning Research 18, 18 (2017), 1–65.
[15] Hanchuan Peng, Fuhui Long, and Chris Ding. 2005. Feature selection basedon mutual information: Criteria of Max-Dependency, Max-Relevance, and MinRedundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence 27,
8 (2005), 1226–1238.
[16] Marko Robnik-Šikonja and Igor Kononenko. 2003. Theoretical and Empirical
Analysis of ReliefF and RReliefF. Machine Learning 53, 1/2 (2003), 23–69.
[17] Shinichi Shirakawa, Yasushi Iwata, and Youhei Akimoto. 2018. Dynamic Optimization of Neural Network Structures Using Probabilistic Modeling. Thirty-Second
AAAI Conference on Artificial Intelligence (AAAI-18) (2018).
[18] Robert Tibshirani. 1994. Regression shrinkage and selection via the lasso. Journalof the Royal Statistical Society, Series B 58 (1994), 267–288.
[19] Zhixiang Xu, Gao Huang, Kilian Q Weinberger, and Alice X Zheng. 2014. Gradientboosted feature selection. In Proceedings of the 20th ACM SIGKDD International
Conference on Knowledge Discovery and Data Mining (KDD ’14). ACM Press, New
York, New York, USA, 522–531.
[20] Bing Xue, Mengjie Zhang, and Will N Browne. 2013. Particle swarm optimization for feature selection in classification: A multi-objective approach. IEEE
Transactions on Cybernetics 43, 6 (2013), 1656–1671.
[21] Bing Xue, Mengjie Zhang, Will N. Browne, and Xin Yao. 2016. A Survey on
Evolutionary Computation Approaches to Feature Selection. IEEE Transactions
on Evolutionary Computation 20, 4 (2016), 606–626.
[22] Makoto Yamada, Wittawat Jitkrittum, Leonid Sigal, Eric P Xing, and Masashi
Sugiyama. 2014. High-Dimensional Feature Selection by Feature-Wise Kernelized
Lasso. Neural Computation 26, 1 (2014), 185–207.