# Feature Selection and Classification for High-Dimensional Incomplete Multimodal Data

Authors: Wan-Yu Deng; Dan Liu; Ying-Ying Dong
Journal: Mathematical Problems in Engineering
 (2018)

---

## Abstract

Due to missing values, incomplete dataset is ubiquitous in multimodal scene. Complete data is a prerequisite of the most existing multimodality data fusion methods. For incomplete multimodal high-dimensional data, we propose a feature selection and classification method. Our method mainly focuses on extracting the most relevant features from the high-dimensional features and then improving the classification accuracy. The experimental results show that our method produces considerably better performance on incomplete multimodal data such as ADNI dataset and Office dataset, compared to the case of complete data.


---

## Body

## 1. Introduction

In the era of Internet, there are many different modalities, such as images, video, and text. Different modalities can provide complementary information; therefore, multimodal classification can generally produce better performance than individual modality in accuracy and reliability. The diagnoses of Alzheimer’s Disease (AD) by multimodal classification are a great example and have achieved remarkable success compared to single modal methods in multiple experiments. Pang et al. [1] explored the possibility of improving emotion prediction by highly nonlinear relationships between low-level features in different modalities. Zhang et al. [2] incorporated three modalities of biomarkers (structural MR imaging (MRI), Positron-Emission Tomography (PET), and cerebrospinal fluid (CSF)) to discriminate AD (or mild cognitive impairment (MCI)) from healthy controls. Pang et al. [3] recommended using multilabel multiple-kernel learning with visual and textual features for multilabel image classification. Hu et al. [4] utilized multimodality data including both tag feature and visual feature for popularity prediction on social media. Ballard [5] suggested a multimodal learning interface which could learn words from natural interactions with users. Liu et al. [6] mentioned a multihypergraph learning (MHL) method to deal with multimodality data. This method achieved promising results in AD/MCI classification. Zhang et al. [7] proposed multimodal multitask learning to jointly predict multiple variables from multimodal data. Liu et al. [8] proposed a linearized and kernelized sparse multitask learning for predicting cognitive outcomes in Alzheimer’s Disease. Li et al. [9, 10] proposed a multitask deep learning method for diagnosing Alzheimer’s Disease by combining MRI, PET, and Assessment Scale-Cognitive subscale (ADAS-Cog) with the restricted Boltzmann machine. Wang et al. [11] explained a novel multimodality multicenter classification method for autism spectrum disorder diagnosis; they regarded the classification of each imaging center as one task and solved the classification for all imaging centers by introducing the task-task and modality-modality regularizations. Liu et al. [12] proposed a view-aligned hypergraph learning (VAHL) method and utilized incomplete multimodality data for AD/MCI diagnosis.Complete data is a prerequisite of the most existing multimodality data fusion methods. Since complete data requires the modality type and the modality number to be consistent, it is rare in reality. In Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset, for example, only about 1/3 of its total samples contain complete MRI, PET, and CSF data at baseline. In view of incomplete multimodality data, it usually explores imputing the missing values [13, 14] and discarding samples, and doing these will lead to waste of or bring unpredictable noise. To address the incomplete multimodality data, Zhao et al. [15] proposed an unsupervised method which processes the incomplete multimodality data by transforming the original and incomplete data to a new and complete representation in a latent space. Thung et al. [16] used incomplete multimodal dataset via matrix shrinkage and completion to identify AD patients. Li et al. [17] proposed a pioneer work to handle two-modal incomplete data case by projecting the partial data into a common latent subspace via nonnegative matrix factorization (NMF) and l1 sparse regularizer. Following this line, Shao et al. [18] proposed a similar idea of weighted NMF and l2,1 regularizer.Most existing incomplete multimodality methods have low efficiency with high-dimensional data. Inspired by this, we propose a feature selection and classification for incomplete multimodal high-dimensional data. Our method has the following features: (1) It focuses on incomplete data and makes full use of the data from different modalities without data wasting. (2) It selects the most relevant features in high-dimensional space and facilitates the discovery of the inherent relationship between features. (3) It achieves better classification accuracy when compared with the other methods.The rest of the paper will demonstrate the details of the proposed approach; experiments on various datasets and comparison between our method and the currently most advanced methods.


## 2. Multimodal High-Dimensional Feature Selection and Classification

### 2.1. The Framework

Figure1 is the schematic illustration of our feature selection and classification framework for incomplete multimodal high-dimensional data. It contains three parts, polynomial kernel explicit expansion, feature selection, and classification. We start with the exploitation of the polynomial kernel explicit method to expand the original low-order feature to high-order feature. Then the identification of the high-order feature subsets by feature selection for incomplete multimodality data is achieved. Next, we classify AD patients and healthy controls. Finally, we integrate classifiers and export the final classification accuracy.Figure 1
Multimodal classification framework based on high-dimensional feature selection.


### 2.2. The Explicit Mapping

In a set of data samplesΧ=x1,…,xi,…,xN,i=1,…,N, N is the total number of samples. Each sample has m modalities; i.e., xi=xi1,xi2,…,xim. For incomplete multimodality data, we define X^=X11,⋯,XN-k11,X12,⋯,XN-k22,⋯,X1m,⋯,XN-kmm as an incomplete modal sample set. In Χm∈RN-km×dm, km, dm represent the number of missing samples and the feature dimension in the mth modalities, respectively. Y=y1,…,yi,…,yN is the corresponding label of Χ, and label yi∈L=1,…,c, and c is the number of data category. Since complete data discard a lot of precious data, the size of complete data is relatively small. As previously indicated, incomplete data is a collection of the various modal data subsets.First-order spatial selection is difficult to reveal the high-order dependency relationship between features. Since the incomplete data has limitations about the number of samples and the feature dimensions, we need more features to discover the correlation between them. We can consider using different kernel functions to extend the data, for example, a linear kernel function or Gaussian kernel function. The linear kernel function directly linearizes the data, which makes it difficult to reflect the correlation between the features and may result in the loss of data. The Gaussian kernel function is too expensive to calculate. Therefore, we transform low-dimensional features into high-dimensional features by nonlinear kernel explicit expression and then reveal high-order correlation between features.For degree-d polynomials, the polynomial kernel is defined as(1)kx,y=γxΤy+ρdwhere x and y are vectors in the input space, and ρ≥0 is a free parameter trading off the influence of higher-order versus lower-order terms in the polynomial. As a kernel, k corresponds to an inner product in a feature space based on some mapping φ:(2)kx,y=φx,φyLet d=2, and we get the special case of the quadratic kernel. After using the multinomial theorem and regrouping,(3)kx,y=∑i=1mxiyi+ρ2=∑i=1mxi2yi2+∑i=2m∑j=1i-12xixj2yiyj+∑i=1m2ρxi2ρyi+ρ2From this, the explicit feature mapping of polynomial kernel is(4)φx=xm2,…,x12,2xmxm-1,…,2xmx1,2xm-1xm-2,…,2xm-1x1,…,2x2x1,2ρxm,…,2ρx1,ρCompared with the linear case, the second-order feature map contains dependency of the feature pair. The key problem of this explicit feature mapping is that its features are high dimensional in the extended feature space. For polynomial kernel expansion, the dimension of the feature map increases exponentially. When d=2, m is the original feature dimension, and extended dimension is (m+2)(m+1)/2. Generally, when m=106, the extended dimension is approximately 1012.


### 2.3. Feature Selection and Classification

At first, we introduce a feature selection vectord∈0,1, whose entries are 1 for selected features and 0 otherwise. Let D=d∣d∈0,1 be the domain of d. We use d1≤B to control the sparsity of the feature selection, where B controls the number of selected features; then the proposed problem can be written as(5)mindminw,ξ12w22+C2∑i=1Nξi2s.t.wxi⊙d-yi=ξi,i=1,⋯,Nin which constant C is a regularization parameter that makes a trade-off between the model complexity and the fitness of the feature selection.By introducing the dual variableα, α∈A=α∣αi≥0,i=1,⋯,n, the Lagrangian function of (5) can be written as(6)Lw,ξ,α=12w22+C2∑i=1Nξi2+α,ξi-w′xi⊙d+yiin which ·,· denotes the inner product. By setting the derivatives of Lw,ξ,α with respect to w and ξ to zero, we can obtain the Karush-Kuhn-Tucker (KKT) conditions, w=αxi⊙d and ξi=-α/C. By substituting the above results into the Lagrangian function, problem (5) can be transformed into the following dual formulation:(7)mind∈Dmaxα∈Afα,dwhere fα,d=1/2∑j=1mdj2wj2+1/2Cα′α and w=∑i=1nαiyixi.Since the feature selection vector is zero-one vector, this is still a nonconvex problem. Following the convex relaxation in [19], we have(8)mind∈Dmaxα∈Afα,d≥maxα∈Amind∈Dfα,dBy introducing an additional variable θ∈R, the above problem can be converted into the following convex quadratically constrained quadratic program (QCQP) problem:(9)maxα∈A,θ∈Rθs.t.θ≤fα,d,∀d∈DIt is very hard to solve as there are infinite number of quadratic inequality constraints in (9), and we solve this problem by the cutting plane algorithm [20]. We generate an active constraint and add it to an active constraint set C which is initialized to empty set ϕ. The active constraint set C is a subset of D; i.e., C⊆D. Based on a new active constraint set C, we need to solve QCQP problem to update α. Specifically, we need to solve the following problem:(10)maxα∈A,θ∈Rθs.t.θ≤fα,dT,∀dT∈CThe cutting plane algorithm can be presented in Algorithm 1.Algorithm 1:The cutting plane algorithm.
(1) Initialize α=1 and the constraint subset C=ϕ. Let iteration indexT=1. (2) Find the most active constraint dT, update set C byC=C∪dT. (3) Update α by solving the problem (10). (4) Let T=T+1. Repeat step ⁡(2)-(3) until convergence.


#### 2.3.1. Learningd

The cutting plane algorithm mainly deals with how to find the most active constraintdT of problem (10) at the Tth iteration. Let cj=wj2, and the optimization problem becomes(11)12∑j=1mwj2dj2=12∑j=1mcjdj2Due to dj∈0,1, problem (11) can be solved by sorting cj, and then find the largest cj.


#### 2.3.2. The Optimization ofα

After updating the active constraint setC, we solve the problem in (10) with constraints which are defined by C. Because the number of constraints in C is no longer large, we can use subgradient method to solve this problem. However, it is very expensive to get the dual variables α when n is very large.In view of the above problems, problem (10) can be solved in the following primal form:(12)minw12∑t=1Twt2+gwwhere gw=C/2∑i=1Nξi2 and ξi=w′xi⊙d-yi.For convenience, we definepw=1/2∑t=1Twt2 and Fw=pw+gw. We solve the primal problem by using the accelerated proximal gradient (APG) [21], which minimizes the following quadratic approximation of (12):(13)Qτw,v=gv+∇gv,w-v+τ2w-v2+pw=τ2w-zF2+pw+gv-12τ∇gvF2in which ∇g denotes the gradient of g at point v, τ>0 denotes the Lipschitz constant, and z=v-1/τ∇gv. We need to solve the following Moreau Projection problem:(14)minwτ2w-zF2+pwProblem (14) has a unique closed-form solution, it can be solved in the following manner via Moreau Projection [22].SupposeSτz be the optimal solution to problem (14) and o=o1,o2,⋯,ok∈Rk be an intermediate variable. Then Sτz is unique and can be calculated as(15)Sτzt=otztzt,ifot≥o,0,otherwise.in which t=1,2,⋯,k. The intermediate vector ot can be calculated via a soft-threshold operator softu,ς in [22, 23](16)ot=softu,ςt=ut-ς,ifut≥ς0,Otherwise.and the threshold value ς can be calculated as in step (4) of Algorithm 2.Algorithm 2:Moreau Projection Sτz.
Given inputz=z1,z2,…,zk and s=1/τ. (1) Calculate u^=ztF for all t=1,…,k. (2) Sort u^ to obtain u such that u1≥⋯≥uk. (3) Find ρ = m a x t ∣ u t - s 1 + t s ∑ i = 1 t u i ≥ 0 ,   t = 1 , ⋯ , k . (4) Compute the threshold value ς=s/1+ρs∑i=1ρui. (5) Calculate o=softu^,ς. (6) Compute and output Sτz.The overall APG algorithm for solving problem (14) is summarized in Algorithm 3. We can obtain w from APG and then predict the results of each modality by our method. It can be expressed as y=wxi⊙d. Eventually, the integration of the prediction result will enable us to do classification.Algorithm 3:Accelerated proximal gradient for solving problem (10).
Initialization: Initialize the Lipschitz constant L t = L t - 1 and set w-1=w0 by warm start, τ 0 = L t, η∈0,1, parameter ρ-1=ρ0=1, and k = 0. (1) Set vk=wk+ρk-1-1/ρkwk-wk-1. (2) Set τ=ητk. For j=0,1,…, Set z=vk-1/τ∇gvk, compute Sτz. if FSτz≤QSτz,vk, set τk=τ, stop; else τ=minη-1τ,Lt. End end (3) Set wk+1=Sτkz. (4) Compute ρk+1=1+1+4ρk2/2. Let k = k + 1. (5) Quit if stopping condition achieves. Otherwise, go to step (1).


## 2.1. The Framework

Figure1 is the schematic illustration of our feature selection and classification framework for incomplete multimodal high-dimensional data. It contains three parts, polynomial kernel explicit expansion, feature selection, and classification. We start with the exploitation of the polynomial kernel explicit method to expand the original low-order feature to high-order feature. Then the identification of the high-order feature subsets by feature selection for incomplete multimodality data is achieved. Next, we classify AD patients and healthy controls. Finally, we integrate classifiers and export the final classification accuracy.Figure 1
Multimodal classification framework based on high-dimensional feature selection.


## 2.2. The Explicit Mapping

In a set of data samplesΧ=x1,…,xi,…,xN,i=1,…,N, N is the total number of samples. Each sample has m modalities; i.e., xi=xi1,xi2,…,xim. For incomplete multimodality data, we define X^=X11,⋯,XN-k11,X12,⋯,XN-k22,⋯,X1m,⋯,XN-kmm as an incomplete modal sample set. In Χm∈RN-km×dm, km, dm represent the number of missing samples and the feature dimension in the mth modalities, respectively. Y=y1,…,yi,…,yN is the corresponding label of Χ, and label yi∈L=1,…,c, and c is the number of data category. Since complete data discard a lot of precious data, the size of complete data is relatively small. As previously indicated, incomplete data is a collection of the various modal data subsets.First-order spatial selection is difficult to reveal the high-order dependency relationship between features. Since the incomplete data has limitations about the number of samples and the feature dimensions, we need more features to discover the correlation between them. We can consider using different kernel functions to extend the data, for example, a linear kernel function or Gaussian kernel function. The linear kernel function directly linearizes the data, which makes it difficult to reflect the correlation between the features and may result in the loss of data. The Gaussian kernel function is too expensive to calculate. Therefore, we transform low-dimensional features into high-dimensional features by nonlinear kernel explicit expression and then reveal high-order correlation between features.For degree-d polynomials, the polynomial kernel is defined as(1)kx,y=γxΤy+ρdwhere x and y are vectors in the input space, and ρ≥0 is a free parameter trading off the influence of higher-order versus lower-order terms in the polynomial. As a kernel, k corresponds to an inner product in a feature space based on some mapping φ:(2)kx,y=φx,φyLet d=2, and we get the special case of the quadratic kernel. After using the multinomial theorem and regrouping,(3)kx,y=∑i=1mxiyi+ρ2=∑i=1mxi2yi2+∑i=2m∑j=1i-12xixj2yiyj+∑i=1m2ρxi2ρyi+ρ2From this, the explicit feature mapping of polynomial kernel is(4)φx=xm2,…,x12,2xmxm-1,…,2xmx1,2xm-1xm-2,…,2xm-1x1,…,2x2x1,2ρxm,…,2ρx1,ρCompared with the linear case, the second-order feature map contains dependency of the feature pair. The key problem of this explicit feature mapping is that its features are high dimensional in the extended feature space. For polynomial kernel expansion, the dimension of the feature map increases exponentially. When d=2, m is the original feature dimension, and extended dimension is (m+2)(m+1)/2. Generally, when m=106, the extended dimension is approximately 1012.


## 2.3. Feature Selection and Classification

At first, we introduce a feature selection vectord∈0,1, whose entries are 1 for selected features and 0 otherwise. Let D=d∣d∈0,1 be the domain of d. We use d1≤B to control the sparsity of the feature selection, where B controls the number of selected features; then the proposed problem can be written as(5)mindminw,ξ12w22+C2∑i=1Nξi2s.t.wxi⊙d-yi=ξi,i=1,⋯,Nin which constant C is a regularization parameter that makes a trade-off between the model complexity and the fitness of the feature selection.By introducing the dual variableα, α∈A=α∣αi≥0,i=1,⋯,n, the Lagrangian function of (5) can be written as(6)Lw,ξ,α=12w22+C2∑i=1Nξi2+α,ξi-w′xi⊙d+yiin which ·,· denotes the inner product. By setting the derivatives of Lw,ξ,α with respect to w and ξ to zero, we can obtain the Karush-Kuhn-Tucker (KKT) conditions, w=αxi⊙d and ξi=-α/C. By substituting the above results into the Lagrangian function, problem (5) can be transformed into the following dual formulation:(7)mind∈Dmaxα∈Afα,dwhere fα,d=1/2∑j=1mdj2wj2+1/2Cα′α and w=∑i=1nαiyixi.Since the feature selection vector is zero-one vector, this is still a nonconvex problem. Following the convex relaxation in [19], we have(8)mind∈Dmaxα∈Afα,d≥maxα∈Amind∈Dfα,dBy introducing an additional variable θ∈R, the above problem can be converted into the following convex quadratically constrained quadratic program (QCQP) problem:(9)maxα∈A,θ∈Rθs.t.θ≤fα,d,∀d∈DIt is very hard to solve as there are infinite number of quadratic inequality constraints in (9), and we solve this problem by the cutting plane algorithm [20]. We generate an active constraint and add it to an active constraint set C which is initialized to empty set ϕ. The active constraint set C is a subset of D; i.e., C⊆D. Based on a new active constraint set C, we need to solve QCQP problem to update α. Specifically, we need to solve the following problem:(10)maxα∈A,θ∈Rθs.t.θ≤fα,dT,∀dT∈CThe cutting plane algorithm can be presented in Algorithm 1.Algorithm 1:The cutting plane algorithm.
(1) Initialize α=1 and the constraint subset C=ϕ. Let iteration indexT=1. (2) Find the most active constraint dT, update set C byC=C∪dT. (3) Update α by solving the problem (10). (4) Let T=T+1. Repeat step ⁡(2)-(3) until convergence.


### 2.3.1. Learningd

The cutting plane algorithm mainly deals with how to find the most active constraintdT of problem (10) at the Tth iteration. Let cj=wj2, and the optimization problem becomes(11)12∑j=1mwj2dj2=12∑j=1mcjdj2Due to dj∈0,1, problem (11) can be solved by sorting cj, and then find the largest cj.


### 2.3.2. The Optimization ofα

After updating the active constraint setC, we solve the problem in (10) with constraints which are defined by C. Because the number of constraints in C is no longer large, we can use subgradient method to solve this problem. However, it is very expensive to get the dual variables α when n is very large.In view of the above problems, problem (10) can be solved in the following primal form:(12)minw12∑t=1Twt2+gwwhere gw=C/2∑i=1Nξi2 and ξi=w′xi⊙d-yi.For convenience, we definepw=1/2∑t=1Twt2 and Fw=pw+gw. We solve the primal problem by using the accelerated proximal gradient (APG) [21], which minimizes the following quadratic approximation of (12):(13)Qτw,v=gv+∇gv,w-v+τ2w-v2+pw=τ2w-zF2+pw+gv-12τ∇gvF2in which ∇g denotes the gradient of g at point v, τ>0 denotes the Lipschitz constant, and z=v-1/τ∇gv. We need to solve the following Moreau Projection problem:(14)minwτ2w-zF2+pwProblem (14) has a unique closed-form solution, it can be solved in the following manner via Moreau Projection [22].SupposeSτz be the optimal solution to problem (14) and o=o1,o2,⋯,ok∈Rk be an intermediate variable. Then Sτz is unique and can be calculated as(15)Sτzt=otztzt,ifot≥o,0,otherwise.in which t=1,2,⋯,k. The intermediate vector ot can be calculated via a soft-threshold operator softu,ς in [22, 23](16)ot=softu,ςt=ut-ς,ifut≥ς0,Otherwise.and the threshold value ς can be calculated as in step (4) of Algorithm 2.Algorithm 2:Moreau Projection Sτz.
Given inputz=z1,z2,…,zk and s=1/τ. (1) Calculate u^=ztF for all t=1,…,k. (2) Sort u^ to obtain u such that u1≥⋯≥uk. (3) Find ρ = m a x t ∣ u t - s 1 + t s ∑ i = 1 t u i ≥ 0 ,   t = 1 , ⋯ , k . (4) Compute the threshold value ς=s/1+ρs∑i=1ρui. (5) Calculate o=softu^,ς. (6) Compute and output Sτz.The overall APG algorithm for solving problem (14) is summarized in Algorithm 3. We can obtain w from APG and then predict the results of each modality by our method. It can be expressed as y=wxi⊙d. Eventually, the integration of the prediction result will enable us to do classification.Algorithm 3:Accelerated proximal gradient for solving problem (10).
Initialization: Initialize the Lipschitz constant L t = L t - 1 and set w-1=w0 by warm start, τ 0 = L t, η∈0,1, parameter ρ-1=ρ0=1, and k = 0. (1) Set vk=wk+ρk-1-1/ρkwk-wk-1. (2) Set τ=ητk. For j=0,1,…, Set z=vk-1/τ∇gvk, compute Sτz. if FSτz≤QSτz,vk, set τk=τ, stop; else τ=minη-1τ,Lt. End end (3) Set wk+1=Sτkz. (4) Compute ρk+1=1+1+4ρk2/2. Let k = k + 1. (5) Quit if stopping condition achieves. Otherwise, go to step (1).


## 2.3.1. Learningd

The cutting plane algorithm mainly deals with how to find the most active constraintdT of problem (10) at the Tth iteration. Let cj=wj2, and the optimization problem becomes(11)12∑j=1mwj2dj2=12∑j=1mcjdj2Due to dj∈0,1, problem (11) can be solved by sorting cj, and then find the largest cj.


## 2.3.2. The Optimization ofα

After updating the active constraint setC, we solve the problem in (10) with constraints which are defined by C. Because the number of constraints in C is no longer large, we can use subgradient method to solve this problem. However, it is very expensive to get the dual variables α when n is very large.In view of the above problems, problem (10) can be solved in the following primal form:(12)minw12∑t=1Twt2+gwwhere gw=C/2∑i=1Nξi2 and ξi=w′xi⊙d-yi.For convenience, we definepw=1/2∑t=1Twt2 and Fw=pw+gw. We solve the primal problem by using the accelerated proximal gradient (APG) [21], which minimizes the following quadratic approximation of (12):(13)Qτw,v=gv+∇gv,w-v+τ2w-v2+pw=τ2w-zF2+pw+gv-12τ∇gvF2in which ∇g denotes the gradient of g at point v, τ>0 denotes the Lipschitz constant, and z=v-1/τ∇gv. We need to solve the following Moreau Projection problem:(14)minwτ2w-zF2+pwProblem (14) has a unique closed-form solution, it can be solved in the following manner via Moreau Projection [22].SupposeSτz be the optimal solution to problem (14) and o=o1,o2,⋯,ok∈Rk be an intermediate variable. Then Sτz is unique and can be calculated as(15)Sτzt=otztzt,ifot≥o,0,otherwise.in which t=1,2,⋯,k. The intermediate vector ot can be calculated via a soft-threshold operator softu,ς in [22, 23](16)ot=softu,ςt=ut-ς,ifut≥ς0,Otherwise.and the threshold value ς can be calculated as in step (4) of Algorithm 2.Algorithm 2:Moreau Projection Sτz.
Given inputz=z1,z2,…,zk and s=1/τ. (1) Calculate u^=ztF for all t=1,…,k. (2) Sort u^ to obtain u such that u1≥⋯≥uk. (3) Find ρ = m a x t ∣ u t - s 1 + t s ∑ i = 1 t u i ≥ 0 ,   t = 1 , ⋯ , k . (4) Compute the threshold value ς=s/1+ρs∑i=1ρui. (5) Calculate o=softu^,ς. (6) Compute and output Sτz.The overall APG algorithm for solving problem (14) is summarized in Algorithm 3. We can obtain w from APG and then predict the results of each modality by our method. It can be expressed as y=wxi⊙d. Eventually, the integration of the prediction result will enable us to do classification.Algorithm 3:Accelerated proximal gradient for solving problem (10).
Initialization: Initialize the Lipschitz constant L t = L t - 1 and set w-1=w0 by warm start, τ 0 = L t, η∈0,1, parameter ρ-1=ρ0=1, and k = 0. (1) Set vk=wk+ρk-1-1/ρkwk-wk-1. (2) Set τ=ητk. For j=0,1,…, Set z=vk-1/τ∇gvk, compute Sτz. if FSτz≤QSτz,vk, set τk=τ, stop; else τ=minη-1τ,Lt. End end (3) Set wk+1=Sτkz. (4) Compute ρk+1=1+1+4ρk2/2. Let k = k + 1. (5) Quit if stopping condition achieves. Otherwise, go to step (1).


## 3. Performance Evaluation

### 3.1. Datasets

We evaluate the performance of our method by employing the ADNI and Office dataset, respectively. The ADNI dataset was launched in 2003 by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, the Food and Drug Administration, private pharmaceutical companies, and nonprofit organizations. The primary purpose of ADNI project was to study the effects of combining multiple biomarkers, such as MRI, PET, and CSF data accompanied with neuropsychological assessments, to predict the progression of MCI and early AD. We employ a 3-modality (MRI, PET, and CSF) dataset with 103 subjects which include 51 AD patients and 52 healthy controls. The Mini-Mental State Examination (MMSE) is a standardized cognitive impairment examination method for screening Alzheimer’s Disease. MMSE scores between 24 and 30 and a Clinical Dementia Rating (CDR) of 0 are designated as healthy controls; MMSE scores between 20 and 26 and CDR of 0.5 or 1.0 are considered as AD. Table1 lists the demographics of all these subjects.Table 1
Subject information.
AD (n=51;18F/33M)  HC (n=52; 18F/34M) Mean SD Range Mean SD Range Age 75.2 7.4 59-88 75.3 5.2 62-85 Education 14.7 3.6 4-20 15.8 3.2 8-20 MMSE 23.8 2.0 20-26 29 1.2 25-30 CDR 0.7 0.3 0.5-1 0 0.0 0-0 The numbers refer to baseline data. AD=Alzheimer’s Disease, HC=Healthy Control, MMSE= Mini-Mental State Examination, and CDR= Clinical Dementia Rating.The multimodality data has 189 dimensionality features; for each subject, we obtain 93 features from MRI image, another 93 features from PET image, and 3 features from the CSF biomarkers. The size of feature dimension is relatively small. The nonlinear explicit expression is used to expand the dimension of data. After each item becomes one dimension, the 189 features are expanded into 8940 features. Now we can obtain the feature of a combination high-order disease.Office dataset is as follows: amazon (e.g., images downloaded from the Internet), webcam (e.g., low-resolution images captured by web cameras), and dslr (e.g., high-resolution images taken from digital SLR cameras). Each dataset has 10 object classes. Specifically, Surf and Decaf features are extracted for all the images, and Decaf-LeNet and Decaf-AlexNet represent different Decaf features by training LeNet and AlexNet model, respectively. The feature dimension of Surf is 800 and the feature dimension of Decaf by training LeNet and AlexNet model is 4096, respectively. Table2 lists the summarization of Office dataset. We expand these features into 180902 dimensional features by applying nonlinear explicit polynomial expression method.Table 2
Summarization of Office dataset.
Data Type of feature Original the dimension of feature Extended the dimension of feature The number of categories amazon Decaf-LeNet/ 4096 180902 10 dslr Surf/ 800 webcam Decaf-AlexNet 4096


### 3.2. Results on ADNI

We first use a 10-fold cross-validation strategy to classify AD and healthy controls in the single modality. We select 29 samples as training data and 10 samples as testing data from the ADNI dataset. For the purpose of the robustness and repeatability, this process is repeated 10 times to calculate the average of the classification accuracy as the final classification accuracy. The results are demonstrated in Table3. For complete data, the classification accuracy on individual modalities MRI, PET, and CSF are 83.50%, 77.50%, and 78.70%, respectively. When using MRI and PET combination, the accuracy is 83.30%. When using PET and CSF combination, the accuracy is only 81.00%. The combined measurements of all three biomarkers of MRI, PET, and CSF achieves a classification accuracy of 81.40%.Table 3
Comparison of classification accuracy of incomplete and complete multimodal data.
The type of data Modalities The size of data ACC (%) Individual modality MRI 39 83.50 PET 39 77.50 CSF 39 78.70  Two modalities MRI+PET 32 83.30 MRI+CSF 32 82.00 PET+CSF 32 81.00  Complete data MRI+CSF+PET 32 81.40  Incomplete data MRI, CSF+PET, MRI+CSF+PET 103 89.30 CSF, MRI+PET, MRI+CSF+PET 103 91.10 PET, MRI+CSF, MRI+CSF+PET 103 87.80Furthermore, due to the limitation of complete data, the size of incomplete data is larger than complete data. Specifically, our multimodal classification method for incomplete data achieves a classification accuracy of 91.10%, while the classification accuracy for complete data is only 81.40%. As we see from Table3, incomplete data demonstrates much better performance than complete data in AD and healthy controls classification. The flexibility of incomplete data is better than complete data, because it takes advantage of valuable data samples and does not lead to waste data.In Figure2, we plot classification accuracy of complete and incomplete data corresponding to different iterations. The classification accuracies of incomplete data are better than complete data.Figure 2
Classification accuracy of complete and incomplete data with respect to different iterations.As mentioned in Section2.3, B controls the sparsity of feature selection and has an important effect in the process of feature selection. In Figure 3, since different B values produce different classification accuracies when MRI is used, the classification accuracy is greatly impacted by the choice of appropriate B value. In Figure 3, when B=30, the mean of classification accuracy is higher than others. Therefore, we choose B=30. So far, our method demonstrates much better performance on incomplete data.Figure 3
Performances of our method using different B parameters.In Table4, we use incomplete multimodality data to compare the proposed method with other methods, including domain transfer support vector machine (denoted as DTSVM) [24] and multiple-kernel learning method (denoted as MKL) proposed in [2] using Lasso as feature selection. Table 4 lists the comparison of different methods for AD and HC classification.Table 4
Comparison of performance of different multimodal classification methods.
Method The type of data The size of data ACC (%) SEN (%) SPE (%) DTSVM (Cheng et al., 2015) Incomplete data 103 62.50 53.82 61.17 MKL (Zhang et al., 2011) Incomplete data 103 83.09 81.85 83.63 FSC (Proposed method) Incomplete data 103 91.10 90.00 91.38 ACC=classification accuracy, SEN=sensitivity, and SPE=specificity.Since our method uses nonlinear kernel explicit expansion and it maps features into high-dimensional features space, it is better in revealing high-order correlation between features. As we see in Table4, our method outperforms the other methods for AD and HC classification. Our method achieves the classification accuracy of 91.10% with 90.00% sensitivity and 91.38% specificity. These results further validate the efficiency of our multimodal classification method.


### 3.3. Results on Office Dataset

In this section, we evaluate our method on Office dataset which includes the following three modalities Surf, Decaf-LeNet, and Decaf-AlexNet. We start the evaluation of conducting image classification by using our method on different modalities. Then we compare classification accuracy of incomplete and complete multimodal data on amazon, dslr, and webcam, respectively. In the experiments, we expand the dimensions of feature to 180902.We test the classification performance on different datasets. Tables5–7 show comparison of classification accuracy of incomplete and complete multimodality data on amazon, dslr, and webcam, respectively. As we see in Tables 5–7, the classification performance on incomplete multimodality data is better compared to complete multimodality data. We want to emphasize that our method maps the low-order feature to the high-dimensional space, and this is helpful to discover the nonlinear related features. Incomplete data not only make the best use of the precious samples, but also utilize the inherent relation and knowledge of all modalities data.Table 5
Comparison of classification accuracy of incomplete and complete multimodality data on amazon.
The type of data Modalities The size of data ACC (%) Individual modality Decaf-LeNet 300 80.00 Surf 300 75.00 Decaf-AlexNet 300 77.50  Two modalities Surf+Decaf-LeNet 300 76.67 Surf+Decaf-AlexNet 300 73.33 Decaf-LeNet+Decaf-AlexNet 300 78.33  Complete data Surf+Decaf-LeNet+Decaf-AlexNet 224 77.50  Incomplete data Decaf-LeNet, Decaf-AlexNet+Surf,Sur+Decaf-LeNet+Decaf-AlexNet 824 94.34 Surf, Surf+Decaf-LeNet,Sur+Decaf-LeNet+Decaf-AlexNet 824 93.40 Decaf-LeNet, Surf+Decaf-LeNet Surf+Decaf-LeNet+Decaf-AlexNet 824 94.10Table 6
Comparison of classification accuracy of incomplete and complete multimodality data on dslr.
The type of data Modalities The size of data ACC (%) Individual modality Decaf-LeNet 50 72.00 Surf 50 70.00 Decaf-AlexNet 50 70.00  Two modalities Surf+Decaf-LeNet 54 61.90 Surf+Decaf-AlexNet 54 61.90 Decaf-LeNet+Decaf-AlexNet 54 71.43  Complete data Surf+Decaf-LeNet+Decaf-AlexNet 50 80.00  Incomplete data Surf, Surf+Decaf-AlexNet,Surf+Decaf-LeNet+Decaf-AlexNet 154 93.25 Decaf-LeNet, Surf+Decaf-LeNet,Sur+Decaf-LeNet+Decaf-AlexNet 154 92.98 Decaf-LeNet, Decaf-AlexNet+Decaf-LeNet,Sur+Decaf-LeNet+Decaf-AlexNet 154 91.23Table 7
Comparison of classification accuracy of incomplete and complete multimodality data on webcam.
The type of data Modalities The size of data ACC(%) Individual modality Decaf-LeNet 85 83.33 Surf 85 83.33 Decaf-AlexNet 85 76.67  Two modalities Surf+Decaf-LeNet 70 88.00 Surf+Decaf-AlexNet 70 85.00 Decaf-LeNet+Decaf-AlexNet 70 85.00  Complete data Surf+Decaf-LeNet+Decaf-AlexNet 80 93.33  Incomplete data Surf, Surf+Decaf-AlexNet,Surf+Decaf-LeNet+Decaf-AlexNet 235 96.08 Decaf-LeNet, Surf+Decaf-LeNet,Sur+Decaf-LeNet+Decaf-AlexNet 235 95.67 Decaf-AlexNe, Surf+Decaf-LeNet,Sur+Decaf-LeNet+Decaf-AlexNet 235 95.93


## 3.1. Datasets

We evaluate the performance of our method by employing the ADNI and Office dataset, respectively. The ADNI dataset was launched in 2003 by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering, the Food and Drug Administration, private pharmaceutical companies, and nonprofit organizations. The primary purpose of ADNI project was to study the effects of combining multiple biomarkers, such as MRI, PET, and CSF data accompanied with neuropsychological assessments, to predict the progression of MCI and early AD. We employ a 3-modality (MRI, PET, and CSF) dataset with 103 subjects which include 51 AD patients and 52 healthy controls. The Mini-Mental State Examination (MMSE) is a standardized cognitive impairment examination method for screening Alzheimer’s Disease. MMSE scores between 24 and 30 and a Clinical Dementia Rating (CDR) of 0 are designated as healthy controls; MMSE scores between 20 and 26 and CDR of 0.5 or 1.0 are considered as AD. Table1 lists the demographics of all these subjects.Table 1
Subject information.
AD (n=51;18F/33M)  HC (n=52; 18F/34M) Mean SD Range Mean SD Range Age 75.2 7.4 59-88 75.3 5.2 62-85 Education 14.7 3.6 4-20 15.8 3.2 8-20 MMSE 23.8 2.0 20-26 29 1.2 25-30 CDR 0.7 0.3 0.5-1 0 0.0 0-0 The numbers refer to baseline data. AD=Alzheimer’s Disease, HC=Healthy Control, MMSE= Mini-Mental State Examination, and CDR= Clinical Dementia Rating.The multimodality data has 189 dimensionality features; for each subject, we obtain 93 features from MRI image, another 93 features from PET image, and 3 features from the CSF biomarkers. The size of feature dimension is relatively small. The nonlinear explicit expression is used to expand the dimension of data. After each item becomes one dimension, the 189 features are expanded into 8940 features. Now we can obtain the feature of a combination high-order disease.Office dataset is as follows: amazon (e.g., images downloaded from the Internet), webcam (e.g., low-resolution images captured by web cameras), and dslr (e.g., high-resolution images taken from digital SLR cameras). Each dataset has 10 object classes. Specifically, Surf and Decaf features are extracted for all the images, and Decaf-LeNet and Decaf-AlexNet represent different Decaf features by training LeNet and AlexNet model, respectively. The feature dimension of Surf is 800 and the feature dimension of Decaf by training LeNet and AlexNet model is 4096, respectively. Table2 lists the summarization of Office dataset. We expand these features into 180902 dimensional features by applying nonlinear explicit polynomial expression method.Table 2
Summarization of Office dataset.
Data Type of feature Original the dimension of feature Extended the dimension of feature The number of categories amazon Decaf-LeNet/ 4096 180902 10 dslr Surf/ 800 webcam Decaf-AlexNet 4096


## 3.2. Results on ADNI

We first use a 10-fold cross-validation strategy to classify AD and healthy controls in the single modality. We select 29 samples as training data and 10 samples as testing data from the ADNI dataset. For the purpose of the robustness and repeatability, this process is repeated 10 times to calculate the average of the classification accuracy as the final classification accuracy. The results are demonstrated in Table3. For complete data, the classification accuracy on individual modalities MRI, PET, and CSF are 83.50%, 77.50%, and 78.70%, respectively. When using MRI and PET combination, the accuracy is 83.30%. When using PET and CSF combination, the accuracy is only 81.00%. The combined measurements of all three biomarkers of MRI, PET, and CSF achieves a classification accuracy of 81.40%.Table 3
Comparison of classification accuracy of incomplete and complete multimodal data.
The type of data Modalities The size of data ACC (%) Individual modality MRI 39 83.50 PET 39 77.50 CSF 39 78.70  Two modalities MRI+PET 32 83.30 MRI+CSF 32 82.00 PET+CSF 32 81.00  Complete data MRI+CSF+PET 32 81.40  Incomplete data MRI, CSF+PET, MRI+CSF+PET 103 89.30 CSF, MRI+PET, MRI+CSF+PET 103 91.10 PET, MRI+CSF, MRI+CSF+PET 103 87.80Furthermore, due to the limitation of complete data, the size of incomplete data is larger than complete data. Specifically, our multimodal classification method for incomplete data achieves a classification accuracy of 91.10%, while the classification accuracy for complete data is only 81.40%. As we see from Table3, incomplete data demonstrates much better performance than complete data in AD and healthy controls classification. The flexibility of incomplete data is better than complete data, because it takes advantage of valuable data samples and does not lead to waste data.In Figure2, we plot classification accuracy of complete and incomplete data corresponding to different iterations. The classification accuracies of incomplete data are better than complete data.Figure 2
Classification accuracy of complete and incomplete data with respect to different iterations.As mentioned in Section2.3, B controls the sparsity of feature selection and has an important effect in the process of feature selection. In Figure 3, since different B values produce different classification accuracies when MRI is used, the classification accuracy is greatly impacted by the choice of appropriate B value. In Figure 3, when B=30, the mean of classification accuracy is higher than others. Therefore, we choose B=30. So far, our method demonstrates much better performance on incomplete data.Figure 3
Performances of our method using different B parameters.In Table4, we use incomplete multimodality data to compare the proposed method with other methods, including domain transfer support vector machine (denoted as DTSVM) [24] and multiple-kernel learning method (denoted as MKL) proposed in [2] using Lasso as feature selection. Table 4 lists the comparison of different methods for AD and HC classification.Table 4
Comparison of performance of different multimodal classification methods.
Method The type of data The size of data ACC (%) SEN (%) SPE (%) DTSVM (Cheng et al., 2015) Incomplete data 103 62.50 53.82 61.17 MKL (Zhang et al., 2011) Incomplete data 103 83.09 81.85 83.63 FSC (Proposed method) Incomplete data 103 91.10 90.00 91.38 ACC=classification accuracy, SEN=sensitivity, and SPE=specificity.Since our method uses nonlinear kernel explicit expansion and it maps features into high-dimensional features space, it is better in revealing high-order correlation between features. As we see in Table4, our method outperforms the other methods for AD and HC classification. Our method achieves the classification accuracy of 91.10% with 90.00% sensitivity and 91.38% specificity. These results further validate the efficiency of our multimodal classification method.


## 3.3. Results on Office Dataset

In this section, we evaluate our method on Office dataset which includes the following three modalities Surf, Decaf-LeNet, and Decaf-AlexNet. We start the evaluation of conducting image classification by using our method on different modalities. Then we compare classification accuracy of incomplete and complete multimodal data on amazon, dslr, and webcam, respectively. In the experiments, we expand the dimensions of feature to 180902.We test the classification performance on different datasets. Tables5–7 show comparison of classification accuracy of incomplete and complete multimodality data on amazon, dslr, and webcam, respectively. As we see in Tables 5–7, the classification performance on incomplete multimodality data is better compared to complete multimodality data. We want to emphasize that our method maps the low-order feature to the high-dimensional space, and this is helpful to discover the nonlinear related features. Incomplete data not only make the best use of the precious samples, but also utilize the inherent relation and knowledge of all modalities data.Table 5
Comparison of classification accuracy of incomplete and complete multimodality data on amazon.
The type of data Modalities The size of data ACC (%) Individual modality Decaf-LeNet 300 80.00 Surf 300 75.00 Decaf-AlexNet 300 77.50  Two modalities Surf+Decaf-LeNet 300 76.67 Surf+Decaf-AlexNet 300 73.33 Decaf-LeNet+Decaf-AlexNet 300 78.33  Complete data Surf+Decaf-LeNet+Decaf-AlexNet 224 77.50  Incomplete data Decaf-LeNet, Decaf-AlexNet+Surf,Sur+Decaf-LeNet+Decaf-AlexNet 824 94.34 Surf, Surf+Decaf-LeNet,Sur+Decaf-LeNet+Decaf-AlexNet 824 93.40 Decaf-LeNet, Surf+Decaf-LeNet Surf+Decaf-LeNet+Decaf-AlexNet 824 94.10Table 6
Comparison of classification accuracy of incomplete and complete multimodality data on dslr.
The type of data Modalities The size of data ACC (%) Individual modality Decaf-LeNet 50 72.00 Surf 50 70.00 Decaf-AlexNet 50 70.00  Two modalities Surf+Decaf-LeNet 54 61.90 Surf+Decaf-AlexNet 54 61.90 Decaf-LeNet+Decaf-AlexNet 54 71.43  Complete data Surf+Decaf-LeNet+Decaf-AlexNet 50 80.00  Incomplete data Surf, Surf+Decaf-AlexNet,Surf+Decaf-LeNet+Decaf-AlexNet 154 93.25 Decaf-LeNet, Surf+Decaf-LeNet,Sur+Decaf-LeNet+Decaf-AlexNet 154 92.98 Decaf-LeNet, Decaf-AlexNet+Decaf-LeNet,Sur+Decaf-LeNet+Decaf-AlexNet 154 91.23Table 7
Comparison of classification accuracy of incomplete and complete multimodality data on webcam.
The type of data Modalities The size of data ACC(%) Individual modality Decaf-LeNet 85 83.33 Surf 85 83.33 Decaf-AlexNet 85 76.67  Two modalities Surf+Decaf-LeNet 70 88.00 Surf+Decaf-AlexNet 70 85.00 Decaf-LeNet+Decaf-AlexNet 70 85.00  Complete data Surf+Decaf-LeNet+Decaf-AlexNet 80 93.33  Incomplete data Surf, Surf+Decaf-AlexNet,Surf+Decaf-LeNet+Decaf-AlexNet 235 96.08 Decaf-LeNet, Surf+Decaf-LeNet,Sur+Decaf-LeNet+Decaf-AlexNet 235 95.67 Decaf-AlexNe, Surf+Decaf-LeNet,Sur+Decaf-LeNet+Decaf-AlexNet 235 95.93


## 4. Conclusion

Authors proposed a feature selection and classification method for incomplete multimodal high-dimensional data. Our algorithm produces considerably better classification performance. The flexibility of incomplete data is better than complete data. Our method takes advantage of valuable data samples and does not lead to waste data. In addition, our method focuses on extracting the relevant features from incomplete multimodal data feature space. We use ADNI and Office dataset to verify the performance of our method. The results show that our method is better than other state-of-the-art methods. The limitation of this study is that it only considers simple model, and we will extend our method in more complex model in the future.



---

*Source: 1583969-2018-08-12.xml*