# Ground-Based Cloud-Type Recognition Using Manifold Kernel Sparse Coding and Dictionary Learning

Authors: Qixiang Luo; Zeming Zhou; Yong Meng; Qian Li; Miaoying Li
Journal: Advances in Meteorology
 (2018)

---

## Abstract

Recognizing cloud type of ground-based images automatically has a great influence on the weather service but poses a significant challenge. Based on the symmetric positive definite (SPD) matrix manifold, a novel method named “manifold kernel sparse coding and dictionary learning” (MKSCDL) is proposed for cloud classification. Different from classical features extracted in the Euclidean space, the SPD matrix fuses multiple features and represents non-Euclidean geometric characteristics. MKSCDL is composed of three steps: feature extraction, dictionary learning, and classification. With the learned dictionary, the SPD matrix of the cloud image can be described with the sparse code. The experiments are conducted on two different ground-based cloud image datasets. Benefitting from the sparse representation on the Riemannian matrix manifold, compared to the recent baselines, experimental results demonstrate that MKSCDL possesses a more competitive performance on both grayscale and colour image datasets.


---

## Body

## 1. Introduction

Clouds play an essential role in the circulation of water vapour and affects the earth’s energy balance [1–3]. In the study of weather forecasting and climate change, clouds are always regarded as the core factor [4]. The traditional cloud observation is much dependent on the observers’ experience, and thus, it is time-consuming. The substantial and noticeable development of hardware and digital imaging techniques makes it possible to observe the cloud automatically and continuously. Compared with satellite images, the ground-based images hold a high spatial resolution at a local scale [5].Attributing to the application of sky imagers and ceilometer, automatic observation has been realized in cloud cover and cloud base height [6]. To identify the cloud type accurately and effectively, many attempts have recently been made to address this challenging issue [5–17]. Buch et al. [7] adopted texture, position, and pixel brightness features together from the whole-sky images and classified the data with decision trees. To test the texture feature extraction approach, autocorrelation, cooccurrence matrices, edge frequency, Law’s features, and primitive length were applied for cloud recognition [8]. Calbó and Sabburg [5] presented statistical texture features, Fourier transform features, and thresholded image features to identify the images taken by the whole-sky imager (WSI) and total sky imager (TSI). Heinle et al. [9] extracted 12 statistical features to represent the image’s colour and texture and then employed the k-nearest-neighbour (KNN) classifier to distinguish seven different sky conditions. As a clear distinction exists in texture orientation between the satellite images and the ground-based images, Gabor-based multiple features were utilized for classification with support vector machine (SVM) and achieved an overall accuracy of 88.3% [10]. Different from the typical local binary patterns (LBPs), weighted local binary patterns (WLBPs) [11] were presented by fusing the variance of a local patch to enhance the contrast for recognizing cloud types. Cheng and Yu [12] combined statistical features and LBPs with the Bayesian classifier to perform block-based classification (BC). In addition to texture features, Liu et al. [13] employed 7 structure features from the edge image to describe the structure characteristic of the infrared clouds. Zhuo et al. [14] indicated that simply using texture or structure features separately may not produce excellent classification performance. Hence, both texture and structure features were captured to obtain the cloud type with SVM. Furthermore, Xia et al. [6] and Xiao et al. [15] proposed to make use of multiple features together, including colour, texture, and structure features for cloud-type recognition, and the experiments validated that the integration of various features performed better than other cases. The physical features are also of great importance to represent clouds. Kazantzidis et al. [16] introduced the solar zenithal angle, the total cloud coverage, the visible percentage of sun, and the existence of rain in sky images to describe the physical property. Besides 12 image features extracted from the sky camera image, Tato et al. [17] combined another 7 cloud layer features from the ceilometer and adopted random forests for classification.To represent the cloud image more effectively, Li et al. [18] put forward a discriminative model based on a bag of microstructures (BoMS), and it showed competitive performance in the cloud-type recognition. To make up the weakness of BoMS that it could not describe these complex categories well, the duplex norm-bounded sparse representation model [19] was reported. This model demonstrated promising classification performance and was validated to be capable of capturing the most prominent patterns from complex cloud categories and naturally to attain a higher accuracy.Recently, the symmetric positive definite (SPD) matrix manifold has gained much popularity in the action recognition, object detection, face recognition, etc. [20, 21]. In addition, sparse representation on SPD matrix manifolds has been applied to these areas to achieve better performances [22, 23]. In spite of its effectiveness, the matrix manifolds method is seldom investigated to address the task of cloud classification [24].In this paper, manifold kernel sparse coding and dictionary learning (MKSCDL) on the SPD matrix manifold is proposed for ground-based cloud classification. The rest of this paper is organized as follows: Section2 introduces the dataset, and Section 3 describes the methodology of MKSCDL. Section 4 reports the experimental results and discussions. Ultimately, conclusions are summarized in Section 5.


## 2. Dataset

### 2.1. Zenithal Dataset

The zenithal dataset is provided by the National University of Defense Technology in China and acquired from historical ground-based infrared images taken by the whole-sky infrared cloud-measuring system (WSIRCMS) [25]. The images are grouped into five categories: stratiform, cumuliform, waveform, and cirriform clouds and clear sky according to the morphology and generating mechanism of the cloud [26]. This dataset is composed of 100 cloud images in each category. The typical samples with a size of 320×240 pixels from each category are demonstrated in Figure 1.Figure 1
Sample images from the zenithal dataset. (a) Stratiform clouds. (b) Cumuliform clouds. (c) Waveform clouds. (d) Cirriform clouds. (e) Clear sky.
(a)
(b)
(c)
(d)
(e)


### 2.2. SWIMCAT Dataset

The SWIMCAT dataset [27] contains 784 images taken from a daytime WSI called the wide-angle high-resolution sky-imaging system (WAHRSIS). The images are classified into 5 distinct categories: clear sky, patterned clouds, thick dark clouds, thick white clouds, and veil clouds. The number of images in per category is 224, 89, 251, 135, and 85, correspondingly. The images were obtained over a period from January 2013 to May 2014. They were selected based on visual characteristics and were categorized with the help of experts from Singapore Meteorological Services. The representative samples with a size of 125×125 pixels from each category are demonstrated in Figure 2.Figure 2
Sample images from the SWIMCAT dataset. (a) Clear sky. (b) Patterned clouds. (c) Thick dark clouds. (d) Thick white clouds. (e) Veil clouds.
(a)
(b)
(c)
(d)
(e)


## 2.1. Zenithal Dataset

The zenithal dataset is provided by the National University of Defense Technology in China and acquired from historical ground-based infrared images taken by the whole-sky infrared cloud-measuring system (WSIRCMS) [25]. The images are grouped into five categories: stratiform, cumuliform, waveform, and cirriform clouds and clear sky according to the morphology and generating mechanism of the cloud [26]. This dataset is composed of 100 cloud images in each category. The typical samples with a size of 320×240 pixels from each category are demonstrated in Figure 1.Figure 1
Sample images from the zenithal dataset. (a) Stratiform clouds. (b) Cumuliform clouds. (c) Waveform clouds. (d) Cirriform clouds. (e) Clear sky.
(a)
(b)
(c)
(d)
(e)


## 2.2. SWIMCAT Dataset

The SWIMCAT dataset [27] contains 784 images taken from a daytime WSI called the wide-angle high-resolution sky-imaging system (WAHRSIS). The images are classified into 5 distinct categories: clear sky, patterned clouds, thick dark clouds, thick white clouds, and veil clouds. The number of images in per category is 224, 89, 251, 135, and 85, correspondingly. The images were obtained over a period from January 2013 to May 2014. They were selected based on visual characteristics and were categorized with the help of experts from Singapore Meteorological Services. The representative samples with a size of 125×125 pixels from each category are demonstrated in Figure 2.Figure 2
Sample images from the SWIMCAT dataset. (a) Clear sky. (b) Patterned clouds. (c) Thick dark clouds. (d) Thick white clouds. (e) Veil clouds.
(a)
(b)
(c)
(d)
(e)


## 3. Method

In this section, the methodology is introduced mainly in three parts: feature extraction, dictionary learning, and classification, which is illustrated overall in Figure3.Figure 3
Framework of the proposed cloud classification method.


### 3.1. Feature Extraction and Stein Kernel

Given an imageI with a size of W×H, the feature image F is defined by computing d-dimensional features at every pixel:(1)Fx,y=fI,x,y,where fI,x,y is the feature mapping, for example:(2)fI,x,y=Ix,yIxIyIx2+Iy2IxxIyyT,where x,y is the pixel location; Ix,y represents the pixel gray value; Ix, Iy, Ixx, and Iyy denote the first- and second-order derivatives of Ix,y in the direction of x and y, respectively; and Ix2+Iy2 means the modulus of the gradient.The covariance descriptor (CovD) of the feature imageF is computed by the following equation:(3)C=1n−1∑u=1nfu−μfu−μT,where μ represents the mean feature vector and n is the pixel amount of the image.In general, the CovD is an SPD matrix. All SPD matrices form a Riemannian manifoldS++d when endowed with a Riemannian metric. Note that the SPD matrix is adopted as the extracted feature to describe the image; therefore, it is different from traditional features used for classification in the Euclidean space.In this paper, we adopt the Stein divergence as a Riemannian metric, and the SPD matrix manifold is mapped into the reproducing kernel Hilbert space (RKHS). The Stein divergenceS is defined as follows:(4)SA,B=logA+B2−12logAB,where A and B are the points on the SPD matrix manifold, and SA,B measures the distance between these two points.The Stein kernel is defined as follows:(5)KA,B=exp−β×SA,B.It is a positive definite kernel for certain choices ofβ>0 [28]. With the Stein kernel, we can map the SPD manifold into RKHS:(6)A⟶ϕA,B⟶ϕB,KA,B=ϕATϕB.Specifically,KA,A=ϕATϕA=1.


### 3.2. Kernel Sparse Coding and Dictionary Learning

In this section, we give a framework for manifold kernel sparse coding and dictionary learning (MKSCDL), which is outlined in Algorithm1. Let X=x1,x2,…,xN denote N points on S++d and D=d1,d2,…,dk be a dictionary with k atoms. With the Stein kernel, we update the dictionary D by two steps iteratively: kernel sparse coding and kernel dictionary learning. The model is an optimization problem with ℓ1 constraint:(7)D∗,α∗=argminD,αϕxm−ϕDα22+λα1,where xm is a sample from X, α represents the sparse coefficient, λ is a regularization parameter, and ϕxm−ϕDα22 denotes the reconstruction error.Algorithm1: Framework for MKSCDL.
Input: SPD matricesX and atom number of the dictionary k
Output: updated dictionaryD
Initialize the dictionaryD0 by k-means on Riemannian manifolds using the Frechet mean [29].
while not converge do
Kernel Sparse Coding
s
⟵
1
while not converge do
α
s
⟵
T
λ
t
s
α
s
−
1
−
t
s
∇
h
α
s
−
1
s
⟵
s
+
1
end
Kernel Dictionary Learning
for i=1 to k
p
⟵
1
while not converge do
d
i
p
⟵
d
i
p
−
1
−
t
p
grad
J
d
i
p
−
1
p
⟵
p
+
1
end
end
end
Return D=dipi=1k


#### 3.2.1. Kernel Sparse Coding

When the dictionary is fixed, the sparse coding problem is formulated as follows:(8)α∗=argminαϕxm−ϕDα22+λα1.The iterative shrinkage-thresholding algorithm (ISTA) [30] is adopted for the optimization solution.Lethα=ϕxm−ϕDα22, then the sparse vector is updated as follows:(9)αs=Tλtsαs−1−ts∇hαs−1,where ts is the step size, αs represents the sparse coefficient at the s-th iteration, and the shrinkage operator is defined as follows:(10)Tqg=g−q+signg.Equation (10) is equal to Tqg=maxg−q,0⋅∗signg.Now, the problem is transformed to calculate the gradient ofhα with respect to α:(11)hα=ϕxmTϕxm−2ϕxmTϕDα+αTϕDTϕDα.AsKxm,xm=ϕxmTϕxm=1, the gradient of hα with respect to α is(12)∇hα=−2ϕDTϕxm+2ϕDTϕDα.The first term of Equation (12) is(13)−2<ϕD,ϕxm>−2Kd1,xmKd2,xm⋮Kdk,xm.Similarly, the second term of Equation (12) is(14)2<ϕD,ϕD>α=2Kd1,d1⋯Kd1,dk⋮⋯⋮Kdk,d1⋯Kdk,dkα.As a result, the gradient∇hα is obtained by adding the right-side terms of Equations (13) and (14). The sparse code is implemented with Equation (9).


#### 3.2.2. Kernel Dictionary Learning

First, the initial dictionary is achieved byk-means on Riemannian manifolds using the Frechet mean [29]. It selects k points as the initial cluster centers from the training data randomly. Then measured by Stein divergence, every point is allocated to the closest cluster center to recompute the corresponding cluster center iteratively based on the Frechet mean via the following equation:(15)mjt+1=1w∑i=1wxi+mjt2−1−1,where xii=1w∈S++d is a set of SPD matrices and mj is the updated cluster center. Ultimately, the mjj=1k∈S++d forms the initial codebook D0.When the sparse coefficient is fixed, the dictionary is updated atom by atom, and the dictionary learning problem is formulated as follows:(16)di∗=argmindiϕxm−∑j≠i,j=1kϕdjαj−ϕdiαi22.We use ISTA [30] to update the dictionary as well. The Euclidean gradient of di is(17)∇Jdi=2βαiKxm,dixm+di−1−di−12−2βαi∑j≠i,j=1kαjKdj,didj+di−1−di−12.As proved in [22], the Riemannian gradient gradJdi is(18)gradJdi=di∇Jdidi.Thei-th atom di is updated as follows:(19)dip=dip−1−tpgradJdip−1,where tp is the step size and dip represents the i -th atom at the p-th iteration.


### 3.3. Classification

After the dictionary is learned from the training set, the sparse coefficient and the reconstruction errors “RE” of the testing set are obtained to predict the cloud type. Algorithm2 details the procedure of classification.Algorithm2: Framework for cloud-type classification.
Input: testing sample xt and learned dictionary Dii=1c
Output: predicted cloud type labelxt
for i=1 to c
Kernel Sparse Coding
α
i
∗
⟵
arg
min
α
i
ϕ
x
t
−
ϕ
D
i
α
i
2
2
+
λ
α
i
1
Computing Reconstruction Error
R
E
i
⟵
ϕ
x
t
−
ϕ
D
i
α
i
∗
2
2
end
l
a
b
e
l
x
t
⟵
arg
min
i
R
E
i
,
i
=
1,2
,
…
,
c
Return labelxtiSince there arec types of the samples, the dictionary as well has c types, which are updated independently, as detailed in Section 3.2.2.The sparse coefficientαi∗ of the testing sample xt is computed as follows:(20)αi∗=argminαiϕxt−ϕDiαi22+λαi1,i=1,2,…,c.With the sparse coefficients on each kind of dictionary, the smallest reconstruction error indicates the class that the testing sample belongs to.(21)labelxt=argminiϕxt−ϕDiαi∗22,i=1,2,…,c.In this paper, the number of cloud classesc in the two datasets is both 5, and the reconstruction errors on 5 classes of the dictionary are compared to decide the cloud type.


## 3.1. Feature Extraction and Stein Kernel

Given an imageI with a size of W×H, the feature image F is defined by computing d-dimensional features at every pixel:(1)Fx,y=fI,x,y,where fI,x,y is the feature mapping, for example:(2)fI,x,y=Ix,yIxIyIx2+Iy2IxxIyyT,where x,y is the pixel location; Ix,y represents the pixel gray value; Ix, Iy, Ixx, and Iyy denote the first- and second-order derivatives of Ix,y in the direction of x and y, respectively; and Ix2+Iy2 means the modulus of the gradient.The covariance descriptor (CovD) of the feature imageF is computed by the following equation:(3)C=1n−1∑u=1nfu−μfu−μT,where μ represents the mean feature vector and n is the pixel amount of the image.In general, the CovD is an SPD matrix. All SPD matrices form a Riemannian manifoldS++d when endowed with a Riemannian metric. Note that the SPD matrix is adopted as the extracted feature to describe the image; therefore, it is different from traditional features used for classification in the Euclidean space.In this paper, we adopt the Stein divergence as a Riemannian metric, and the SPD matrix manifold is mapped into the reproducing kernel Hilbert space (RKHS). The Stein divergenceS is defined as follows:(4)SA,B=logA+B2−12logAB,where A and B are the points on the SPD matrix manifold, and SA,B measures the distance between these two points.The Stein kernel is defined as follows:(5)KA,B=exp−β×SA,B.It is a positive definite kernel for certain choices ofβ>0 [28]. With the Stein kernel, we can map the SPD manifold into RKHS:(6)A⟶ϕA,B⟶ϕB,KA,B=ϕATϕB.Specifically,KA,A=ϕATϕA=1.


## 3.2. Kernel Sparse Coding and Dictionary Learning

In this section, we give a framework for manifold kernel sparse coding and dictionary learning (MKSCDL), which is outlined in Algorithm1. Let X=x1,x2,…,xN denote N points on S++d and D=d1,d2,…,dk be a dictionary with k atoms. With the Stein kernel, we update the dictionary D by two steps iteratively: kernel sparse coding and kernel dictionary learning. The model is an optimization problem with ℓ1 constraint:(7)D∗,α∗=argminD,αϕxm−ϕDα22+λα1,where xm is a sample from X, α represents the sparse coefficient, λ is a regularization parameter, and ϕxm−ϕDα22 denotes the reconstruction error.Algorithm1: Framework for MKSCDL.
Input: SPD matricesX and atom number of the dictionary k
Output: updated dictionaryD
Initialize the dictionaryD0 by k-means on Riemannian manifolds using the Frechet mean [29].
while not converge do
Kernel Sparse Coding
s
⟵
1
while not converge do
α
s
⟵
T
λ
t
s
α
s
−
1
−
t
s
∇
h
α
s
−
1
s
⟵
s
+
1
end
Kernel Dictionary Learning
for i=1 to k
p
⟵
1
while not converge do
d
i
p
⟵
d
i
p
−
1
−
t
p
grad
J
d
i
p
−
1
p
⟵
p
+
1
end
end
end
Return D=dipi=1k


### 3.2.1. Kernel Sparse Coding

When the dictionary is fixed, the sparse coding problem is formulated as follows:(8)α∗=argminαϕxm−ϕDα22+λα1.The iterative shrinkage-thresholding algorithm (ISTA) [30] is adopted for the optimization solution.Lethα=ϕxm−ϕDα22, then the sparse vector is updated as follows:(9)αs=Tλtsαs−1−ts∇hαs−1,where ts is the step size, αs represents the sparse coefficient at the s-th iteration, and the shrinkage operator is defined as follows:(10)Tqg=g−q+signg.Equation (10) is equal to Tqg=maxg−q,0⋅∗signg.Now, the problem is transformed to calculate the gradient ofhα with respect to α:(11)hα=ϕxmTϕxm−2ϕxmTϕDα+αTϕDTϕDα.AsKxm,xm=ϕxmTϕxm=1, the gradient of hα with respect to α is(12)∇hα=−2ϕDTϕxm+2ϕDTϕDα.The first term of Equation (12) is(13)−2<ϕD,ϕxm>−2Kd1,xmKd2,xm⋮Kdk,xm.Similarly, the second term of Equation (12) is(14)2<ϕD,ϕD>α=2Kd1,d1⋯Kd1,dk⋮⋯⋮Kdk,d1⋯Kdk,dkα.As a result, the gradient∇hα is obtained by adding the right-side terms of Equations (13) and (14). The sparse code is implemented with Equation (9).


### 3.2.2. Kernel Dictionary Learning

First, the initial dictionary is achieved byk-means on Riemannian manifolds using the Frechet mean [29]. It selects k points as the initial cluster centers from the training data randomly. Then measured by Stein divergence, every point is allocated to the closest cluster center to recompute the corresponding cluster center iteratively based on the Frechet mean via the following equation:(15)mjt+1=1w∑i=1wxi+mjt2−1−1,where xii=1w∈S++d is a set of SPD matrices and mj is the updated cluster center. Ultimately, the mjj=1k∈S++d forms the initial codebook D0.When the sparse coefficient is fixed, the dictionary is updated atom by atom, and the dictionary learning problem is formulated as follows:(16)di∗=argmindiϕxm−∑j≠i,j=1kϕdjαj−ϕdiαi22.We use ISTA [30] to update the dictionary as well. The Euclidean gradient of di is(17)∇Jdi=2βαiKxm,dixm+di−1−di−12−2βαi∑j≠i,j=1kαjKdj,didj+di−1−di−12.As proved in [22], the Riemannian gradient gradJdi is(18)gradJdi=di∇Jdidi.Thei-th atom di is updated as follows:(19)dip=dip−1−tpgradJdip−1,where tp is the step size and dip represents the i -th atom at the p-th iteration.


## 3.2.1. Kernel Sparse Coding

When the dictionary is fixed, the sparse coding problem is formulated as follows:(8)α∗=argminαϕxm−ϕDα22+λα1.The iterative shrinkage-thresholding algorithm (ISTA) [30] is adopted for the optimization solution.Lethα=ϕxm−ϕDα22, then the sparse vector is updated as follows:(9)αs=Tλtsαs−1−ts∇hαs−1,where ts is the step size, αs represents the sparse coefficient at the s-th iteration, and the shrinkage operator is defined as follows:(10)Tqg=g−q+signg.Equation (10) is equal to Tqg=maxg−q,0⋅∗signg.Now, the problem is transformed to calculate the gradient ofhα with respect to α:(11)hα=ϕxmTϕxm−2ϕxmTϕDα+αTϕDTϕDα.AsKxm,xm=ϕxmTϕxm=1, the gradient of hα with respect to α is(12)∇hα=−2ϕDTϕxm+2ϕDTϕDα.The first term of Equation (12) is(13)−2<ϕD,ϕxm>−2Kd1,xmKd2,xm⋮Kdk,xm.Similarly, the second term of Equation (12) is(14)2<ϕD,ϕD>α=2Kd1,d1⋯Kd1,dk⋮⋯⋮Kdk,d1⋯Kdk,dkα.As a result, the gradient∇hα is obtained by adding the right-side terms of Equations (13) and (14). The sparse code is implemented with Equation (9).


## 3.2.2. Kernel Dictionary Learning

First, the initial dictionary is achieved byk-means on Riemannian manifolds using the Frechet mean [29]. It selects k points as the initial cluster centers from the training data randomly. Then measured by Stein divergence, every point is allocated to the closest cluster center to recompute the corresponding cluster center iteratively based on the Frechet mean via the following equation:(15)mjt+1=1w∑i=1wxi+mjt2−1−1,where xii=1w∈S++d is a set of SPD matrices and mj is the updated cluster center. Ultimately, the mjj=1k∈S++d forms the initial codebook D0.When the sparse coefficient is fixed, the dictionary is updated atom by atom, and the dictionary learning problem is formulated as follows:(16)di∗=argmindiϕxm−∑j≠i,j=1kϕdjαj−ϕdiαi22.We use ISTA [30] to update the dictionary as well. The Euclidean gradient of di is(17)∇Jdi=2βαiKxm,dixm+di−1−di−12−2βαi∑j≠i,j=1kαjKdj,didj+di−1−di−12.As proved in [22], the Riemannian gradient gradJdi is(18)gradJdi=di∇Jdidi.Thei-th atom di is updated as follows:(19)dip=dip−1−tpgradJdip−1,where tp is the step size and dip represents the i -th atom at the p-th iteration.


## 3.3. Classification

After the dictionary is learned from the training set, the sparse coefficient and the reconstruction errors “RE” of the testing set are obtained to predict the cloud type. Algorithm2 details the procedure of classification.Algorithm2: Framework for cloud-type classification.
Input: testing sample xt and learned dictionary Dii=1c
Output: predicted cloud type labelxt
for i=1 to c
Kernel Sparse Coding
α
i
∗
⟵
arg
min
α
i
ϕ
x
t
−
ϕ
D
i
α
i
2
2
+
λ
α
i
1
Computing Reconstruction Error
R
E
i
⟵
ϕ
x
t
−
ϕ
D
i
α
i
∗
2
2
end
l
a
b
e
l
x
t
⟵
arg
min
i
R
E
i
,
i
=
1,2
,
…
,
c
Return labelxtiSince there arec types of the samples, the dictionary as well has c types, which are updated independently, as detailed in Section 3.2.2.The sparse coefficientαi∗ of the testing sample xt is computed as follows:(20)αi∗=argminαiϕxt−ϕDiαi22+λαi1,i=1,2,…,c.With the sparse coefficients on each kind of dictionary, the smallest reconstruction error indicates the class that the testing sample belongs to.(21)labelxt=argminiϕxt−ϕDiαi∗22,i=1,2,…,c.In this paper, the number of cloud classesc in the two datasets is both 5, and the reconstruction errors on 5 classes of the dictionary are compared to decide the cloud type.


## 4. Results and Discussions

In this section, compared with the baselines [11, 12, 27], the performance of MKSCDL is evaluated with the same experimental setup on two different image datasets: zenithal dataset, captured by WSIRCMS, and SWIMCAT dataset, gathered by WAHRSIS. Each experiment is implemented three times with 10-fold cross validation, and average values are treated as final results.Note that the feature mapf is defined independently on two different datasets due to the different nature between the grayscale and colour images. fI,x,y is defined in Equation (2) to the grayscale image in the zenithal dataset, while in the SWIMCAT dataset fI,x,y=fIR,x,yTfIG,x,yTfIB,x,yTT to the RGB image, where IR, IG, and IB represent the intensity images in three channels, respectively. The parameters used in our experiments are set empirically as follows: β=0.1, λ=0.01, ts=10−4, and tp=10−4.


### 4.1. Results of the Zenithal Dataset

#### 4.1.1. Performance of MKSCDL

The first experiment is carried out on the zenithal dataset. With diverse choices of the atom number in the dictionary, the performance of MKSCDL varies correspondingly. Figure4 shows the overall accuracy on the updated dictionary with different sizes. It is illustrated that when the dictionary size k equals 14, the overall accuracy achieves up to 96.33%, which outperforms the other cases.Figure 4
The overall accuracy on the updated dictionary with different sizes.Figure5 reports the confusion matrix when k=14. The element of row i and column j in the confusion matrix means the percentage of the i-th class recognized as the j-th class. As a result, the diagonal elements correspond to the recognition rates of all categories. It is illustrated all of the stratiform clouds and 99.3% clear sky images are identified correctly, which means that these two cloud types possess prominent characteristics to be distinguished. Likewise, the recognition rates of cumuliform, waveform, and cirriform clouds achieve more than 93%. On the whole, MKSCDL reveals an ideal performance in identifying the ground-based grayscale cloud images with a rather high accuracy.Figure 5
The confusion matrix (%) of MKSCDL on the zenithal dataset. The labels 1–5 refer to the corresponding cloud types: 1—stratiform clouds, 2—cumuliform clouds, 3—waveform clouds, 4—cirriform clouds, and 5—clear sky.


#### 4.1.2. Performance Comparison with the Baselines

To assess the effectiveness of MKSCDL further, WLBP [11] and BC [12] are applied for comparison:(i)
WLBP [11] is the method fusing the variance of a local patch into LBP. The KNN classifier is employed for cloud classification based on the chi-square distance.(ii)
BC [12] integrates statistical and local texture features and adopts the Bayesian classifier with regularized discriminant analysis. Note that the statistical features have only 8 dimensions because the infrared images are lack of colour information.Table1 presents the comparison results of 10-fold cross validation. The performance of MKSCDL exceeds that of the other two methods especially with regard to cumuliform, waveform, and cirriform clouds. MKSCDL achieves the highest overall accuracy of 96.33% among them. That means the dictionary is learned well and the samples are described adequately on the corresponding dictionary instead of different types of dictionary, which makes contribution to the competitive performance.Table 1
Recognition rates of different methods in 10-fold cross validation (%).
Methods
Stratiform
Cumuliform
Waveform
Cirriform
Clear sky
Overall
WLBP
100
87.84
89.41
92.51
99.06
93.56
BC
100
58.04
81.10
75.73
99.04
82.80
MKSCDL
100
93.47
95.72
93.27
99.30
96.33


### 4.2. Results of the SWIMCAT Dataset

#### 4.2.1. Performance of MKSCDL

The second experiment is conducted on the SWIMCAT dataset. Similar to the first experiment, Figure6 exhibits the overall accuracy on the learned dictionary with different sizes. As the dictionary size increases, the overall accuracy improves in general. When k is 20, MKSCDL performs best with a result of 98.34%. In consideration of excellent classification performance and computational cost of MKSCDL, we confirm that k=20 can satisfy the experimental requirement.Figure 6
The overall accuracy on the updated dictionary with different sizes.Figure7 demonstrates the confusion matrix when k=20. Patterned clouds and thick white clouds possess obvious characteristics for discrimination with an accuracy of 100%. Likewise, the results of the clear sky and thick dark clouds achieve over 98%. In addition, the challenging veil clouds resembling clear sky [27] attain a decent result of 92.94%. It is shown that the misclassification rate for each class is rather low, which means that on the whole, the proposed method works well in categorizing the ground-based RGB images.Figure 7
The confusion matrix (%) of MKSCDL on the SWIMCAT dataset. The labels 1–5 refer to the corresponding cloud types: 1—clear sky, 2—patterned clouds, 3—thick dark clouds, 4—thick white clouds, and 5—veil clouds.


#### 4.2.2. Performance Comparison with the Baselines

Besides the two baselines mentioned in Section4.1.2, the texton-based method [27] integrating both colour and texture features is used for comparison as well. Note that different from the grayscale images, the statistical features extracted from colour images in the BC method have 12 dimensions.Table2 presents the comparison results of 10-fold cross validation. By contrast, MKSCDL performs better than WLBP and BC overall. It is clear that MKSCDL has a strong power for the task of cloud categorization of the SWIMCAT dataset.Table 2
Recognition rates of different methods in 10-fold cross validation (%).
Methods
Clear sky
Patterned
Thick dark
Thick white
Veil
Overall
WLBP
88.77
100
82.56
98.51
77.27
88.50
BC
98.66
94.27
95.96
90.79
91.52
95.17
MKSCDL
98.71
100
98.50
100
92.94
98.34To compare with the texton-based approach well, we also implement the experiment with the same configuration as that in [27], which chose a training set with 40 images per category while another 45 ones for testing randomly (40/45). Table 3 lists the results. In comparison with Table 2, the overall performance of different methods changes little. The texton-based method achieves perfect classification accuracy for all categories except veil clouds, whose accuracy remains to be improved. Viewing the results of MKSCDL, each cloud type acquires a fair high recognition rate.Table 3
Recognition rates of different methods in 40/45 (%).
Methods
Clear sky
Patterned
Thick dark
Thick white
Veil
Overall
WLBP
84.44
95.56
62.22
97.78
77.78
83.56
BC
100
95.56
97.78
88.89
86.67
93.78
Texton-based
100
100
98.00
100
78.00
95.00
MKSCDL
100
100
95.56
97.78
95.56
97.78Comprehensively, in comparison with the other three methods, MKSCDL is validated to be the most effective one in recognizing the ground-based colour images.


## 4.1. Results of the Zenithal Dataset

### 4.1.1. Performance of MKSCDL

The first experiment is carried out on the zenithal dataset. With diverse choices of the atom number in the dictionary, the performance of MKSCDL varies correspondingly. Figure4 shows the overall accuracy on the updated dictionary with different sizes. It is illustrated that when the dictionary size k equals 14, the overall accuracy achieves up to 96.33%, which outperforms the other cases.Figure 4
The overall accuracy on the updated dictionary with different sizes.Figure5 reports the confusion matrix when k=14. The element of row i and column j in the confusion matrix means the percentage of the i-th class recognized as the j-th class. As a result, the diagonal elements correspond to the recognition rates of all categories. It is illustrated all of the stratiform clouds and 99.3% clear sky images are identified correctly, which means that these two cloud types possess prominent characteristics to be distinguished. Likewise, the recognition rates of cumuliform, waveform, and cirriform clouds achieve more than 93%. On the whole, MKSCDL reveals an ideal performance in identifying the ground-based grayscale cloud images with a rather high accuracy.Figure 5
The confusion matrix (%) of MKSCDL on the zenithal dataset. The labels 1–5 refer to the corresponding cloud types: 1—stratiform clouds, 2—cumuliform clouds, 3—waveform clouds, 4—cirriform clouds, and 5—clear sky.


### 4.1.2. Performance Comparison with the Baselines

To assess the effectiveness of MKSCDL further, WLBP [11] and BC [12] are applied for comparison:(i)
WLBP [11] is the method fusing the variance of a local patch into LBP. The KNN classifier is employed for cloud classification based on the chi-square distance.(ii)
BC [12] integrates statistical and local texture features and adopts the Bayesian classifier with regularized discriminant analysis. Note that the statistical features have only 8 dimensions because the infrared images are lack of colour information.Table1 presents the comparison results of 10-fold cross validation. The performance of MKSCDL exceeds that of the other two methods especially with regard to cumuliform, waveform, and cirriform clouds. MKSCDL achieves the highest overall accuracy of 96.33% among them. That means the dictionary is learned well and the samples are described adequately on the corresponding dictionary instead of different types of dictionary, which makes contribution to the competitive performance.Table 1
Recognition rates of different methods in 10-fold cross validation (%).
Methods
Stratiform
Cumuliform
Waveform
Cirriform
Clear sky
Overall
WLBP
100
87.84
89.41
92.51
99.06
93.56
BC
100
58.04
81.10
75.73
99.04
82.80
MKSCDL
100
93.47
95.72
93.27
99.30
96.33


## 4.1.1. Performance of MKSCDL

The first experiment is carried out on the zenithal dataset. With diverse choices of the atom number in the dictionary, the performance of MKSCDL varies correspondingly. Figure4 shows the overall accuracy on the updated dictionary with different sizes. It is illustrated that when the dictionary size k equals 14, the overall accuracy achieves up to 96.33%, which outperforms the other cases.Figure 4
The overall accuracy on the updated dictionary with different sizes.Figure5 reports the confusion matrix when k=14. The element of row i and column j in the confusion matrix means the percentage of the i-th class recognized as the j-th class. As a result, the diagonal elements correspond to the recognition rates of all categories. It is illustrated all of the stratiform clouds and 99.3% clear sky images are identified correctly, which means that these two cloud types possess prominent characteristics to be distinguished. Likewise, the recognition rates of cumuliform, waveform, and cirriform clouds achieve more than 93%. On the whole, MKSCDL reveals an ideal performance in identifying the ground-based grayscale cloud images with a rather high accuracy.Figure 5
The confusion matrix (%) of MKSCDL on the zenithal dataset. The labels 1–5 refer to the corresponding cloud types: 1—stratiform clouds, 2—cumuliform clouds, 3—waveform clouds, 4—cirriform clouds, and 5—clear sky.


## 4.1.2. Performance Comparison with the Baselines

To assess the effectiveness of MKSCDL further, WLBP [11] and BC [12] are applied for comparison:(i)
WLBP [11] is the method fusing the variance of a local patch into LBP. The KNN classifier is employed for cloud classification based on the chi-square distance.(ii)
BC [12] integrates statistical and local texture features and adopts the Bayesian classifier with regularized discriminant analysis. Note that the statistical features have only 8 dimensions because the infrared images are lack of colour information.Table1 presents the comparison results of 10-fold cross validation. The performance of MKSCDL exceeds that of the other two methods especially with regard to cumuliform, waveform, and cirriform clouds. MKSCDL achieves the highest overall accuracy of 96.33% among them. That means the dictionary is learned well and the samples are described adequately on the corresponding dictionary instead of different types of dictionary, which makes contribution to the competitive performance.Table 1
Recognition rates of different methods in 10-fold cross validation (%).
Methods
Stratiform
Cumuliform
Waveform
Cirriform
Clear sky
Overall
WLBP
100
87.84
89.41
92.51
99.06
93.56
BC
100
58.04
81.10
75.73
99.04
82.80
MKSCDL
100
93.47
95.72
93.27
99.30
96.33


## 4.2. Results of the SWIMCAT Dataset

### 4.2.1. Performance of MKSCDL

The second experiment is conducted on the SWIMCAT dataset. Similar to the first experiment, Figure6 exhibits the overall accuracy on the learned dictionary with different sizes. As the dictionary size increases, the overall accuracy improves in general. When k is 20, MKSCDL performs best with a result of 98.34%. In consideration of excellent classification performance and computational cost of MKSCDL, we confirm that k=20 can satisfy the experimental requirement.Figure 6
The overall accuracy on the updated dictionary with different sizes.Figure7 demonstrates the confusion matrix when k=20. Patterned clouds and thick white clouds possess obvious characteristics for discrimination with an accuracy of 100%. Likewise, the results of the clear sky and thick dark clouds achieve over 98%. In addition, the challenging veil clouds resembling clear sky [27] attain a decent result of 92.94%. It is shown that the misclassification rate for each class is rather low, which means that on the whole, the proposed method works well in categorizing the ground-based RGB images.Figure 7
The confusion matrix (%) of MKSCDL on the SWIMCAT dataset. The labels 1–5 refer to the corresponding cloud types: 1—clear sky, 2—patterned clouds, 3—thick dark clouds, 4—thick white clouds, and 5—veil clouds.


### 4.2.2. Performance Comparison with the Baselines

Besides the two baselines mentioned in Section4.1.2, the texton-based method [27] integrating both colour and texture features is used for comparison as well. Note that different from the grayscale images, the statistical features extracted from colour images in the BC method have 12 dimensions.Table2 presents the comparison results of 10-fold cross validation. By contrast, MKSCDL performs better than WLBP and BC overall. It is clear that MKSCDL has a strong power for the task of cloud categorization of the SWIMCAT dataset.Table 2
Recognition rates of different methods in 10-fold cross validation (%).
Methods
Clear sky
Patterned
Thick dark
Thick white
Veil
Overall
WLBP
88.77
100
82.56
98.51
77.27
88.50
BC
98.66
94.27
95.96
90.79
91.52
95.17
MKSCDL
98.71
100
98.50
100
92.94
98.34To compare with the texton-based approach well, we also implement the experiment with the same configuration as that in [27], which chose a training set with 40 images per category while another 45 ones for testing randomly (40/45). Table 3 lists the results. In comparison with Table 2, the overall performance of different methods changes little. The texton-based method achieves perfect classification accuracy for all categories except veil clouds, whose accuracy remains to be improved. Viewing the results of MKSCDL, each cloud type acquires a fair high recognition rate.Table 3
Recognition rates of different methods in 40/45 (%).
Methods
Clear sky
Patterned
Thick dark
Thick white
Veil
Overall
WLBP
84.44
95.56
62.22
97.78
77.78
83.56
BC
100
95.56
97.78
88.89
86.67
93.78
Texton-based
100
100
98.00
100
78.00
95.00
MKSCDL
100
100
95.56
97.78
95.56
97.78Comprehensively, in comparison with the other three methods, MKSCDL is validated to be the most effective one in recognizing the ground-based colour images.


## 4.2.1. Performance of MKSCDL

The second experiment is conducted on the SWIMCAT dataset. Similar to the first experiment, Figure6 exhibits the overall accuracy on the learned dictionary with different sizes. As the dictionary size increases, the overall accuracy improves in general. When k is 20, MKSCDL performs best with a result of 98.34%. In consideration of excellent classification performance and computational cost of MKSCDL, we confirm that k=20 can satisfy the experimental requirement.Figure 6
The overall accuracy on the updated dictionary with different sizes.Figure7 demonstrates the confusion matrix when k=20. Patterned clouds and thick white clouds possess obvious characteristics for discrimination with an accuracy of 100%. Likewise, the results of the clear sky and thick dark clouds achieve over 98%. In addition, the challenging veil clouds resembling clear sky [27] attain a decent result of 92.94%. It is shown that the misclassification rate for each class is rather low, which means that on the whole, the proposed method works well in categorizing the ground-based RGB images.Figure 7
The confusion matrix (%) of MKSCDL on the SWIMCAT dataset. The labels 1–5 refer to the corresponding cloud types: 1—clear sky, 2—patterned clouds, 3—thick dark clouds, 4—thick white clouds, and 5—veil clouds.


## 4.2.2. Performance Comparison with the Baselines

Besides the two baselines mentioned in Section4.1.2, the texton-based method [27] integrating both colour and texture features is used for comparison as well. Note that different from the grayscale images, the statistical features extracted from colour images in the BC method have 12 dimensions.Table2 presents the comparison results of 10-fold cross validation. By contrast, MKSCDL performs better than WLBP and BC overall. It is clear that MKSCDL has a strong power for the task of cloud categorization of the SWIMCAT dataset.Table 2
Recognition rates of different methods in 10-fold cross validation (%).
Methods
Clear sky
Patterned
Thick dark
Thick white
Veil
Overall
WLBP
88.77
100
82.56
98.51
77.27
88.50
BC
98.66
94.27
95.96
90.79
91.52
95.17
MKSCDL
98.71
100
98.50
100
92.94
98.34To compare with the texton-based approach well, we also implement the experiment with the same configuration as that in [27], which chose a training set with 40 images per category while another 45 ones for testing randomly (40/45). Table 3 lists the results. In comparison with Table 2, the overall performance of different methods changes little. The texton-based method achieves perfect classification accuracy for all categories except veil clouds, whose accuracy remains to be improved. Viewing the results of MKSCDL, each cloud type acquires a fair high recognition rate.Table 3
Recognition rates of different methods in 40/45 (%).
Methods
Clear sky
Patterned
Thick dark
Thick white
Veil
Overall
WLBP
84.44
95.56
62.22
97.78
77.78
83.56
BC
100
95.56
97.78
88.89
86.67
93.78
Texton-based
100
100
98.00
100
78.00
95.00
MKSCDL
100
100
95.56
97.78
95.56
97.78Comprehensively, in comparison with the other three methods, MKSCDL is validated to be the most effective one in recognizing the ground-based colour images.


## 5. Conclusions

In this paper, a novel cloud classification named “MKSCDL” on manifolds is proposed. The SPD matrix is extracted from each image and acts as the image feature. To maintain the non-Euclidean geometrical characteristics of the SPD matrix, kernel sparse coding and dictionary learning are conducted to obtain a representative dictionary. The testing sample’s reconstruction errors on different categories of dictionary are calculated to identify the specific cloud type. By comparing recent methods on grayscale and colour datasets, it is interesting to find that WLBP performs better in grayscale images, while BC performs better in colour images. Comparatively speaking, MKSCDL is suitable for both grayscale and colour images and is equipped with high capacity for automatic ground-based cloud classification.The proposed method MKSCDL can be applied to the real life, and the results provided by the visual observation are adopted for comparison and evaluating the automatic classification method. With limited images, it may not produce perfect performance in cloud-type recognition. As the dataset becomes more representative and more adequate, it will work better and can satisfy the task of cloud classification well. In the future, multiple improvements should be considered for better automatic cloud-type classification. As for the feature extraction, more features like the gray-level cooccurrence matrix and Gabor filtering could be fused into the feature mapping to describe the image well. In terms of dictionary learning, the interclass difference can be taken into consideration for a more discriminative dictionary. What is more, samples’ sparse coefficients can be applied to build the SVM model. In addition, complex sky condition existing in multiple cloud categories is supposed to attract our attention in the next work.



---

*Source: 9684206-2018-10-16.xml*