# Research on Chest Disease Recognition Based on Deep Hierarchical Learning Algorithm

Authors: Lingling Li; Yangyang Long; Bangtong Huang; Zihong Chen; Zheng Liu; Zekun Yang
Journal: Journal of Healthcare Engineering
 (2022)

---

## Abstract

Chest X-ray has become one of the most common ways in diagnostic radiology exams, and this technology assists expert radiologists with finding the patients at potential risk of cardiopathy and lung diseases. However, it is still a challenge for expert radiologists to assess thousands of cases in a short period so that deep learning methods are introduced to tackle this problem. Since the diseases have correlations with each other and have hierarchical features, the traditional classification scheme could not achieve a good performance. In order to extract the correlation features among the diseases, some GCN-based models are introduced to combine the features extracted from the images to make prediction. This scheme can work well with the high quality of image features, so backbone with high computation cost plays a vital role in this scheme. However, a fast prediction in diagnostic radiology is also needed especially in case of emergency or region with low computation facilities, so we proposed an efficient convolutional neural network with GCN, which is named SGGCN, to meet the need of efficient computation and considerable accuracy. SGGCN used SGNet-101 as backbone, which is built by ShuffleGhost Block (Huang et al., 2021) to extract features with a low computation cost. In order to make sufficient usage of the information in GCN, a new GCN architecture is designed to combine information from different layers together in GCNM module so that we can utilize various hierarchical features and meanwhile make the GCN scheme faster. The experiment on CheXPert datasets illustrated that SGGCN achieves a considerable performance. Compared with GCN and ResNet-101 (He et al., 2015) backbone (test AUC 0.8080, parameters 4.7M and FLOPs 16.0B), the SGGCN achieves 0.7831 (−3.08%) test AUC with parameters 1.2M (−73.73%) and FLOPs 3.1B (−80.82%), where GCN with MobileNet (Sandler and Howard, 2018) backbone achieves 0.7531 (−6.79%) test AUC with parameters 0.5M (−88.46%) and FLOPs 0.66B (−95.88%).


---

## Body

## 1. Introduction

A potential risk of cardiopathy and lung disease threatens millions of lives, and most of these diseases are preventable due to the chest X-ray (CXR) technology. Now, CXR technology becomes a regular examination of heart and lung disease, which assists in clinical diagnosis and treatment. Some algorithms like convolutional neural network (CNN) and Bayesian models are introduced to process and make diseases prediction by CXR images, and they really make a difference. On the one hand, they reduce the workload of expert radiologists with the high speed of computation and make it possible for expert radiologists to process a huge number of radiology samples. On the other hand, these algorithms can filter out some low-risk radiology samples with a considerably low-false-negative rate so that expert radiologists can more easily find out the samples with potential risk.CNN-based models can extract the features from images and use a fully connected layers to make prediction. Comparing to multi-class image classification [1], the multilabel task is more challenging due to the combinatorial nature of the output space. With the advent of deep learning, a more recent focus has been on adapting deep networks, typically convolutional neural networks (CNNs), for hierarchical classification [2, 3]. ResNet [4] was proposed to extract features with a deep convolutional network and improved the accuracy of ImageNet classification task. And now, ResNet is wildly used as a backbone to extract features, as well as pretrained model is adopted to accelerate the training procedure. But chest disease recognition task is a multilabel classification task, and the label (diseases) has hierarchical features, so the trick in classical image classification task might not work, if the hierarchical features are not properly extracted. Given the outstanding performance, deep learning has been applied in some safety and security critical tasks, such as self-driving, malware detection, identification [5], and anomaly detection [6].In some previous work, Graph Convolution Network (GCN) [7] is introduced to learn the hierarchical features among the labels, and this kind of structure might be suitable for this chest disease recognition task. And works like MLGCN [8] designed a proper structure, utilized the hierarchical features of labels, and achieved a better performance, but most of them adopt a deep neural network like ResNet-101 as backbone to extract image features, which would suffer high cost of computation. In this work, we focus on the efficient computation in GCN. In order to decrease the parameters and FLOPs, firstly we designed a new backbone named SGNet-101, which is built by ShuffleGhost [9] block. The SGNet-101 utilized the redundancy of feature map in convolution and used ghost convolution to simulate the convolution scheme. Compared with light models which have wide usage of depthwise and elementwise convolution, SGNet-101 could reduce the FLOPs and parameters and maintain the image features more easily. In order to make sufficient usage of the information in GCN, we designed a new GCN architecture to combine information from different layers together so that we can utilize various hierarchical features and meanwhile make the GCN scheme faster. With the SGNet-101 as backbone and new GCN architecture, a new model named SGGCN is proposed by us.


## 2. Related Work

With the development of deep learning, researchers have achieved great performance in image classification tasks and made good efforts in medical image classification and segmentation. In the chest disease recognition task, the diseases share co-occurrence features and have hierarchical structures, so special techniques should be adopted to tackle this hierarchical multilabel learning classification task. ChestX-ray14 dataset [10] and CheXpert [11] dataset with hierarchical multilabel features have been widely used, as well as some methods with probability modelling, attention learning, and graph neural network are also introduced to learn the hierarchical features. Chen et al. [12] mainly focused on probability modelling and tried to predict the conditional probability for each label and fined-tuning this model with unconditional probability. Guan and Huang [13] used ResNet-50 or DenseNet-121 as the backbone, designed an attention module to obtain normalized attention scores, and integrated the features from backbone and the attention scores into a residual attention block to make classifications. In order to utilize the co-occurrence features in the datasets MS-COCO [14] and VOC2007, Chen and Wei et al. [8] used graph convolution network to capture the correlations of the labels and applied these features on the features extracted from input images by ResNet-101. Chen and Li et al. [15] further applied this graph convolution network method on multilabel cheset X-ray image classification and proposed CheXGCN, which achieved considerable results on Chest X-ray14 and CheXpert.


## 3. Methods

### 3.1. Word Embedding

GloVe [16] word embedding is adopted to convert label words into vectors so that this vector can take the place of the one-hot encoding. Our method used 300-dim word vectors from GloVe text model which trained on the Wikipedia dataset to convert the labels in the CheXpert dataset into vectors so that it produced a 14×300 matrix, and this matrix would further be fed into graph convolution network, which is regarded as Graph Convolution Network Module (GCNM) in SGGCN that we proposed.


### 3.2. Unbalanced Learning

As will be mentioned in Section5.1, CheXpert datasets have unbalanced the data. The Fracture class have the least samples of 7270 with 484 uncertain, while the Lung Opacity has the largest samples of 92669 with 4341 uncertain. In order to tackle the imbalance of dataset, we adopted Weighted Cross Entropy Loss, which is proposed in CheXGCN:(1)Lpi,li=−ωp∑li=1logσpi−ωn∑li=0log1−σpi,ωp=P+N+1P+1,ωn=P+N+1N+1,where σ is the sigmoid function and P and N are the number of positive samples and negative samples. In SGGCN, we computed P and N as the positive samples and negative samples in the whole training set to improve the stability.


### 3.3. Graph Neural Network

#### 3.3.1. Fourier Transform

When given a periodic functionf (x), we can break it apart by Fourier series:(2)fx=α02⋅1+∑n=1+∞αnsinnωx+∑n=1+∞bncosnωx,an=∫0Tfxsinnωxdx∫0Tsin2nωxdx,bn=∫0Tfxcosnωxdx∫0Tcos2nωxdx.It can be rewritten in a complex formula:(3)fx=∑n=−∞+∞cnei2πnxT=∑t=−∞+∞cteiωt.It is noteworthy to mention that we can takeeiωt as orthonormal set and take ct as the coordinate.If we want to convert a nonperiodic function into Fourier series, we could regard it as aT=∞ periodic function and use Fourier transform:(4)FTω=∫−∞+∞ft−iωtdt.When givenω, it used e−iωt to decompose ft and get the coordinate of eiωt. And the inverse Fourier transform is(5)ft=∫−∞+∞Fweiωtdw.


#### 3.3.2. Graph Laplacian

When we consider Laplace operator in images, it can be defined by the sum of second derivative for the nearest four dimensions:(6)Δf=∑i=1n∂2f∂xi2.If Laplace operator is moved into an undirected graph structure withN nodes, the Laplace operator of each node might be different due to the different relations and connections. The Laplace operator of node i should be defined as follows:(7)Δfi=∑j∂2f∂j2=∑jωijfi−fj=∑jωijfi−∑jωijfj=difi−ωif,where fi is the function value of node i, j are the nodes connected with i, ωij is the weight of i−j connection, di is the degree of i, and ωif is the sum of multiplication of all j and its weight. It can be rewritten in matrix form as follows:(8)Δf=d1…0⋮⋱⋮0⋯dNf1⋮fN−−ωi⋅−⋮−ωN⋅−f1⋮fN=D−WF=LF.And, we get the Laplacian matrixL, and we further get the normalized Laplacian matrix L∼.(9)L∼=D−1/2D−WD−1/2.The decomposition of Laplacian matrixL is(10)Luk=λuk.


#### 3.3.3. Graph Fourier Transform

It can be proved by Helmholtz equation thatuk can be used as orthonormal set to decompose fi:(11)Fkλk=f^k=∑i=1Nfiuki,where λk and uk are the eigenvalues and eigenvectors of Laplacian matrix L, and k=1,…,N because L is an N×N symmetric matrix. It can be rewritten in matrix form:(12)f^=f^1⋮f^N=u11…u1n⋮⋱⋮uN1⋯uNNf1⋮fN=UTf.And, the inverse Fourier transform is(13)f=Uf^.


#### 3.3.4. Graph Convolution Network

According to convolution theorem, the Fourier transform of a convolution of two signals is the pointwise product of their Fourier Transforms under suitable conditions:(14)Ff∗g=Ff⋅Fg,where F is the Fourier transform, f and g are two signals, ∗ is the convolution operation, and ⋅ is the pointwise product. When applied in graph G, with input f and kernel h, convolution operation in graph can convert to pointwise product under Fourier domain:(15)f∗gG=F−1Ff⋅Fh=F−1UTf⋅h^=Uh^1…0⋮⋱⋮0⋯hN∧UTf=Uh^1…0⋮⋱⋮0⋯hN∧UTf=Uθ1…0⋮⋱⋮0⋯θNUTf.The trainable variablesg convert into θ in Fourier domain. And in graph neural network, we can directly learn θ instead of g. We also get the following formula, where σ is the activation function:(16)y=σUgθUTx=σUθ1⋱θNUTx.Here, we have defined the propagation rule of graph network. But this rule has some drawbacks: (1)N might be a large number, which would be due to large trainable parameters; (2) it is hard to share weight θi in θ; (3) U is computed from the decomposition of L, whose computation cost is ON3. In order to tackle these problems, gθ could be rewritten as a function gθ⋅ in the following formula:(17)θ1⋱θN=gθ=gθΛ,Λ=λ1⋱λN.And Taylor series expansion is adopted to approximategθ.(18)gθΛ≈∑k=0K−1θkΛk.This approximation takes the place ofgθΛ, and we rewrite equation (16):(19)y=σUgθUTx=σUgθΛUTx≈σU∑k=0K−1θkΛkUTx=σ∑k=0K−1θkUΛkUTx=σ∑k=0K−1θkUΛUTkx=σ∑k=0K−1θkLkx.So here, we avoid the computation of decomposition ofL, but Lk still suffers high computation cost. And Chebyshev polynomials are adopted to approximate Lk:(20)Lk≈TkL˜,L˜=2λmaxL−IN,TkL=1,k=0,L,k=1,2LTK−1L+Tk−2L,k≥2.And, equation (19) can be rewritten as follows:(21)y≈σ∑k=0K−1θkTkL˜x.Ifk is set as 2, we get the following formula:(22)gθ∗x≈∑k=01θkTkL˜x=θ0x+θ1L˜x.Sinceθ0 and θ1 influence the scale, it would be less effective after operation of normalization, so they can be set equal: θ=θ0=θ1, and equation (22) can be rewritten as follows:(23)gθ∗x≈θ0x+θ1L˜x=θIN+L˜x.And normalizing the matrixA=IN+L˜, we get(24)A=IN+L˜,A=D˜−1/2AD˜−1/2.In order to learn the relations, weightW is introduced, and a new propagation rule can be obtained:(25)Hl+1=σD˜−1/2AD˜−1/2HlWl,(26)D˜ii=∑jAij,(27)A=IN+L˜,(28)L˜=2λmaxL−IN,where Hl is the output from layer l and Wl is the trainable variables in layer l. And the propagation rule in the graph convolution layer is(29)Hl+1=σA˜HlWl=σD˜−1/2AD˜−1/2HlWl.


### 3.4. Graph Presentation

In order to follow the propagation rule of equation (29), we should compute correlation matrix A. The way to compute A mentioned in equation (27) cannot work, because in this task, the graph is a weighted, directed graph.We adopt the method introduced in ChexGCN, which used a nonlinear method to preprocess the correlation matrixp by equation (34) to reduce the noise and protect the correlations of labels:(30)Aij=λpij∑i=0CPij+θ,ifPij≥ϕ,0,ifPij<ϕ,where λ is a hyperparameter to control the correlation state between the node and its neighborhood, ϕ is the threshold to filter the noise, and θ is an innately small quantity to ensure the denominator is not equal to zero.


## 3.1. Word Embedding

GloVe [16] word embedding is adopted to convert label words into vectors so that this vector can take the place of the one-hot encoding. Our method used 300-dim word vectors from GloVe text model which trained on the Wikipedia dataset to convert the labels in the CheXpert dataset into vectors so that it produced a 14×300 matrix, and this matrix would further be fed into graph convolution network, which is regarded as Graph Convolution Network Module (GCNM) in SGGCN that we proposed.


## 3.2. Unbalanced Learning

As will be mentioned in Section5.1, CheXpert datasets have unbalanced the data. The Fracture class have the least samples of 7270 with 484 uncertain, while the Lung Opacity has the largest samples of 92669 with 4341 uncertain. In order to tackle the imbalance of dataset, we adopted Weighted Cross Entropy Loss, which is proposed in CheXGCN:(1)Lpi,li=−ωp∑li=1logσpi−ωn∑li=0log1−σpi,ωp=P+N+1P+1,ωn=P+N+1N+1,where σ is the sigmoid function and P and N are the number of positive samples and negative samples. In SGGCN, we computed P and N as the positive samples and negative samples in the whole training set to improve the stability.


## 3.3. Graph Neural Network

### 3.3.1. Fourier Transform

When given a periodic functionf (x), we can break it apart by Fourier series:(2)fx=α02⋅1+∑n=1+∞αnsinnωx+∑n=1+∞bncosnωx,an=∫0Tfxsinnωxdx∫0Tsin2nωxdx,bn=∫0Tfxcosnωxdx∫0Tcos2nωxdx.It can be rewritten in a complex formula:(3)fx=∑n=−∞+∞cnei2πnxT=∑t=−∞+∞cteiωt.It is noteworthy to mention that we can takeeiωt as orthonormal set and take ct as the coordinate.If we want to convert a nonperiodic function into Fourier series, we could regard it as aT=∞ periodic function and use Fourier transform:(4)FTω=∫−∞+∞ft−iωtdt.When givenω, it used e−iωt to decompose ft and get the coordinate of eiωt. And the inverse Fourier transform is(5)ft=∫−∞+∞Fweiωtdw.


### 3.3.2. Graph Laplacian

When we consider Laplace operator in images, it can be defined by the sum of second derivative for the nearest four dimensions:(6)Δf=∑i=1n∂2f∂xi2.If Laplace operator is moved into an undirected graph structure withN nodes, the Laplace operator of each node might be different due to the different relations and connections. The Laplace operator of node i should be defined as follows:(7)Δfi=∑j∂2f∂j2=∑jωijfi−fj=∑jωijfi−∑jωijfj=difi−ωif,where fi is the function value of node i, j are the nodes connected with i, ωij is the weight of i−j connection, di is the degree of i, and ωif is the sum of multiplication of all j and its weight. It can be rewritten in matrix form as follows:(8)Δf=d1…0⋮⋱⋮0⋯dNf1⋮fN−−ωi⋅−⋮−ωN⋅−f1⋮fN=D−WF=LF.And, we get the Laplacian matrixL, and we further get the normalized Laplacian matrix L∼.(9)L∼=D−1/2D−WD−1/2.The decomposition of Laplacian matrixL is(10)Luk=λuk.


### 3.3.3. Graph Fourier Transform

It can be proved by Helmholtz equation thatuk can be used as orthonormal set to decompose fi:(11)Fkλk=f^k=∑i=1Nfiuki,where λk and uk are the eigenvalues and eigenvectors of Laplacian matrix L, and k=1,…,N because L is an N×N symmetric matrix. It can be rewritten in matrix form:(12)f^=f^1⋮f^N=u11…u1n⋮⋱⋮uN1⋯uNNf1⋮fN=UTf.And, the inverse Fourier transform is(13)f=Uf^.


### 3.3.4. Graph Convolution Network

According to convolution theorem, the Fourier transform of a convolution of two signals is the pointwise product of their Fourier Transforms under suitable conditions:(14)Ff∗g=Ff⋅Fg,where F is the Fourier transform, f and g are two signals, ∗ is the convolution operation, and ⋅ is the pointwise product. When applied in graph G, with input f and kernel h, convolution operation in graph can convert to pointwise product under Fourier domain:(15)f∗gG=F−1Ff⋅Fh=F−1UTf⋅h^=Uh^1…0⋮⋱⋮0⋯hN∧UTf=Uh^1…0⋮⋱⋮0⋯hN∧UTf=Uθ1…0⋮⋱⋮0⋯θNUTf.The trainable variablesg convert into θ in Fourier domain. And in graph neural network, we can directly learn θ instead of g. We also get the following formula, where σ is the activation function:(16)y=σUgθUTx=σUθ1⋱θNUTx.Here, we have defined the propagation rule of graph network. But this rule has some drawbacks: (1)N might be a large number, which would be due to large trainable parameters; (2) it is hard to share weight θi in θ; (3) U is computed from the decomposition of L, whose computation cost is ON3. In order to tackle these problems, gθ could be rewritten as a function gθ⋅ in the following formula:(17)θ1⋱θN=gθ=gθΛ,Λ=λ1⋱λN.And Taylor series expansion is adopted to approximategθ.(18)gθΛ≈∑k=0K−1θkΛk.This approximation takes the place ofgθΛ, and we rewrite equation (16):(19)y=σUgθUTx=σUgθΛUTx≈σU∑k=0K−1θkΛkUTx=σ∑k=0K−1θkUΛkUTx=σ∑k=0K−1θkUΛUTkx=σ∑k=0K−1θkLkx.So here, we avoid the computation of decomposition ofL, but Lk still suffers high computation cost. And Chebyshev polynomials are adopted to approximate Lk:(20)Lk≈TkL˜,L˜=2λmaxL−IN,TkL=1,k=0,L,k=1,2LTK−1L+Tk−2L,k≥2.And, equation (19) can be rewritten as follows:(21)y≈σ∑k=0K−1θkTkL˜x.Ifk is set as 2, we get the following formula:(22)gθ∗x≈∑k=01θkTkL˜x=θ0x+θ1L˜x.Sinceθ0 and θ1 influence the scale, it would be less effective after operation of normalization, so they can be set equal: θ=θ0=θ1, and equation (22) can be rewritten as follows:(23)gθ∗x≈θ0x+θ1L˜x=θIN+L˜x.And normalizing the matrixA=IN+L˜, we get(24)A=IN+L˜,A=D˜−1/2AD˜−1/2.In order to learn the relations, weightW is introduced, and a new propagation rule can be obtained:(25)Hl+1=σD˜−1/2AD˜−1/2HlWl,(26)D˜ii=∑jAij,(27)A=IN+L˜,(28)L˜=2λmaxL−IN,where Hl is the output from layer l and Wl is the trainable variables in layer l. And the propagation rule in the graph convolution layer is(29)Hl+1=σA˜HlWl=σD˜−1/2AD˜−1/2HlWl.


## 3.3.1. Fourier Transform

When given a periodic functionf (x), we can break it apart by Fourier series:(2)fx=α02⋅1+∑n=1+∞αnsinnωx+∑n=1+∞bncosnωx,an=∫0Tfxsinnωxdx∫0Tsin2nωxdx,bn=∫0Tfxcosnωxdx∫0Tcos2nωxdx.It can be rewritten in a complex formula:(3)fx=∑n=−∞+∞cnei2πnxT=∑t=−∞+∞cteiωt.It is noteworthy to mention that we can takeeiωt as orthonormal set and take ct as the coordinate.If we want to convert a nonperiodic function into Fourier series, we could regard it as aT=∞ periodic function and use Fourier transform:(4)FTω=∫−∞+∞ft−iωtdt.When givenω, it used e−iωt to decompose ft and get the coordinate of eiωt. And the inverse Fourier transform is(5)ft=∫−∞+∞Fweiωtdw.


## 3.3.2. Graph Laplacian

When we consider Laplace operator in images, it can be defined by the sum of second derivative for the nearest four dimensions:(6)Δf=∑i=1n∂2f∂xi2.If Laplace operator is moved into an undirected graph structure withN nodes, the Laplace operator of each node might be different due to the different relations and connections. The Laplace operator of node i should be defined as follows:(7)Δfi=∑j∂2f∂j2=∑jωijfi−fj=∑jωijfi−∑jωijfj=difi−ωif,where fi is the function value of node i, j are the nodes connected with i, ωij is the weight of i−j connection, di is the degree of i, and ωif is the sum of multiplication of all j and its weight. It can be rewritten in matrix form as follows:(8)Δf=d1…0⋮⋱⋮0⋯dNf1⋮fN−−ωi⋅−⋮−ωN⋅−f1⋮fN=D−WF=LF.And, we get the Laplacian matrixL, and we further get the normalized Laplacian matrix L∼.(9)L∼=D−1/2D−WD−1/2.The decomposition of Laplacian matrixL is(10)Luk=λuk.


## 3.3.3. Graph Fourier Transform

It can be proved by Helmholtz equation thatuk can be used as orthonormal set to decompose fi:(11)Fkλk=f^k=∑i=1Nfiuki,where λk and uk are the eigenvalues and eigenvectors of Laplacian matrix L, and k=1,…,N because L is an N×N symmetric matrix. It can be rewritten in matrix form:(12)f^=f^1⋮f^N=u11…u1n⋮⋱⋮uN1⋯uNNf1⋮fN=UTf.And, the inverse Fourier transform is(13)f=Uf^.


## 3.3.4. Graph Convolution Network

According to convolution theorem, the Fourier transform of a convolution of two signals is the pointwise product of their Fourier Transforms under suitable conditions:(14)Ff∗g=Ff⋅Fg,where F is the Fourier transform, f and g are two signals, ∗ is the convolution operation, and ⋅ is the pointwise product. When applied in graph G, with input f and kernel h, convolution operation in graph can convert to pointwise product under Fourier domain:(15)f∗gG=F−1Ff⋅Fh=F−1UTf⋅h^=Uh^1…0⋮⋱⋮0⋯hN∧UTf=Uh^1…0⋮⋱⋮0⋯hN∧UTf=Uθ1…0⋮⋱⋮0⋯θNUTf.The trainable variablesg convert into θ in Fourier domain. And in graph neural network, we can directly learn θ instead of g. We also get the following formula, where σ is the activation function:(16)y=σUgθUTx=σUθ1⋱θNUTx.Here, we have defined the propagation rule of graph network. But this rule has some drawbacks: (1)N might be a large number, which would be due to large trainable parameters; (2) it is hard to share weight θi in θ; (3) U is computed from the decomposition of L, whose computation cost is ON3. In order to tackle these problems, gθ could be rewritten as a function gθ⋅ in the following formula:(17)θ1⋱θN=gθ=gθΛ,Λ=λ1⋱λN.And Taylor series expansion is adopted to approximategθ.(18)gθΛ≈∑k=0K−1θkΛk.This approximation takes the place ofgθΛ, and we rewrite equation (16):(19)y=σUgθUTx=σUgθΛUTx≈σU∑k=0K−1θkΛkUTx=σ∑k=0K−1θkUΛkUTx=σ∑k=0K−1θkUΛUTkx=σ∑k=0K−1θkLkx.So here, we avoid the computation of decomposition ofL, but Lk still suffers high computation cost. And Chebyshev polynomials are adopted to approximate Lk:(20)Lk≈TkL˜,L˜=2λmaxL−IN,TkL=1,k=0,L,k=1,2LTK−1L+Tk−2L,k≥2.And, equation (19) can be rewritten as follows:(21)y≈σ∑k=0K−1θkTkL˜x.Ifk is set as 2, we get the following formula:(22)gθ∗x≈∑k=01θkTkL˜x=θ0x+θ1L˜x.Sinceθ0 and θ1 influence the scale, it would be less effective after operation of normalization, so they can be set equal: θ=θ0=θ1, and equation (22) can be rewritten as follows:(23)gθ∗x≈θ0x+θ1L˜x=θIN+L˜x.And normalizing the matrixA=IN+L˜, we get(24)A=IN+L˜,A=D˜−1/2AD˜−1/2.In order to learn the relations, weightW is introduced, and a new propagation rule can be obtained:(25)Hl+1=σD˜−1/2AD˜−1/2HlWl,(26)D˜ii=∑jAij,(27)A=IN+L˜,(28)L˜=2λmaxL−IN,where Hl is the output from layer l and Wl is the trainable variables in layer l. And the propagation rule in the graph convolution layer is(29)Hl+1=σA˜HlWl=σD˜−1/2AD˜−1/2HlWl.


## 3.4. Graph Presentation

In order to follow the propagation rule of equation (29), we should compute correlation matrix A. The way to compute A mentioned in equation (27) cannot work, because in this task, the graph is a weighted, directed graph.We adopt the method introduced in ChexGCN, which used a nonlinear method to preprocess the correlation matrixp by equation (34) to reduce the noise and protect the correlations of labels:(30)Aij=λpij∑i=0CPij+θ,ifPij≥ϕ,0,ifPij<ϕ,where λ is a hyperparameter to control the correlation state between the node and its neighborhood, ϕ is the threshold to filter the noise, and θ is an innately small quantity to ensure the denominator is not equal to zero.


## 4. Network Architecture

In this paper, we designed an efficient network architecture named SGGCN as illustrated in Figure1, containing Feature Representation Module (FRM) and Graph Convolution Network Module (GCNM). The FRM used an SGNet-101 efficient neural network architecture to extract image features. GCNM used a small network architecture to extract correlations features from the labels. Finally, the features from FRM and GCNM are combined together and make multilabel prediction by matrix multiplication.Figure 1
The architecture of SGGCN.


### 4.1. Feature Representation Module

In this module, we would use light models to extract image features with low computational consumption. Since some diseases like lung opacity have small scale and low resolution of feature maps might loss information of small target, especially pooling operation and convolution operation with large kernel scale would loss information. So, deep convolution neural network architectures like residual network can help to keep the information, but they suffer high computation cost. In order to design an efficient deep convolution neural network, ShuffleGhost Module is adopted to form ShuffleGhost Block and used this block to build a deep convolution neural network architecture SGNet-101. In ShuffleGhost Module, primary convolution conducts group convolution and generates primary feature with partial channels, and ghost convolution utilizes the redundant information of feature map to recover the ghost feature with rest channels by efficient operation like depthwise convolution; finally, the primary feature is concatenated with and ghost feature and disrupted the channel order with shuffle layer. So, ShuffleGhost can maintain the feature information with high computation efficiency, and SGNet-101 can extract features from multiple resolution with deep neural network. Figure2 shows the structure of ShuffleGhost Module and Block. One ShuffleGhost Block contains two ShuffleGhost Module; each one contains primary convolution part and ghost convolution part. In primary convolution part, group convolution is enrolled. In ghost convolution, cheap convolution is adopted to produce ghost feature map. The outputs from primary convolution part and ghost convolution part are concatenated together to generate output feature.Figure 2
The structure of ShuffleGhost Block and Module.At the end of this module, the backbone SGNet-101 is followed by Global Average Pooling (GAP) layer to compress the features into 1024-d, where we denoted asFFRM.


### 4.2. Graph Convolutional Network Module

This module takes the embedding word of the labels and the graph presentation as input and uses graph convolution network to extract the correlation of the labels. The embedding wordsXemb can be computed in Section 3.1, and the graph presentation A˜ is shown in equation (30). And Xemb and A˜ are fed to the first layer of IFE model:(31)H1=σA˜XembW0=σA˜H0W0,where W0 is the weight of the first layer, H1 is the output of the first layer, σ is the activation function, and Xemb is denoted as H0. The GCNM module consists of two graph convolution layers and one concatenate layer. For each graph convolution layers, the correlation information in different scale is extracted and generated as the output feature, and the output features from two graph convolution layers have the same shape as 512×14, and the two features are concatenated together to generate the output of GCNM module, which is denoted as matrix W. Finally, the information FFRM and W from FRM and GCNM module are combined together by matrix multiplication, followed by sigmoid layer to generate multilabel class prediction.


## 4.1. Feature Representation Module

In this module, we would use light models to extract image features with low computational consumption. Since some diseases like lung opacity have small scale and low resolution of feature maps might loss information of small target, especially pooling operation and convolution operation with large kernel scale would loss information. So, deep convolution neural network architectures like residual network can help to keep the information, but they suffer high computation cost. In order to design an efficient deep convolution neural network, ShuffleGhost Module is adopted to form ShuffleGhost Block and used this block to build a deep convolution neural network architecture SGNet-101. In ShuffleGhost Module, primary convolution conducts group convolution and generates primary feature with partial channels, and ghost convolution utilizes the redundant information of feature map to recover the ghost feature with rest channels by efficient operation like depthwise convolution; finally, the primary feature is concatenated with and ghost feature and disrupted the channel order with shuffle layer. So, ShuffleGhost can maintain the feature information with high computation efficiency, and SGNet-101 can extract features from multiple resolution with deep neural network. Figure2 shows the structure of ShuffleGhost Module and Block. One ShuffleGhost Block contains two ShuffleGhost Module; each one contains primary convolution part and ghost convolution part. In primary convolution part, group convolution is enrolled. In ghost convolution, cheap convolution is adopted to produce ghost feature map. The outputs from primary convolution part and ghost convolution part are concatenated together to generate output feature.Figure 2
The structure of ShuffleGhost Block and Module.At the end of this module, the backbone SGNet-101 is followed by Global Average Pooling (GAP) layer to compress the features into 1024-d, where we denoted asFFRM.


## 4.2. Graph Convolutional Network Module

This module takes the embedding word of the labels and the graph presentation as input and uses graph convolution network to extract the correlation of the labels. The embedding wordsXemb can be computed in Section 3.1, and the graph presentation A˜ is shown in equation (30). And Xemb and A˜ are fed to the first layer of IFE model:(31)H1=σA˜XembW0=σA˜H0W0,where W0 is the weight of the first layer, H1 is the output of the first layer, σ is the activation function, and Xemb is denoted as H0. The GCNM module consists of two graph convolution layers and one concatenate layer. For each graph convolution layers, the correlation information in different scale is extracted and generated as the output feature, and the output features from two graph convolution layers have the same shape as 512×14, and the two features are concatenated together to generate the output of GCNM module, which is denoted as matrix W. Finally, the information FFRM and W from FRM and GCNM module are combined together by matrix multiplication, followed by sigmoid layer to generate multilabel class prediction.


## 5. Experiment

### 5.1. Datasets

This paper mainly focused on CheXPert datasets, which is widely used in deep hierarchical learning for chest disease recognition. The datasets have 14 classes (diseases); the label of each class is one of the four possible labels: NULL, −1, 0, and 1, and they represent empty, uncertain, negative, and positive, respectively. And the distribution of this dataset is illustrated as Table1. We used CheXPert-v1.0-small (https://stanfordmlgroup.github.io/competitions/chexpert/) dataset, and the images in this dataset are not as high resolution as the origin CheXPert dataset, so this would influence the accuracy we can get in CheXGCN. The training set of this dataset has 223414 samples, and the label of each class might be one of four values as mentioned above. And the validation set has 234 samples, and the label of each class might be one of the two labels: positive and negative. After this procedure, the other NULL labels are replaced with negative labels.Table 1
Summary of 14 classes in CheXPert dataset (https://stanfordmlgroup.github.io/competitions/chexpert/).
PathologyPositiveNegativeUncertainEmptyPathologyPositiveNegativeNo finding2238100201033No finding38196Enlarged cardiom.107982163812403178575Enlarged cardiom.109125Cardiomegaly27000111168087177211Cardiomegaly68166Lung opacity10558165995598105636Lung opacity126108Lung lesion918612701488211470Lung lesion1233Edema522462072612984137458Edema45189Consolidation147832809727742152792Consolidation33201Pneumonia6039279918770195806Pneumonia8226Atelectasis33376132833739154971Atelectasis80154Pneumothorax19448563413145144480Pneumothorax8226Pleural effusion86187353961162890203Pleural effusion67167Pleural other35233162653216922Pleural other1233Fracture90402512642211220Fracture0234Support devices11600161371079100197Support devices107127At present, the testing dataset is not yet available, and some classes like Lung Leision, Plerual Other, and Fracture in the validation set are not enough. We divided the dataset into 70% for training, 10% for validation, and 20% for testing.Table1 is the summary of the training set. The right side is the summary of the validation set. The training set of this dataset has 223414 samples, and the label of each class might be one of four values as mentioned above. And the validation set has 234 samples, and the label of each class might be one of the two labels: positive and negative.


### 5.2. Hierarchical Labels

Since this paper focuses on hierarchical learning, this means that labeli might have a strong relationship with label j. The label NULL does not simply mean negative, because in fact, if disease i is a subset of disease j, doctors do not need to check disease j if disease i is positive, so disease j is denoted by NULL.In this situation, the diseasej is positive if disease i is positive, although the label of disease j is NULL. If we replace NULL with negative, we would loss this relation and decrease the correlation between these two diseases. We notice that the validation set only has positive and negative labels in each class, which contain abundant information of relations among the classes. We use the validation set to mine the information.The method this paper used is to compute the conditional probability for each pair of 14 diseases. When computing conditional probability ofi when j: Pi|j, firstly, count the number i and j both appear in validation set Xνa:(32)Ni,j=∑n=1NπXvan,i=1,Xvan,j=1,where N is the number of samples and π is the indicator function. Later, count the number j appear in Xνa.(33)Nj=∑n=1NπXvan,j=1.And, it can be approximatedPi|j as follows:(34)Pi|j=Ni,jNj+10−6.So, the conditional probability for each pair of 14 diseases can be computed. The result is illustrated in Table2. It is noteworthy to mention that the probability p at row j and column i means Pi|j. We can find the following relations:(35)PEnca=1|Card=1=1,(36)POpca=1|Atel=1=1,(37)POpca=1|Pnuel=1=1,(38)POpac=1|Cons=1=1,(39)POpac=1|Edema=1=1,PCons=1|Pnuel=1=1,where Enca, Card, Opca, Atel, Pnue1, Cons, and Edema mean enlarged cardiomediastinum, atelectasis, pneumonia, consolidation, and edema. And, we do not take the positive labels in Lesi (lung lesion), Other (pleural other), and Frac (fracture) into consideration because of the lack of data. And, this paper mainly used the relations equations (35)–(38) because these relations can be proved medically. In this way, we can fill some NULL, Negative, and Uncertain labels in training set to positive labels if it meets the relations above. Table 3 illustrates the result of the extended training data.Table 2
The condition probability of 14 classes in CheXpert.
NofiEncaCardOpacLesiEdemConsPnue1AtelPneu2EffuOtherFracDeviNofi10000000000000EnCa010.6240.7520.0090.3390.2200.0370.4770.0280.431000.495Card0110.76500.3240.2650.0590.5150.0150.441000.515Opac00.6510.41310.0080.3570.2620.0630.6350.0480.4760.00800.492Lesi01011000100001Edem00.8220.4891010.3110.0440.4670.0220.356000.533Cons00.7270.545100.42410.2420.8180.0300.8180.03000.485Pneu100.5000.500100.250110.87500.8750.12500.375Atel00.6500.43710.0120.2620.3370.08710.0120.6120.01200.525Pneu200.3750.1250.75000.1250.12500.12510.250000.500Effu00.7010.4480.89600.2390.4030.1040.7310.03010.01500.493Other00010011101101Frac00000000000000Devi00.5050.3270.5790.0090.2240.1500.0280.3930.0370.3080.00901Table 3
The summary of some classes that are extended by proposed method.
PathologyPositiveNegativeUncertainEmptyOriginEnlarged cardiom.107982163812403178575ProposalEnlarged cardiom.358972146612092153959OriginLung opacity10558165995598105636ProposalLung opacity1342626081398479087


### 5.3. Model Training

In order to discuss the computation and accuracy performance of SGGCN we proposed, we would make comparison with models with backbones of ResNet-101 and MobileNetV2 [17] in Feature Representation Module, respectively. We set θ, ϕ, and λ to 10−6, 0.30, and 0.10 respectively, according to equation (30). In the exploratory experiment, we set initial learning rate lr to 10−3 and decent to 0.1×lr every 5 epoch, as well as set the max epochs to 20, and trained SGGCN with scratch, GCN with ResNet-101 and MobileNetV2 with pretrained models. In order to discuss the performance of GCN, we also trained SGNet-101 without GCNM module.


### 5.4. Results

We trained SGGCN, GCN with ResNet-101 (denoted as ResNet-101-GCN), and MobileNetV2 (denoted as MobilenetV2-GCN), respectively, and get the performance on validation AUC trend as in Figure3, and Table 4 illustrates the result of AUC on training, validation, and testing set, respectively. We could find that SGGCN-101 did not suffer from overfitting, and the performance on validation AUC and test AUC has about 3% lower than ResNet-101-GCN, where MobileNetV2-GCN has about 7% lower than ResNet-101-GCN.Figure 3
The AUC performance trend on validation set of SGGCN-101, ResNet-101-GCN, and MobileNetV2-GCN.Table 4
The AUC performance on the result of AUC on training, validation, and testing set of SGGCN-101, ResNet-101-GCN, and MobileNetV2-GCN.
ModelsTrain AUCValid AUCTest AUCResNet-101-GCN0.85280.80750.8080SGGCN-1010.8027 (−5.87%)0.7834 (−2.98%)0.7831 (−3.08%)MobileNetV2-GCN0.7650 (−10.30%)0.7509 (−7.01%)0.7531 (−6.79%)Since the SGGCN-101, we focus on the efficient computing, and we compared the trainable parameters and FLOPs, as shown in Table5. We could find SGGCN-101 and MobileNetV2-GCN meet a significant decrease in trainable parameters and FLOPs. When the trainable parameters and FLOPs meet about 80% decrease in SGGCN-101, it only has 3% decrease in validation AUC and test AUC.Table 5
The trainable parameters and FLOPs of ResNet-101-GCN, SGGCN-101, and MobileNetV2-GCN.
StructureTrainable parametersFLOPsResNet-101-GCN47,308,86416,017,450,516SGGCN-10112,427,684 (−73.73%)3,072,345,732 (−80.82%)MobileNetV2-GCN5,459,712 (−88.46%)661,395,120 (−95.88%)


### 5.5. Discussion

In graph convolution layers in GCNM in SGGCN, the weights are300×512, 512×512, respectively. And as the structure of SGGCN in Figure 1, when the 14×300 embedding words are fed into GCNM, the features H1 and H2 from graph convolution layers are concatenated and form the output WGCNM, whose dimension is 14×1024. Then, WGCNM is used to do matrix multiplication with the features extracted from FRM (Feature Representation Module). And we can find in this place that WGCNM has similar action as a weight and carries attention information from GCNM module and weight the features in FRM. In order to discuss the influence of GCN, we trained SGNet-101 without GCNM module, which means that the model only has FRM module with backbone of SGNet-101 to extract features, but used a 14×1024 random initialed weight in the fully connected layer WFC to do matrix multiplication with the features.We used Principal Component Analysis [12] to do dimensionality reduction on both WGCNM and WFC and showed the result in Figure 4, where the first figure shows the PCA dimensionality reduction of WGCNM, as well as the second one shows that of WFC. We can find in the 2-dimensional subspace, the distances of these two classes Enlarged Cardiomediastinum and Cardiomegaly in both WGCNM and WFC are small, with 0.0862 of WGCNM and 0.0410, and they all meet the rule of equation (35). But if we focus on the distances among these four diseases: Lung Opacity, Consolidation, Pneumonia, and Atelectasis, we can find WGCNM works much better, because the mean distances among the four diseases is 0.2343, while the mean distances of WFC is 0.3153. The first figure also shows that these four diseases are separated in the subspace of WFC, while the diseases in the subspace of WGCNM still accumulated and retained relationships, and meet the rules of equations (36)–(38).Figure 4
PCA is adopted to reduce the data into two dimensions on bothWGCNM and WFC. The left figure is the PCA result of WGCNM; the other is that of WFC.So far, we have found thatWGCNM can retain the information of equations (35)–(38), and we would mine more potential relationships information to explore its performance. Firstly, we extracted potential relationships information from training data by equation (34), and we got the conditional probability Pi|j. But in the result of dimensionality reduction, the way we judge the relationship of a pair classes is to compare their distance, which is an undirected information, while Pi|j may be different from Pi|j since it contains directed information. In order to tackle this problem, we compress the information of conditional probability into an undirected information:(40)Ii,j=Pi|j+Pj|i2.Table6 shows the information matrix I. We consider using a threshold ε = 0.37 to find out the potential relationships of i,j pair if Ii,j>ε and visualize them by adding edges onto Figure 4, and we get the result of Figure 5. We can find that except class Support Devices, WGCNM also learn some potential relationships, which are not mentioned in equations (36)–(38), the distances of pairs (Edema, Lung Opacity), (Pleural Effusion, Lung Opacity), and (Pleural Effusion, Edema) are much smaller than those of WFC. Meanwhile, Lung Opacity has considerable relations with classes Pneumonia, Consolidation, Atelectasis, Edema, and Pleural Effusion, and it is placed in the center of them in the dimensionality reduction of WGCNM, while the dimensionality reduction of WFC does not have those appearances.Table 6
The undirected information matrixI.
NofiEncaCardOpacLesiEdemConsPnue1AtelPneu2EffuOtherFracDeviNofi10000000000000.233EnCa010.8750.3790.0720.3150.0900.0600.1490.0580.2970.0740.0980.369Card00.87510.3610.0450.3290.0730.0500.1270.0340.2870.0490.0610.358Opac00.3790.36110.3540.4660.5550.5220.6240.3330.6270.3250.2720.621Lesi00.0720.0450.35410.0570.0660.0650.0700.0680.1950.0560.0380.185Edem00.3150.3290.4660.05710.1370.1130.2170.0520.4070.0480.0640.465Conso00.0900.0730.5550.0660.13710.1180.1000.0450.2910.0520.0360.298Pneu100.0600.0500.5220.0650.1130.11810.0590.0150.1530.0280.0190.155Atel00.1490.1270.6240.0700.2170.1000.05910.1260.3390.0610.0880.388Pneu200.0580.0340.3330.0680.0520.0450.0150.12610.2090.0410.0840.350Effu00.2970.2870.6270.1950.4070.2910.1530.3390.20910.1330.1500.534Other00.0740.0490.3250.0560.0480.0520.0280.0610.0410.13310.0630.193Frac00.0980.0610.2720.0380.0640.0360.0190.0880.0840.1500.06310.215Devi0.2330.3690.3580.6210.1850.4650.2980.1550.3880.3500.5340.1930.2151Figure 5
The undirected information matrixI when set threshold ε=0.37.We later applied dimensionality reduction on the outputs of 8000 samples in validation set from SGGCN and SGNet-101, respectively. In detail, we applied PCA on 14 classes, respectively, reduced the data to two dimensions, and applied Gaussian Mixture Model with one class to fit an analogous Gaussian distribution. Figure6 shows the dimensionality reduction of the output. The three figures in the first row show the 2D-PCA from the output of 14 classes, pair (Enlarged Cardiomediastinum, Cardiomegaly) and [Pleural Opacity, Consolidation, Pneumonia, Atelectasis] from SGGCN. And the second row shows the result from SGNet-101. We can find that although WGCNM can take the correlation information, when conducting matrix multiplication with features from FRM, the appearance seems not considerable.Figure 6
The first row shows the 2D-PCA from the output of 14 classes.


## 5.1. Datasets

This paper mainly focused on CheXPert datasets, which is widely used in deep hierarchical learning for chest disease recognition. The datasets have 14 classes (diseases); the label of each class is one of the four possible labels: NULL, −1, 0, and 1, and they represent empty, uncertain, negative, and positive, respectively. And the distribution of this dataset is illustrated as Table1. We used CheXPert-v1.0-small (https://stanfordmlgroup.github.io/competitions/chexpert/) dataset, and the images in this dataset are not as high resolution as the origin CheXPert dataset, so this would influence the accuracy we can get in CheXGCN. The training set of this dataset has 223414 samples, and the label of each class might be one of four values as mentioned above. And the validation set has 234 samples, and the label of each class might be one of the two labels: positive and negative. After this procedure, the other NULL labels are replaced with negative labels.Table 1
Summary of 14 classes in CheXPert dataset (https://stanfordmlgroup.github.io/competitions/chexpert/).
PathologyPositiveNegativeUncertainEmptyPathologyPositiveNegativeNo finding2238100201033No finding38196Enlarged cardiom.107982163812403178575Enlarged cardiom.109125Cardiomegaly27000111168087177211Cardiomegaly68166Lung opacity10558165995598105636Lung opacity126108Lung lesion918612701488211470Lung lesion1233Edema522462072612984137458Edema45189Consolidation147832809727742152792Consolidation33201Pneumonia6039279918770195806Pneumonia8226Atelectasis33376132833739154971Atelectasis80154Pneumothorax19448563413145144480Pneumothorax8226Pleural effusion86187353961162890203Pleural effusion67167Pleural other35233162653216922Pleural other1233Fracture90402512642211220Fracture0234Support devices11600161371079100197Support devices107127At present, the testing dataset is not yet available, and some classes like Lung Leision, Plerual Other, and Fracture in the validation set are not enough. We divided the dataset into 70% for training, 10% for validation, and 20% for testing.Table1 is the summary of the training set. The right side is the summary of the validation set. The training set of this dataset has 223414 samples, and the label of each class might be one of four values as mentioned above. And the validation set has 234 samples, and the label of each class might be one of the two labels: positive and negative.


## 5.2. Hierarchical Labels

Since this paper focuses on hierarchical learning, this means that labeli might have a strong relationship with label j. The label NULL does not simply mean negative, because in fact, if disease i is a subset of disease j, doctors do not need to check disease j if disease i is positive, so disease j is denoted by NULL.In this situation, the diseasej is positive if disease i is positive, although the label of disease j is NULL. If we replace NULL with negative, we would loss this relation and decrease the correlation between these two diseases. We notice that the validation set only has positive and negative labels in each class, which contain abundant information of relations among the classes. We use the validation set to mine the information.The method this paper used is to compute the conditional probability for each pair of 14 diseases. When computing conditional probability ofi when j: Pi|j, firstly, count the number i and j both appear in validation set Xνa:(32)Ni,j=∑n=1NπXvan,i=1,Xvan,j=1,where N is the number of samples and π is the indicator function. Later, count the number j appear in Xνa.(33)Nj=∑n=1NπXvan,j=1.And, it can be approximatedPi|j as follows:(34)Pi|j=Ni,jNj+10−6.So, the conditional probability for each pair of 14 diseases can be computed. The result is illustrated in Table2. It is noteworthy to mention that the probability p at row j and column i means Pi|j. We can find the following relations:(35)PEnca=1|Card=1=1,(36)POpca=1|Atel=1=1,(37)POpca=1|Pnuel=1=1,(38)POpac=1|Cons=1=1,(39)POpac=1|Edema=1=1,PCons=1|Pnuel=1=1,where Enca, Card, Opca, Atel, Pnue1, Cons, and Edema mean enlarged cardiomediastinum, atelectasis, pneumonia, consolidation, and edema. And, we do not take the positive labels in Lesi (lung lesion), Other (pleural other), and Frac (fracture) into consideration because of the lack of data. And, this paper mainly used the relations equations (35)–(38) because these relations can be proved medically. In this way, we can fill some NULL, Negative, and Uncertain labels in training set to positive labels if it meets the relations above. Table 3 illustrates the result of the extended training data.Table 2
The condition probability of 14 classes in CheXpert.
NofiEncaCardOpacLesiEdemConsPnue1AtelPneu2EffuOtherFracDeviNofi10000000000000EnCa010.6240.7520.0090.3390.2200.0370.4770.0280.431000.495Card0110.76500.3240.2650.0590.5150.0150.441000.515Opac00.6510.41310.0080.3570.2620.0630.6350.0480.4760.00800.492Lesi01011000100001Edem00.8220.4891010.3110.0440.4670.0220.356000.533Cons00.7270.545100.42410.2420.8180.0300.8180.03000.485Pneu100.5000.500100.250110.87500.8750.12500.375Atel00.6500.43710.0120.2620.3370.08710.0120.6120.01200.525Pneu200.3750.1250.75000.1250.12500.12510.250000.500Effu00.7010.4480.89600.2390.4030.1040.7310.03010.01500.493Other00010011101101Frac00000000000000Devi00.5050.3270.5790.0090.2240.1500.0280.3930.0370.3080.00901Table 3
The summary of some classes that are extended by proposed method.
PathologyPositiveNegativeUncertainEmptyOriginEnlarged cardiom.107982163812403178575ProposalEnlarged cardiom.358972146612092153959OriginLung opacity10558165995598105636ProposalLung opacity1342626081398479087


## 5.3. Model Training

In order to discuss the computation and accuracy performance of SGGCN we proposed, we would make comparison with models with backbones of ResNet-101 and MobileNetV2 [17] in Feature Representation Module, respectively. We set θ, ϕ, and λ to 10−6, 0.30, and 0.10 respectively, according to equation (30). In the exploratory experiment, we set initial learning rate lr to 10−3 and decent to 0.1×lr every 5 epoch, as well as set the max epochs to 20, and trained SGGCN with scratch, GCN with ResNet-101 and MobileNetV2 with pretrained models. In order to discuss the performance of GCN, we also trained SGNet-101 without GCNM module.


## 5.4. Results

We trained SGGCN, GCN with ResNet-101 (denoted as ResNet-101-GCN), and MobileNetV2 (denoted as MobilenetV2-GCN), respectively, and get the performance on validation AUC trend as in Figure3, and Table 4 illustrates the result of AUC on training, validation, and testing set, respectively. We could find that SGGCN-101 did not suffer from overfitting, and the performance on validation AUC and test AUC has about 3% lower than ResNet-101-GCN, where MobileNetV2-GCN has about 7% lower than ResNet-101-GCN.Figure 3
The AUC performance trend on validation set of SGGCN-101, ResNet-101-GCN, and MobileNetV2-GCN.Table 4
The AUC performance on the result of AUC on training, validation, and testing set of SGGCN-101, ResNet-101-GCN, and MobileNetV2-GCN.
ModelsTrain AUCValid AUCTest AUCResNet-101-GCN0.85280.80750.8080SGGCN-1010.8027 (−5.87%)0.7834 (−2.98%)0.7831 (−3.08%)MobileNetV2-GCN0.7650 (−10.30%)0.7509 (−7.01%)0.7531 (−6.79%)Since the SGGCN-101, we focus on the efficient computing, and we compared the trainable parameters and FLOPs, as shown in Table5. We could find SGGCN-101 and MobileNetV2-GCN meet a significant decrease in trainable parameters and FLOPs. When the trainable parameters and FLOPs meet about 80% decrease in SGGCN-101, it only has 3% decrease in validation AUC and test AUC.Table 5
The trainable parameters and FLOPs of ResNet-101-GCN, SGGCN-101, and MobileNetV2-GCN.
StructureTrainable parametersFLOPsResNet-101-GCN47,308,86416,017,450,516SGGCN-10112,427,684 (−73.73%)3,072,345,732 (−80.82%)MobileNetV2-GCN5,459,712 (−88.46%)661,395,120 (−95.88%)


## 5.5. Discussion

In graph convolution layers in GCNM in SGGCN, the weights are300×512, 512×512, respectively. And as the structure of SGGCN in Figure 1, when the 14×300 embedding words are fed into GCNM, the features H1 and H2 from graph convolution layers are concatenated and form the output WGCNM, whose dimension is 14×1024. Then, WGCNM is used to do matrix multiplication with the features extracted from FRM (Feature Representation Module). And we can find in this place that WGCNM has similar action as a weight and carries attention information from GCNM module and weight the features in FRM. In order to discuss the influence of GCN, we trained SGNet-101 without GCNM module, which means that the model only has FRM module with backbone of SGNet-101 to extract features, but used a 14×1024 random initialed weight in the fully connected layer WFC to do matrix multiplication with the features.We used Principal Component Analysis [12] to do dimensionality reduction on both WGCNM and WFC and showed the result in Figure 4, where the first figure shows the PCA dimensionality reduction of WGCNM, as well as the second one shows that of WFC. We can find in the 2-dimensional subspace, the distances of these two classes Enlarged Cardiomediastinum and Cardiomegaly in both WGCNM and WFC are small, with 0.0862 of WGCNM and 0.0410, and they all meet the rule of equation (35). But if we focus on the distances among these four diseases: Lung Opacity, Consolidation, Pneumonia, and Atelectasis, we can find WGCNM works much better, because the mean distances among the four diseases is 0.2343, while the mean distances of WFC is 0.3153. The first figure also shows that these four diseases are separated in the subspace of WFC, while the diseases in the subspace of WGCNM still accumulated and retained relationships, and meet the rules of equations (36)–(38).Figure 4
PCA is adopted to reduce the data into two dimensions on bothWGCNM and WFC. The left figure is the PCA result of WGCNM; the other is that of WFC.So far, we have found thatWGCNM can retain the information of equations (35)–(38), and we would mine more potential relationships information to explore its performance. Firstly, we extracted potential relationships information from training data by equation (34), and we got the conditional probability Pi|j. But in the result of dimensionality reduction, the way we judge the relationship of a pair classes is to compare their distance, which is an undirected information, while Pi|j may be different from Pi|j since it contains directed information. In order to tackle this problem, we compress the information of conditional probability into an undirected information:(40)Ii,j=Pi|j+Pj|i2.Table6 shows the information matrix I. We consider using a threshold ε = 0.37 to find out the potential relationships of i,j pair if Ii,j>ε and visualize them by adding edges onto Figure 4, and we get the result of Figure 5. We can find that except class Support Devices, WGCNM also learn some potential relationships, which are not mentioned in equations (36)–(38), the distances of pairs (Edema, Lung Opacity), (Pleural Effusion, Lung Opacity), and (Pleural Effusion, Edema) are much smaller than those of WFC. Meanwhile, Lung Opacity has considerable relations with classes Pneumonia, Consolidation, Atelectasis, Edema, and Pleural Effusion, and it is placed in the center of them in the dimensionality reduction of WGCNM, while the dimensionality reduction of WFC does not have those appearances.Table 6
The undirected information matrixI.
NofiEncaCardOpacLesiEdemConsPnue1AtelPneu2EffuOtherFracDeviNofi10000000000000.233EnCa010.8750.3790.0720.3150.0900.0600.1490.0580.2970.0740.0980.369Card00.87510.3610.0450.3290.0730.0500.1270.0340.2870.0490.0610.358Opac00.3790.36110.3540.4660.5550.5220.6240.3330.6270.3250.2720.621Lesi00.0720.0450.35410.0570.0660.0650.0700.0680.1950.0560.0380.185Edem00.3150.3290.4660.05710.1370.1130.2170.0520.4070.0480.0640.465Conso00.0900.0730.5550.0660.13710.1180.1000.0450.2910.0520.0360.298Pneu100.0600.0500.5220.0650.1130.11810.0590.0150.1530.0280.0190.155Atel00.1490.1270.6240.0700.2170.1000.05910.1260.3390.0610.0880.388Pneu200.0580.0340.3330.0680.0520.0450.0150.12610.2090.0410.0840.350Effu00.2970.2870.6270.1950.4070.2910.1530.3390.20910.1330.1500.534Other00.0740.0490.3250.0560.0480.0520.0280.0610.0410.13310.0630.193Frac00.0980.0610.2720.0380.0640.0360.0190.0880.0840.1500.06310.215Devi0.2330.3690.3580.6210.1850.4650.2980.1550.3880.3500.5340.1930.2151Figure 5
The undirected information matrixI when set threshold ε=0.37.We later applied dimensionality reduction on the outputs of 8000 samples in validation set from SGGCN and SGNet-101, respectively. In detail, we applied PCA on 14 classes, respectively, reduced the data to two dimensions, and applied Gaussian Mixture Model with one class to fit an analogous Gaussian distribution. Figure6 shows the dimensionality reduction of the output. The three figures in the first row show the 2D-PCA from the output of 14 classes, pair (Enlarged Cardiomediastinum, Cardiomegaly) and [Pleural Opacity, Consolidation, Pneumonia, Atelectasis] from SGGCN. And the second row shows the result from SGNet-101. We can find that although WGCNM can take the correlation information, when conducting matrix multiplication with features from FRM, the appearance seems not considerable.Figure 6
The first row shows the 2D-PCA from the output of 14 classes.


## 6. Conclusion

In this paper, an efficient X-ray classification method SGGCN is proposed, which adopts SGNet-101 backbone built with ShuffleGhost Module and applies this method on CheXpert datasets to do chest disease classification. We also compare the AUC, trainable parameters, and FLOPs with ResNet-101 with GCN and MobileNetV2 with GCN. It is found that although the trainable parameters and FLOPs meet a significant decrease, SGGCN still keeps a high AUC on validation and testing set.



---

*Source: 6996444-2022-01-07.xml*