# The Classification of Multi-Domain Samples Based on the Cooperation of Multiple Models

Authors: Qingzeng Song; Junting Xu; Lei Ma; Ping Yang; Guanghao Jin
Journal: Complexity
 (2022)

---

## Abstract

This article proposed a novel classification framework that can classify the samples of multiple domains based on the outputs of multiple models. Different from the existing methods that train single model on all domains, our framework trains multiple models on each domain. On a testing sample, the outputs of all trained models are used to predict the domain of this sample. Then, this sample is classified by the output of models that belong to the predicted domain. Experiments show that our framework achieved higher accuracy than the existing methods. Furthermore, our framework achieves good scalability on multiple domains.


---

## Body

## 1. Introduction

In these days, deep learning models can achieve good performance in many applications [1–5]. Generally, the performance of a deep learning model depends on the captured features [6–10]. More domains are important to the scalability of deep learning system while these increase the difficulty of training high-performance models. Figure 1 introduces an example that includes multiple domains. We use the dataset to present the collected samples of the corresponding domain. These datasets may have a different number of labels and various samples, which increase the difficulty of training high-performance models. Transfer learning [11–13] can temporarily solve this problem while the performance is still limited by the structure of models. As Figure 1(a) introduces, a bigger model (deeper structure and more layers) is a general solution as this can capture more features while this is limited by the computational resource or may cause the vanishing gradient problem [14–16]. Thus, there should be another way to improve the performance of deep learning system on multiple domains like using multiple models.Figure 1
The solutions to classify the samples of multiple domains. (a) Bigger model. (b) More models.Some fusion methods [17–19] can utilize multiple models to increase the classification accuracy, where performance depends on the selection of high accurate trained models. Compared with training a single model on multiple domains, training each model on the corresponding domain can be a good solution as this can achieve good scalability as we introduced in Figure 1(b). Generally, the trained model easily ensures high accuracy in the corresponding domain. On the contrary, this model may have low accuracy on the other domains, which reduces the performance of these fusion methods. Thus, on a testing sample, the prediction of the domain is important to the fusion methods.In this article, we built a novel framework (CMS-CMM, the Classification of Multi-Domain Samples Based on the Cooperation of Multiple Models) to increase the accuracy of classification on the samples of multiple domains. Our contribution can be summarized as the following. (1) We built a novel framework that achieves the scalability of the deep learning system. As the number of domains increased, the difficulty of transfer learning is increased as it has to consider the performance of all domains. On the contrary, our framework only needs to train some deep learning models on the training set of a new domain, which benefits the scalability. (2) Our framework increases the accuracy of classification without increasing the structure of models. Generally, a bigger model increases the classification accuracy while it needs more space of memory that is impossible to satisfy in some applications. Instead, our framework only increases the number of models where each of them lowers the consumption of memory space compared with that of a big model.The rest of the article is organized as follows: Section1.1 introduces the existing methods and their problems. In Section 2, we present our framework and related analyses. The experiment is organized in Section 3. Section 5 gives the conclusion and future work.


### 1.1. Related Works

VoVNet-57 (Variety of View Network) is designed for object classification task, which consists of blocks including 3 convolution layers and 4 stages modules and outputs stride 32 [20]. The sample is passed through convolutional layers, where here the filters consist of a small receptive field. ResNeSt (residual networks) is a state-of-art deep learning model for image classification that uses a modular structure with a split-attention block and applies an attention mechanism to feature map groups [21]. From ResNeSt50 to ResNeSt269, the structure becomes bigger and more complicated, so that these can get higher accuracy especially when there are more and bigger size training samples. Based on the size of testing samples and computational resource, we use ResNeSt101 in this article. RepVGG (re-parameterization visual geometry group) is a classification model, which is improved on the basis of the existing models [22]. DenseNet (densely connected convolutional network) is a convolutional neural network with dense connections [23]. In this network, there is a direct connection between any two layers, which means the input of each layer connects to all the previous layers. VGG16 is a variant of VGG (visual geometry group) models for image classification [24]. ResNet (residual neural network) allows the original input information to be detoured directly to the output, which simplifies the process and reduces the difficulty of training [25].Some fusion methods have been applied to improve the performance of classification, which applies multiple models [17]. In that article, weighted voting method achieved the highest accuracy among all of the other ones. Weighted voting is also utilized to construct a more reliable classification system [18]. A sliding window is applied to the weighted majority voting algorithm in that article. This method is applied to a DNN (deep neural network), a CNN (convolutional neural network), and an LSTM (long short-term memory) network to improve the performance [19]. These methods can combine the results of models to improve the accuracy. As the weights play an important role in the combination, there should be a validation set to compute these weights. Furthermore, more various models can benefit the improvement of the accuracy. In this article, we also apply these fusion methods to our system for higher accuracy with some optimizations.When using these methods, the performance of each model is important. Training a model on the single domain can ensure high accuracy on this domain while it may cause low accuracy on the other domains. At the same time, training a model on multiple domains may reduce the accuracy on each domain. Thus, our framework tries to solve this problem, which is introduced in the next section.


## 1.1. Related Works

VoVNet-57 (Variety of View Network) is designed for object classification task, which consists of blocks including 3 convolution layers and 4 stages modules and outputs stride 32 [20]. The sample is passed through convolutional layers, where here the filters consist of a small receptive field. ResNeSt (residual networks) is a state-of-art deep learning model for image classification that uses a modular structure with a split-attention block and applies an attention mechanism to feature map groups [21]. From ResNeSt50 to ResNeSt269, the structure becomes bigger and more complicated, so that these can get higher accuracy especially when there are more and bigger size training samples. Based on the size of testing samples and computational resource, we use ResNeSt101 in this article. RepVGG (re-parameterization visual geometry group) is a classification model, which is improved on the basis of the existing models [22]. DenseNet (densely connected convolutional network) is a convolutional neural network with dense connections [23]. In this network, there is a direct connection between any two layers, which means the input of each layer connects to all the previous layers. VGG16 is a variant of VGG (visual geometry group) models for image classification [24]. ResNet (residual neural network) allows the original input information to be detoured directly to the output, which simplifies the process and reduces the difficulty of training [25].Some fusion methods have been applied to improve the performance of classification, which applies multiple models [17]. In that article, weighted voting method achieved the highest accuracy among all of the other ones. Weighted voting is also utilized to construct a more reliable classification system [18]. A sliding window is applied to the weighted majority voting algorithm in that article. This method is applied to a DNN (deep neural network), a CNN (convolutional neural network), and an LSTM (long short-term memory) network to improve the performance [19]. These methods can combine the results of models to improve the accuracy. As the weights play an important role in the combination, there should be a validation set to compute these weights. Furthermore, more various models can benefit the improvement of the accuracy. In this article, we also apply these fusion methods to our system for higher accuracy with some optimizations.When using these methods, the performance of each model is important. Training a model on the single domain can ensure high accuracy on this domain while it may cause low accuracy on the other domains. At the same time, training a model on multiple domains may reduce the accuracy on each domain. Thus, our framework tries to solve this problem, which is introduced in the next section.


## 2. Our Framework

Before giving the details of our framework, we give the following definitions. These definitions are to explain the implementation of the methods.


### 2.1. Preliminaries

We setSn as a sample and Lk as the label of an object. We set Lgroundtruth as the ground truth on Sn where Lgroundtruth∈Lk [26, 27]. The label is to benefit the computation, which is generally a number [28, 29]. For example, when there are 10 objects to be classified, the label is from 0 to 9.


### 2.2. The Illustration of Our Framework

Figure2 illustrates our framework, which is named as CMS-CMM. At the first step, our framework trains some existing deep learning models (like deep learning models 0, 1, and 2 in this figure) on each domain (like domains 0 and 1 in this figure). Then, on a testing sample, each model outputs the probability of labels. Firstly, based on the difference in these probabilities, we can predict the domain of this sample (illustrated by the chart). Secondly, we select the trained models of the predicted domain. Then, we can use the output of these models to predict the label of this sample (illustrated by the chart).Figure 2
The illustration of the framework of our methods.


### 2.3. Training the Models and Outputting the Probability of Labels

We select a deep learning modelMi. Then, we train Mi on a domain Du to get a trained model Mi,u. We define PMi,uSn=Lk as the probability of label Lk on the sample Sn, which is the output by the trained model Mi,u. Generally, the most possible result is selected by the following equation:(1)Lresult=argmaxLkPMi,uSn=Lk.Which is used as the predicted result.


### 2.4. Predicting the Domain

In our framework, we firstly select some existing deep learning modelsMi. Then, we train these models on each domain Du to get a set of trained models Mi,u. When we assume a sample Sn belongs to a domain Du, we can get a probability of labels PMi,uSn=Lk by a model Mi,u. Then, by the other model Mj,u, we can also get PMj,uSn=Lk. We define the difference between the model Mi,u and the model Mj,u on a sample Sn as follows:(2)Mi,uSn−Mj,uSnLk=∑LkPMi,uSn=Lk−PMj,uSn=Lk.We can define the difference betweenMi,u and Mj,u on a sample Sn as follows:(3)∑j‖Mi,uSn−Mj,uSn‖Lk=∑j∑LkPMi,uSn=Lk−PMj,uSn=Lk.Generally, we can selectMi,u as the model that can achieve the highest accuracy on the validation set. Thus, (3) is to present the difference between the highest accurate model with the other ones. Then, we can select the domain Dresult as the predicted domain of sample Sn as below:(4)Dresult=argminDu∑j‖Mi,uSn−Mj,uSn‖Lk.Figure3 uses an example to explain how to predict the domain. We assume that the domain D1 contains the sample and there are three models trained on this domain, which are M0,1, M1,1, M2,1. Thus, these models can well capture the key features of this sample, which leads the probability of dog being high and those of other labels being low. We assume the domain D0 does not contain the samples of dog. Then, we can also get corresponding three trained models M0,0, M1,0, M2,0 of this domain. As these models have not captured the features of dog on the training set, these models may capture noisy features of dog (also included by other labels), which causes big difference between the outputs of these models.Figure 3
The output of trained models with and without dog.


### 2.5. Predicting the Label

Once our framework has predicted the domain of a sample, we can use the corresponding models that are trained on this domain to predict the label of this sample. To increase the accuracy of the prediction, our framework uses the fusion method [19] that is weighted model average as follows:(5)Wi=11−PMi,uSn=Lgoundtruth,Wi=Wi∑jWj,Lresult=argmaxLk∑iPMi,uSn=Lk×Wi,where Wi presents the weight that is applied to the output of models. By using these weights, the output of higher accurate model plays a more important role to the final result. We can compute the Wi by using the validation set. We name our framework with this optimization as CMS-CMM from now on.


### 2.6. Optimization by the Distribution of the Labels

There are two cases that may cause the wrong prediction of the domain. Figure4 introduces the two cases. We assume that domain 0 has 100 labels and domain 1 has 10 labels. By the general setting, the trained models of domain 0 will output the probability of 100 labels. At the same time, the trained models of domain 1 will output the probability of 10 labels. In 4(a) of this figure, we input the testing sample of domain 0 into the trained models. The difference between the trained models of domain 1 may be lower than that between the trained models of domain 0 occasionally because the range of error labels is reduced. Especially when the accuracy of models is low, this case easily causes wrong prediction of the domain. In 4(b) of this figure, we input the testing sample of domain 1 into the trained models. The difference between the trained models of domain 0 may be lower than that between the trained models of domain 1 occasionally as the error labels are scattered to a wider range. Especially when the accuracy of models is high, this case also easily causes the wrong prediction of domain.Figure 4
The two cases causing the wrong prediction of domain.
(a)(b)To solve this problem, we make all of the trained models predict the same number of labels. For example, the trained models onD0 (100 labels) or D1 (10 labels) can predict 10 labels, which is the maximum number of labels among these domains. Then, when the testing samples belong to D1, we only consider labels 0 to 9 as the possible correct one. Thus, we revise equation (4) as the following when there are different number of labels between domains.(6)Dresult=argminDu∑j‖Mi,uSn−Mj,uSn‖L¯k+φu×∑j‖Mi,uSn−Mj,uSn‖L¯k,where Lk is the possible correct labels of the corresponding domains Du. We set L¯k as the remain labels. For example, when the D0 contains 100 labels and the domain D1 contains 10 labels, we can set L¯k is from 10 to 99 for D1. Thus, all models of these domains output the probability of the same number labels. We can use φu by using the validation set. We name our framework with this optimization as CMS-CMM-opt from now on.


## 2.1. Preliminaries

We setSn as a sample and Lk as the label of an object. We set Lgroundtruth as the ground truth on Sn where Lgroundtruth∈Lk [26, 27]. The label is to benefit the computation, which is generally a number [28, 29]. For example, when there are 10 objects to be classified, the label is from 0 to 9.


## 2.2. The Illustration of Our Framework

Figure2 illustrates our framework, which is named as CMS-CMM. At the first step, our framework trains some existing deep learning models (like deep learning models 0, 1, and 2 in this figure) on each domain (like domains 0 and 1 in this figure). Then, on a testing sample, each model outputs the probability of labels. Firstly, based on the difference in these probabilities, we can predict the domain of this sample (illustrated by the chart). Secondly, we select the trained models of the predicted domain. Then, we can use the output of these models to predict the label of this sample (illustrated by the chart).Figure 2
The illustration of the framework of our methods.


## 2.3. Training the Models and Outputting the Probability of Labels

We select a deep learning modelMi. Then, we train Mi on a domain Du to get a trained model Mi,u. We define PMi,uSn=Lk as the probability of label Lk on the sample Sn, which is the output by the trained model Mi,u. Generally, the most possible result is selected by the following equation:(1)Lresult=argmaxLkPMi,uSn=Lk.Which is used as the predicted result.


## 2.4. Predicting the Domain

In our framework, we firstly select some existing deep learning modelsMi. Then, we train these models on each domain Du to get a set of trained models Mi,u. When we assume a sample Sn belongs to a domain Du, we can get a probability of labels PMi,uSn=Lk by a model Mi,u. Then, by the other model Mj,u, we can also get PMj,uSn=Lk. We define the difference between the model Mi,u and the model Mj,u on a sample Sn as follows:(2)Mi,uSn−Mj,uSnLk=∑LkPMi,uSn=Lk−PMj,uSn=Lk.We can define the difference betweenMi,u and Mj,u on a sample Sn as follows:(3)∑j‖Mi,uSn−Mj,uSn‖Lk=∑j∑LkPMi,uSn=Lk−PMj,uSn=Lk.Generally, we can selectMi,u as the model that can achieve the highest accuracy on the validation set. Thus, (3) is to present the difference between the highest accurate model with the other ones. Then, we can select the domain Dresult as the predicted domain of sample Sn as below:(4)Dresult=argminDu∑j‖Mi,uSn−Mj,uSn‖Lk.Figure3 uses an example to explain how to predict the domain. We assume that the domain D1 contains the sample and there are three models trained on this domain, which are M0,1, M1,1, M2,1. Thus, these models can well capture the key features of this sample, which leads the probability of dog being high and those of other labels being low. We assume the domain D0 does not contain the samples of dog. Then, we can also get corresponding three trained models M0,0, M1,0, M2,0 of this domain. As these models have not captured the features of dog on the training set, these models may capture noisy features of dog (also included by other labels), which causes big difference between the outputs of these models.Figure 3
The output of trained models with and without dog.


## 2.5. Predicting the Label

Once our framework has predicted the domain of a sample, we can use the corresponding models that are trained on this domain to predict the label of this sample. To increase the accuracy of the prediction, our framework uses the fusion method [19] that is weighted model average as follows:(5)Wi=11−PMi,uSn=Lgoundtruth,Wi=Wi∑jWj,Lresult=argmaxLk∑iPMi,uSn=Lk×Wi,where Wi presents the weight that is applied to the output of models. By using these weights, the output of higher accurate model plays a more important role to the final result. We can compute the Wi by using the validation set. We name our framework with this optimization as CMS-CMM from now on.


## 2.6. Optimization by the Distribution of the Labels

There are two cases that may cause the wrong prediction of the domain. Figure4 introduces the two cases. We assume that domain 0 has 100 labels and domain 1 has 10 labels. By the general setting, the trained models of domain 0 will output the probability of 100 labels. At the same time, the trained models of domain 1 will output the probability of 10 labels. In 4(a) of this figure, we input the testing sample of domain 0 into the trained models. The difference between the trained models of domain 1 may be lower than that between the trained models of domain 0 occasionally because the range of error labels is reduced. Especially when the accuracy of models is low, this case easily causes wrong prediction of the domain. In 4(b) of this figure, we input the testing sample of domain 1 into the trained models. The difference between the trained models of domain 0 may be lower than that between the trained models of domain 1 occasionally as the error labels are scattered to a wider range. Especially when the accuracy of models is high, this case also easily causes the wrong prediction of domain.Figure 4
The two cases causing the wrong prediction of domain.
(a)(b)To solve this problem, we make all of the trained models predict the same number of labels. For example, the trained models onD0 (100 labels) or D1 (10 labels) can predict 10 labels, which is the maximum number of labels among these domains. Then, when the testing samples belong to D1, we only consider labels 0 to 9 as the possible correct one. Thus, we revise equation (4) as the following when there are different number of labels between domains.(6)Dresult=argminDu∑j‖Mi,uSn−Mj,uSn‖L¯k+φu×∑j‖Mi,uSn−Mj,uSn‖L¯k,where Lk is the possible correct labels of the corresponding domains Du. We set L¯k as the remain labels. For example, when the D0 contains 100 labels and the domain D1 contains 10 labels, we can set L¯k is from 10 to 99 for D1. Thus, all models of these domains output the probability of the same number labels. We can use φu by using the validation set. We name our framework with this optimization as CMS-CMM-opt from now on.


## 3. Experiment

We evaluate our methods with the existing ones on some real datasets. When we randomized the parameters, we evaluate 1000 times. We trained the deep learning models (VoVNet-57 [20], ResNeSt50 [21], RepVGG [22], DenseNet [23], VGG16 [24], and ResNet [25]) on some real datasets by the reported default settings of these models. We set the number of epochs [30, 31] as 10 for all these models on any training set. We do not focus on the designing of structure or tuning the hyper-parameters. Instead, we focus on how to use multiple models to achieve the scalability and ensure high accuracy at the same time. We set a random number of validation samples, which is from 500 to 800.


### 3.1. Introduction of the Datasets

CIFAR-10 [32, 33] has 50000 training samples and 10000 testing samples that belong to 10 labels. We use 50000 training samples to train the models. Then we have 10000 samples left to the validation and testing. CIFAR-100 dataset is just like the CIFAR-10, except it has 100 classes containing 600 images each [34, 35]. There are 500 training images and 100 testing images per label. We use 50000 training samples to train the models. Then we have 10000 samples left to the validation and testing. The Mini-ImageNet [36, 37] dataset is for few-shot learning evaluation. Its complexity is high due to the use of ImageNet images but requires fewer resources and infrastructure than running on the full ImageNet dataset. We use 48000 training samples to train the models. Then we have 12000 samples left to the validation and testing. EuroSAT [38, 39] dataset is based on satellite images consisting of 10 classes with 27000 labelled samples. We use 21600 as training samples and 5400 as the testing ones. Intel Image Classification [40] dataset contains natural scenes around the world. There are around 14 k images for training, 3 k for testing, and 7 k in prediction (without labels).


### 3.2. Introduction of the Evaluation Metrics

We introduce some metrics to compare the methods in different dimensions on the testing samples. We assume a sampleSn belongs to the domain Dgroundtruth and the corresponding ground truth of label is Lgroundtruth. We define Lresult is the predicted result of label and Dresult as the predicted result of domain by a method. Then we can define the following evaluation metrics.CD presents the accuracy of predicting correct domains as below:(7)CD=∑Dresult=Dgroundtruth∑Lresult1∑Dresult∑Lresult1,where the higher one is better. CDCL presents the accuracy of predicting correct domains and correct labels as below:(8)CDCL=∑Dresult=Dgroundtruth∑Lresult=Lgroundtruth1∑Dresult∑Lresult1,where the higher one is better. CDWL presents that the percentage of predicting correct domains and wrong labels as below:(9)CDWL=∑Dresult=Dgroundtruth∑Lresult1Lgroundtruth1∑Dresult∑Lresult1,where the lower one is better.WD presents the percentage of predicting wrong domains as below:(10)WD=∑Dresult≠Dgroundtruth∑Lresult1∑Dresult∑Lresult1,where the lower one is better. WDCL presents the percentage of predicting wrong domains and correct labels as below:(11)WDCL=∑Dresult≠Dgroundtruth∑Lresult=Lgroundtruth1∑Dresult∑Lresult1,where the lower one is better. When the prediction of the domain goes wrong, the predicting of labels is meaningless as the labels of different datasets indicates different kinds of objects. WDWL presents the percentage of predicting wrong domains and wrong labels as below:(12)WDWL=∑Dresult≠Dgroundtruth∑Lresult≠Lgroundtruth1∑Dresult∑Lresult1,where the lower one is better.


### 3.3. Evaluation of Domain Prediction

We do not use additional information (like the resolution or size of sample) to predict the correct domain of samples. In Table1, Maximum appeared method predicts the domain by the maximum appeared label. In more detail, we select the result that appeared maximum times from the trained models of each domain. Among all these results, we select the one that appeared maximum times and set the corresponding domain as the predicted domain. Following the same way, the fusion method [19] predicts the domain by the maximum value of weight probabilities.Table 1
Evaluation based on CD (the accuracy of predicting correct domains).
MethodsCIFAR-10 (%)CIFAR-100 (%)Mini-ImageNet (%)EuroSAT (%)Intel image classification (%)Maximum appeared method79.2140.9350.3855.0453.26Fusion method [19]71.7940.3261.4690.9183.03CMS-CMM85.5971.2773.6592.4865.70CMS-CMM-opt95.7877.6375.5299.0990.62We used CD (the accuracy of predicting correct domains) to evaluate the methods. As we can see in Table1, our CMS-CMM-opt achieves higher accuracy than the existing methods, which is 16.62% higher on average. Furthermore, CMS-CMM-opt achieves higher accuracy than our CMS-CMM, which proves the efficiency of the optimization.


### 3.4. Evaluation of Label Classification

Our final objective is to classify the samples. Thus, based on the prediction of domain, there must be also the classification of the samples at the following step. Thus, on a testing sample, only when a method correctly predicted the domain and label at the same time, we admit this method correctly output the result. For example, the label 9 of CIFAR-10 and the label 9 of CIFAR-100 mean different kinds of objects.We used CDCL (the accuracy of predicting correct domains and correct labels) to evaluate the methods. As we can see in Table2, our CMS-CMM-opt achieves higher accuracy than the existing methods, which is 14.01% higher on average. Compared with the domain prediction, the increase of the accuracy is reduced from 16.62% to 14.01% because there is also error when predicting the labels. CMS-CMM use the fusion method [19] as the prediction of labels after the domain prediction. We can see CMS-CMM-opt also achieves higher accuracy than CMS-CMM, which is 11.25% higher on average.Table 2
Evaluation based on CDCL (the accuracy of predicting correct domains and correct labels).
MethodsCIFAR-10 (%)CIFAR-100 (%)Mini-ImageNet (%)EuroSAT (%)Intel image classification (%)Maximum appeared76.2837.7548.0054.0751.50Fusion method [19]69.9838.2259.2688.9380.87CMS-CMM80.3057.9966.3789.4663.27CMS-CMM-opt89.2170.7471.3995.4386.86


### 3.5. Evaluation of the Scalability

In this subsection, we do research about the scalability of our framework based on the metric of CDCL (the accuracy of predicting correct domains and correct labels). We added the domain one by one and computed the label classification as Table3 shows.Table 3
Evaluation of domain scalability based on CDCL.
MethodsCIFAR-10 (%)CIFAR-100Mini-ImageNetEuroSATIntel image classification1 domain92.76————2 domains89.2172.43%———3 domains89.2172.28%72.13%——4 domains89.2171.12%71.62%95.43%—5 domains89.2170.74%71.39%95.43%86.86%As we can see in Table3, the accuracy of CIFAR-10 remains the same as the number of domains becomes big. On the other side, the accuracies of CIFAR-100 and Mini-ImageNet becomes lower. The accuracy of each models plays important role to the classification accuracy. The other important factor to the accuracy is the similarity between domains, which will be introduced in the next subsection.


### 3.6. Impact between Domains

We can analyse the impact of a domain to other domains as Table4 shows. In this table, we compared the CDCL (the accuracy of predicting correct domains and correct labels) of 5 domains with that of 4 domains, which means we drop one domain to evaluate the relation between this domain and other ones. When we drop CIFAR-10, we found the accuracy of CIFAR-100 is increased more than that of others. By the same way, we can find relations between domains. When there are similar labels between domains, the prediction of domain and labels may easily go wrong. For example, the label “fox” of CIFAR-100 is similar to the label “white fox” of Mini-ImageNet. Thus, how to consider the similarity between datasets are important to the increase of accuracy.Table 4
Impact between domains based on CDCL.
MethodsCIFAR-10CIFAR-100Mini-ImageNetEuroSATIntel image classificationWithout CIFAR-10—+2.68%+1.63%+0.00%+0.00%Without CIFAR-100+2.79%—+8.13%+0.48%+1.32%Without Mini-ImageNet+0.00%+0.15%—+0.03%+4.32%Without EuroSAT+0.00%+1.16%+0.52%—+0.00%Without intel Image classification+0.00%+0.38%+0.23%+0.00%—


### 3.7. Evaluation on the Number of Models

In this subsection, we evaluate the relation between the number of models and CDCL (the accuracy of predicting correct domains and correct labels). We set the number of models is from 2 to 6. As there may be different combinations of models, we evaluate the average accuracy of these combinations. As we can see in Figure5, the accuracy of all datasets increases as the number of models becomes bigger. On the other side, when the number of models is 6, the accuracy of some datasets becomes lower than that of 5 models. When there are low accurate models, these may lower the classification accuracy of our framework. The proper number of models can be computed by the validation set.Figure 5
The evaluation on the number of models based on CDCL.


### 3.8. Evaluation by More Metrics on All Testing Samples

As we can see in Table5, our methods achieved better performance in the most of metrics. In the CDWL (the percentage of predicting correct domains and wrong labels) case, the percentages of our methods are higher than those of the other methods. This is because our methods can predict more correct domains, which may cause more wrong labels. Compared with the fusion method [19], our method can increase 19.14% of CDCL while only increasing 9.01% CDWL.Table 5
Evaluation by more metrics on all testing samples.
MethodsCD: correct domainsCDCL: correct domains and correct labelsCDWL: correct domains and wrong labelsWD: wrong domainsWDCL: wrong domains and correct labelsWDWL: wrong domains and wrong labelsMaximum appeared55.98%53.49%2.49%44.02%32.36%11.66%Fusion method [19]62.85%60.72%2.13%37.15%25.13%12.02%CMS-CMM77.70%70.35%7.35%22.30%15.51%6.79%CMS-CMM-opt91.00%79.86%11.14%9.00%5.97%3.03%


### 3.9. Evaluation of Execution Time and Memory Consumption

Table6 shows the total execution time and maximum memory consumption of each trained model on the corresponding dataset. We use Tesla K80 of NVIDIA [41] to run the models. In more detail, we use Tesla K80 of NVIDIA to run the model VoVNet-57 on CIFAR-10 and record the total execution time and maximum memory consumption of this model, which is shown in this table. Then, we also can use Tesla K80 of NVIDIA to run the model VoVNet-57 on the other datasets and record the total execution time and maximum memory consumption of this model. By the same pattern, we can run the other models on each dataset and record the total execution time and maximum memory consumption of these models.Table 6
Execution time and memory consumption.
MethodDatasetThe execution time (second)Max memory consumption (MB)CIFAR-10CIFAR-100Mini-ImageNetEuroSATIntel image classificationVoVNet-57106.05104.95475.3560.2046.63136.25Res2Net50105.20396.89474.8059.3744.0491.01RepVGG-A034.6735.5545.6920.7015.9330.35DenseNet12185.24376.06451.0348.9136.6527.52VGG16122.20121.59495.9772.7955.64513.73ResNet5080.16373.67449.1446.7835.0690.46CMS-CMM-opt in serial604.751479.952472.50369.07294.28687.04CMS-CMM-opt in parallel222.52497.24600.29164.02146.87839.29Our methods generate multiple models on each dataset, which causes the runtime our methods become bigger than those of using single model.CMS-CMM-opt in serial runs the model one by one, which causes the execution time equals to the following: the execution time of single model × the number of models + the execution time of our fusion process. In more detail, our CMS-CMM-opt in serial run multiple models (one after one) on CIFAR-10. Then we run our fusion method. During these processes, we record the total execution time and maximum memory consumption that is shown in Table 6.A simple solution to reduce the execution time is that we can use less models but this may lower the accuracy. To further reduce the execution time without lowing the accuracy, we run the models on the distributed computational nodes of a cluster based on the parallel pattern of paper [42]. In more detail, CMS-CMM-opt in parallel utilizes multiple computational nodes where each node has a Tesla K80 of NVIDIA [41]. As each node can run the model at the same time, the total execution time can be reduced. The total execution time is recorded as the end of all nodes and the maximum memory consumption is counted as the maximum one among these nodes. As CMS-CMM-opt in parallel of Table 6 indicates, the total runtime is reduced compared with CMS-CMM-opt in serial. The additional execution time is caused by the communication and our fusion process. The additional memory consumption is caused by the buffers of communication and our fusion process.To achieve higher accuracy by single model on multiple domains, the structure becomes deeper and more complex which causes the memory consumption becomes bigger. For example, the V-MoE of Google achieves high accuracy with a trained model of 15 billion parameters on the ImageNet [43]. Compared with super model solution, our framework is more scalable.


### 3.10. Illustration of the Domain and Label Classification

Firstly, we use an example to explain the domain classification. As Figure6 shows, we select the testing samples of label 1 that belongs to domain CIFAR-10. Then, we run all of the models on this sample and compute the average difference between the outputs of trained models that belong to the same datasets. Figure 6(a) is the difference of the outputs between the models that are trained on CIFAR-10. Figures 6(a)–6(e) are the differences of the outputs between the models that are trained on CIFAR-100, Mini-ImageNet, EuroSAT, and Intel image classification. As the testing samples belong to the CIFAR-10, the difference between the trained models of CIFAR-10 is obviously smaller than those of the other datasets. Thus, the domain classification based on model difference is reasonable.Figure 6
The difference between the output of trained models. The ground truth is label 1 sample of domain 0. (a) The models trained on CIFAR-10. (b) The models trained on CIFAR-100. (c) The models trained on Mini-ImageNet. (d) The models trained on EuroSAT. (e) The models trained on Intel image classification.
(a)(b)(c)(d)(e)We present the statistical result of model difference by the following. For a sampleSn that belongs to the domain Du, we use avgDMSn∈Du to present the average difference between the trained models of Du as below:(13)avgDMSn∈Du=∑Sn∈Du,u,j≠0‖M0,uSn−Mj,uSn‖Lk∑Sn∈Du,u,j≠0,Lk1,where M0,uSn−Mj,uSnLk is defined by (2), M0,uSn is the output of highest accurate model, and Mj,uSn is the output of the other model. For a sample Sn that belongs to the domain Du, we use avgDMSn∉Dv to present the difference between the models of Dv (v≠u) as below:(14)avgDMSn∉Dv=∑Sn∈Du,v,j≠0‖M0,vSn−Mj,vSn‖Lk∑Sn∈Du,v,j≠0,Lk,1,where M0,vSn is the output of highest accurate model on the validation samples of Dv, Mj,vSn is the output of the other model the validation samples of Dv. As we can see in Table 7, avgDMSn∈Du is obviously smaller than avgDMSn∉Dv on each dataset that means we can use this value the predict the domain. Based on this analysis, our methods further optimize the prediction of domain. CIFAR-100 and Mini-ImageNet has bigger number of labels than the other datasets, which causes the model difference bigger than that of the other datasets.Table 7
Statistical analysis of model difference.
MethodsCIFAR-10CIFAR-100Mini-ImageNetEuroSATIntel image classificationavgDMSn∈Du0.30930.72160.64680.31950.2487avgDMSn∉Dv0.48601.2111.56310.58410.3040Secondly, we use three examples to explain the label classification based on the trained models of corresponding domain. In the first example, we select the testing samples of label 1 from CIFAR-10. Then, we run the trained models of CIFAR-10 on these samples. As Figure7(a) shows, the average probability of label 1 by each model (trained on CIFAR-10) is obviously higher than those of the other labels. In Figure 7(b) case, we select the label 9 testing samples from CIFAR-100. Then, we run the trained models of CIFAR-100 on these samples. In Figure 7(c) case, we select label 90 testing samples from Mini-ImageNet. Then, we run the trained models of Mini-ImageNet on these samples. All of these cases show that the average probability of ground truth label is obviously higher than those of the other labels when we correctly select the trained models of corresponding dataset. Thus, the classification based on the probability of labels is reasonable.Figure 7
The probability of labels by different models of corresponding domain. (a) The ground truth is label 1 of domain 0. (b) The ground truth is label 9 of domain 1. (c) The ground truth is label 90 of domain 2.
(a)(b)(c)We present the statistical analysis of label probability by the models. For a sampleSn that belongs to the domain Du, we define the average probability of ground truth label by the trained models of Du as below:(15)avgPMi,uSn=Lgroundtruth=∑i,u,SnPMi,uSn=Lgroundtruth∑i,u,Sn,1,where PMi,uSn=Lgroundtruth is introduced in equation (1). For a sample Sn that belongs to the domain Du, we define the average value of maximum probability of other label by the models as below:(16)avgPMi,uSn≠Lgroundtruth=∑i,u,SnMaximumLk≠LgroundtruthPMi,uSn=Lk∑i,u,Sn1.As we can see in Table8, avgPMi,uSn=Lgroundtruth is obviously bigger than avgPMi,uSn≠Lgroundtruth on each dataset that means we can use this value to predict the label. Based on this analysis, our methods further optimize the prediction of label.Table 8
Statistical analysis of predicting label.
MethodsCIFAR-10CIFAR-100Mini-ImageNetEuroSATIntel image classificationavgPMi,uSn=Lgroundtruth0.81010.58940.70740.84910.8588avgPMi,uSn≠Lgroundtruth0.06590.07490.05780.05520.0950


### 3.11. Introduction of the Employed Acronyms

We use Table9 to give the introduction of the employed acronyms in this article for reader’s convenience.Table 9
Introduction of the employed acronyms.
NumAcronymsIntroduction1CIFAR-10Dataset that is introduced in [32, 33].2CIFAR-100Dataset that is introduced in [34, 35].3Mini-ImageNetDataset that is introduced in [36, 37].4EuroSATDataset that is introduced in [38, 39].5Intel image classificationDataset that is introduced in [40].6SnPresents a sample.7LkPresents a label.8GnPresents the ground truth onSn.9CMS-CMMOur framework that is introduced in subsection3.310CMS-CMM-optOur framework that is introduced in subsection3.411Tesla K80NVIDIA GPU that is introduced in subsection 4.912CMS-CMM-opt in serialOur serial execution that is introduced in subsection 4.913CMS-CMM-opt in parallelOur parallel execution that is introduced in subsection 4.914VoVNet-57A deep learning model that is introduced in [20].15ResNeSt50S deep learning model that is introduced in [21].16RepVGGA deep learning model that is introduced in [22].17DenseNetA deep learning model that is introduced in [23].18VGG16A deep learning model that is introduced in [24].19ResNetA deep learning model that is introduced in [25].20CDThe accuracy of predicting correct domains21CDCLThe accuracy of predicting correct domains and correct labels22CDWLThe percentage of predicting correct domains and wrong labels23WDThe percentage of predicting wrong domains24WDCLThe percentage of predicting wrong domains and correct labels25WDWLThe percentage of predicting wrong domains and wrong labels


## 3.1. Introduction of the Datasets

CIFAR-10 [32, 33] has 50000 training samples and 10000 testing samples that belong to 10 labels. We use 50000 training samples to train the models. Then we have 10000 samples left to the validation and testing. CIFAR-100 dataset is just like the CIFAR-10, except it has 100 classes containing 600 images each [34, 35]. There are 500 training images and 100 testing images per label. We use 50000 training samples to train the models. Then we have 10000 samples left to the validation and testing. The Mini-ImageNet [36, 37] dataset is for few-shot learning evaluation. Its complexity is high due to the use of ImageNet images but requires fewer resources and infrastructure than running on the full ImageNet dataset. We use 48000 training samples to train the models. Then we have 12000 samples left to the validation and testing. EuroSAT [38, 39] dataset is based on satellite images consisting of 10 classes with 27000 labelled samples. We use 21600 as training samples and 5400 as the testing ones. Intel Image Classification [40] dataset contains natural scenes around the world. There are around 14 k images for training, 3 k for testing, and 7 k in prediction (without labels).


## 3.2. Introduction of the Evaluation Metrics

We introduce some metrics to compare the methods in different dimensions on the testing samples. We assume a sampleSn belongs to the domain Dgroundtruth and the corresponding ground truth of label is Lgroundtruth. We define Lresult is the predicted result of label and Dresult as the predicted result of domain by a method. Then we can define the following evaluation metrics.CD presents the accuracy of predicting correct domains as below:(7)CD=∑Dresult=Dgroundtruth∑Lresult1∑Dresult∑Lresult1,where the higher one is better. CDCL presents the accuracy of predicting correct domains and correct labels as below:(8)CDCL=∑Dresult=Dgroundtruth∑Lresult=Lgroundtruth1∑Dresult∑Lresult1,where the higher one is better. CDWL presents that the percentage of predicting correct domains and wrong labels as below:(9)CDWL=∑Dresult=Dgroundtruth∑Lresult1Lgroundtruth1∑Dresult∑Lresult1,where the lower one is better.WD presents the percentage of predicting wrong domains as below:(10)WD=∑Dresult≠Dgroundtruth∑Lresult1∑Dresult∑Lresult1,where the lower one is better. WDCL presents the percentage of predicting wrong domains and correct labels as below:(11)WDCL=∑Dresult≠Dgroundtruth∑Lresult=Lgroundtruth1∑Dresult∑Lresult1,where the lower one is better. When the prediction of the domain goes wrong, the predicting of labels is meaningless as the labels of different datasets indicates different kinds of objects. WDWL presents the percentage of predicting wrong domains and wrong labels as below:(12)WDWL=∑Dresult≠Dgroundtruth∑Lresult≠Lgroundtruth1∑Dresult∑Lresult1,where the lower one is better.


## 3.3. Evaluation of Domain Prediction

We do not use additional information (like the resolution or size of sample) to predict the correct domain of samples. In Table1, Maximum appeared method predicts the domain by the maximum appeared label. In more detail, we select the result that appeared maximum times from the trained models of each domain. Among all these results, we select the one that appeared maximum times and set the corresponding domain as the predicted domain. Following the same way, the fusion method [19] predicts the domain by the maximum value of weight probabilities.Table 1
Evaluation based on CD (the accuracy of predicting correct domains).
MethodsCIFAR-10 (%)CIFAR-100 (%)Mini-ImageNet (%)EuroSAT (%)Intel image classification (%)Maximum appeared method79.2140.9350.3855.0453.26Fusion method [19]71.7940.3261.4690.9183.03CMS-CMM85.5971.2773.6592.4865.70CMS-CMM-opt95.7877.6375.5299.0990.62We used CD (the accuracy of predicting correct domains) to evaluate the methods. As we can see in Table1, our CMS-CMM-opt achieves higher accuracy than the existing methods, which is 16.62% higher on average. Furthermore, CMS-CMM-opt achieves higher accuracy than our CMS-CMM, which proves the efficiency of the optimization.


## 3.4. Evaluation of Label Classification

Our final objective is to classify the samples. Thus, based on the prediction of domain, there must be also the classification of the samples at the following step. Thus, on a testing sample, only when a method correctly predicted the domain and label at the same time, we admit this method correctly output the result. For example, the label 9 of CIFAR-10 and the label 9 of CIFAR-100 mean different kinds of objects.We used CDCL (the accuracy of predicting correct domains and correct labels) to evaluate the methods. As we can see in Table2, our CMS-CMM-opt achieves higher accuracy than the existing methods, which is 14.01% higher on average. Compared with the domain prediction, the increase of the accuracy is reduced from 16.62% to 14.01% because there is also error when predicting the labels. CMS-CMM use the fusion method [19] as the prediction of labels after the domain prediction. We can see CMS-CMM-opt also achieves higher accuracy than CMS-CMM, which is 11.25% higher on average.Table 2
Evaluation based on CDCL (the accuracy of predicting correct domains and correct labels).
MethodsCIFAR-10 (%)CIFAR-100 (%)Mini-ImageNet (%)EuroSAT (%)Intel image classification (%)Maximum appeared76.2837.7548.0054.0751.50Fusion method [19]69.9838.2259.2688.9380.87CMS-CMM80.3057.9966.3789.4663.27CMS-CMM-opt89.2170.7471.3995.4386.86


## 3.5. Evaluation of the Scalability

In this subsection, we do research about the scalability of our framework based on the metric of CDCL (the accuracy of predicting correct domains and correct labels). We added the domain one by one and computed the label classification as Table3 shows.Table 3
Evaluation of domain scalability based on CDCL.
MethodsCIFAR-10 (%)CIFAR-100Mini-ImageNetEuroSATIntel image classification1 domain92.76————2 domains89.2172.43%———3 domains89.2172.28%72.13%——4 domains89.2171.12%71.62%95.43%—5 domains89.2170.74%71.39%95.43%86.86%As we can see in Table3, the accuracy of CIFAR-10 remains the same as the number of domains becomes big. On the other side, the accuracies of CIFAR-100 and Mini-ImageNet becomes lower. The accuracy of each models plays important role to the classification accuracy. The other important factor to the accuracy is the similarity between domains, which will be introduced in the next subsection.


## 3.6. Impact between Domains

We can analyse the impact of a domain to other domains as Table4 shows. In this table, we compared the CDCL (the accuracy of predicting correct domains and correct labels) of 5 domains with that of 4 domains, which means we drop one domain to evaluate the relation between this domain and other ones. When we drop CIFAR-10, we found the accuracy of CIFAR-100 is increased more than that of others. By the same way, we can find relations between domains. When there are similar labels between domains, the prediction of domain and labels may easily go wrong. For example, the label “fox” of CIFAR-100 is similar to the label “white fox” of Mini-ImageNet. Thus, how to consider the similarity between datasets are important to the increase of accuracy.Table 4
Impact between domains based on CDCL.
MethodsCIFAR-10CIFAR-100Mini-ImageNetEuroSATIntel image classificationWithout CIFAR-10—+2.68%+1.63%+0.00%+0.00%Without CIFAR-100+2.79%—+8.13%+0.48%+1.32%Without Mini-ImageNet+0.00%+0.15%—+0.03%+4.32%Without EuroSAT+0.00%+1.16%+0.52%—+0.00%Without intel Image classification+0.00%+0.38%+0.23%+0.00%—


## 3.7. Evaluation on the Number of Models

In this subsection, we evaluate the relation between the number of models and CDCL (the accuracy of predicting correct domains and correct labels). We set the number of models is from 2 to 6. As there may be different combinations of models, we evaluate the average accuracy of these combinations. As we can see in Figure5, the accuracy of all datasets increases as the number of models becomes bigger. On the other side, when the number of models is 6, the accuracy of some datasets becomes lower than that of 5 models. When there are low accurate models, these may lower the classification accuracy of our framework. The proper number of models can be computed by the validation set.Figure 5
The evaluation on the number of models based on CDCL.


## 3.8. Evaluation by More Metrics on All Testing Samples

As we can see in Table5, our methods achieved better performance in the most of metrics. In the CDWL (the percentage of predicting correct domains and wrong labels) case, the percentages of our methods are higher than those of the other methods. This is because our methods can predict more correct domains, which may cause more wrong labels. Compared with the fusion method [19], our method can increase 19.14% of CDCL while only increasing 9.01% CDWL.Table 5
Evaluation by more metrics on all testing samples.
MethodsCD: correct domainsCDCL: correct domains and correct labelsCDWL: correct domains and wrong labelsWD: wrong domainsWDCL: wrong domains and correct labelsWDWL: wrong domains and wrong labelsMaximum appeared55.98%53.49%2.49%44.02%32.36%11.66%Fusion method [19]62.85%60.72%2.13%37.15%25.13%12.02%CMS-CMM77.70%70.35%7.35%22.30%15.51%6.79%CMS-CMM-opt91.00%79.86%11.14%9.00%5.97%3.03%


## 3.9. Evaluation of Execution Time and Memory Consumption

Table6 shows the total execution time and maximum memory consumption of each trained model on the corresponding dataset. We use Tesla K80 of NVIDIA [41] to run the models. In more detail, we use Tesla K80 of NVIDIA to run the model VoVNet-57 on CIFAR-10 and record the total execution time and maximum memory consumption of this model, which is shown in this table. Then, we also can use Tesla K80 of NVIDIA to run the model VoVNet-57 on the other datasets and record the total execution time and maximum memory consumption of this model. By the same pattern, we can run the other models on each dataset and record the total execution time and maximum memory consumption of these models.Table 6
Execution time and memory consumption.
MethodDatasetThe execution time (second)Max memory consumption (MB)CIFAR-10CIFAR-100Mini-ImageNetEuroSATIntel image classificationVoVNet-57106.05104.95475.3560.2046.63136.25Res2Net50105.20396.89474.8059.3744.0491.01RepVGG-A034.6735.5545.6920.7015.9330.35DenseNet12185.24376.06451.0348.9136.6527.52VGG16122.20121.59495.9772.7955.64513.73ResNet5080.16373.67449.1446.7835.0690.46CMS-CMM-opt in serial604.751479.952472.50369.07294.28687.04CMS-CMM-opt in parallel222.52497.24600.29164.02146.87839.29Our methods generate multiple models on each dataset, which causes the runtime our methods become bigger than those of using single model.CMS-CMM-opt in serial runs the model one by one, which causes the execution time equals to the following: the execution time of single model × the number of models + the execution time of our fusion process. In more detail, our CMS-CMM-opt in serial run multiple models (one after one) on CIFAR-10. Then we run our fusion method. During these processes, we record the total execution time and maximum memory consumption that is shown in Table 6.A simple solution to reduce the execution time is that we can use less models but this may lower the accuracy. To further reduce the execution time without lowing the accuracy, we run the models on the distributed computational nodes of a cluster based on the parallel pattern of paper [42]. In more detail, CMS-CMM-opt in parallel utilizes multiple computational nodes where each node has a Tesla K80 of NVIDIA [41]. As each node can run the model at the same time, the total execution time can be reduced. The total execution time is recorded as the end of all nodes and the maximum memory consumption is counted as the maximum one among these nodes. As CMS-CMM-opt in parallel of Table 6 indicates, the total runtime is reduced compared with CMS-CMM-opt in serial. The additional execution time is caused by the communication and our fusion process. The additional memory consumption is caused by the buffers of communication and our fusion process.To achieve higher accuracy by single model on multiple domains, the structure becomes deeper and more complex which causes the memory consumption becomes bigger. For example, the V-MoE of Google achieves high accuracy with a trained model of 15 billion parameters on the ImageNet [43]. Compared with super model solution, our framework is more scalable.


## 3.10. Illustration of the Domain and Label Classification

Firstly, we use an example to explain the domain classification. As Figure6 shows, we select the testing samples of label 1 that belongs to domain CIFAR-10. Then, we run all of the models on this sample and compute the average difference between the outputs of trained models that belong to the same datasets. Figure 6(a) is the difference of the outputs between the models that are trained on CIFAR-10. Figures 6(a)–6(e) are the differences of the outputs between the models that are trained on CIFAR-100, Mini-ImageNet, EuroSAT, and Intel image classification. As the testing samples belong to the CIFAR-10, the difference between the trained models of CIFAR-10 is obviously smaller than those of the other datasets. Thus, the domain classification based on model difference is reasonable.Figure 6
The difference between the output of trained models. The ground truth is label 1 sample of domain 0. (a) The models trained on CIFAR-10. (b) The models trained on CIFAR-100. (c) The models trained on Mini-ImageNet. (d) The models trained on EuroSAT. (e) The models trained on Intel image classification.
(a)(b)(c)(d)(e)We present the statistical result of model difference by the following. For a sampleSn that belongs to the domain Du, we use avgDMSn∈Du to present the average difference between the trained models of Du as below:(13)avgDMSn∈Du=∑Sn∈Du,u,j≠0‖M0,uSn−Mj,uSn‖Lk∑Sn∈Du,u,j≠0,Lk1,where M0,uSn−Mj,uSnLk is defined by (2), M0,uSn is the output of highest accurate model, and Mj,uSn is the output of the other model. For a sample Sn that belongs to the domain Du, we use avgDMSn∉Dv to present the difference between the models of Dv (v≠u) as below:(14)avgDMSn∉Dv=∑Sn∈Du,v,j≠0‖M0,vSn−Mj,vSn‖Lk∑Sn∈Du,v,j≠0,Lk,1,where M0,vSn is the output of highest accurate model on the validation samples of Dv, Mj,vSn is the output of the other model the validation samples of Dv. As we can see in Table 7, avgDMSn∈Du is obviously smaller than avgDMSn∉Dv on each dataset that means we can use this value the predict the domain. Based on this analysis, our methods further optimize the prediction of domain. CIFAR-100 and Mini-ImageNet has bigger number of labels than the other datasets, which causes the model difference bigger than that of the other datasets.Table 7
Statistical analysis of model difference.
MethodsCIFAR-10CIFAR-100Mini-ImageNetEuroSATIntel image classificationavgDMSn∈Du0.30930.72160.64680.31950.2487avgDMSn∉Dv0.48601.2111.56310.58410.3040Secondly, we use three examples to explain the label classification based on the trained models of corresponding domain. In the first example, we select the testing samples of label 1 from CIFAR-10. Then, we run the trained models of CIFAR-10 on these samples. As Figure7(a) shows, the average probability of label 1 by each model (trained on CIFAR-10) is obviously higher than those of the other labels. In Figure 7(b) case, we select the label 9 testing samples from CIFAR-100. Then, we run the trained models of CIFAR-100 on these samples. In Figure 7(c) case, we select label 90 testing samples from Mini-ImageNet. Then, we run the trained models of Mini-ImageNet on these samples. All of these cases show that the average probability of ground truth label is obviously higher than those of the other labels when we correctly select the trained models of corresponding dataset. Thus, the classification based on the probability of labels is reasonable.Figure 7
The probability of labels by different models of corresponding domain. (a) The ground truth is label 1 of domain 0. (b) The ground truth is label 9 of domain 1. (c) The ground truth is label 90 of domain 2.
(a)(b)(c)We present the statistical analysis of label probability by the models. For a sampleSn that belongs to the domain Du, we define the average probability of ground truth label by the trained models of Du as below:(15)avgPMi,uSn=Lgroundtruth=∑i,u,SnPMi,uSn=Lgroundtruth∑i,u,Sn,1,where PMi,uSn=Lgroundtruth is introduced in equation (1). For a sample Sn that belongs to the domain Du, we define the average value of maximum probability of other label by the models as below:(16)avgPMi,uSn≠Lgroundtruth=∑i,u,SnMaximumLk≠LgroundtruthPMi,uSn=Lk∑i,u,Sn1.As we can see in Table8, avgPMi,uSn=Lgroundtruth is obviously bigger than avgPMi,uSn≠Lgroundtruth on each dataset that means we can use this value to predict the label. Based on this analysis, our methods further optimize the prediction of label.Table 8
Statistical analysis of predicting label.
MethodsCIFAR-10CIFAR-100Mini-ImageNetEuroSATIntel image classificationavgPMi,uSn=Lgroundtruth0.81010.58940.70740.84910.8588avgPMi,uSn≠Lgroundtruth0.06590.07490.05780.05520.0950


## 3.11. Introduction of the Employed Acronyms

We use Table9 to give the introduction of the employed acronyms in this article for reader’s convenience.Table 9
Introduction of the employed acronyms.
NumAcronymsIntroduction1CIFAR-10Dataset that is introduced in [32, 33].2CIFAR-100Dataset that is introduced in [34, 35].3Mini-ImageNetDataset that is introduced in [36, 37].4EuroSATDataset that is introduced in [38, 39].5Intel image classificationDataset that is introduced in [40].6SnPresents a sample.7LkPresents a label.8GnPresents the ground truth onSn.9CMS-CMMOur framework that is introduced in subsection3.310CMS-CMM-optOur framework that is introduced in subsection3.411Tesla K80NVIDIA GPU that is introduced in subsection 4.912CMS-CMM-opt in serialOur serial execution that is introduced in subsection 4.913CMS-CMM-opt in parallelOur parallel execution that is introduced in subsection 4.914VoVNet-57A deep learning model that is introduced in [20].15ResNeSt50S deep learning model that is introduced in [21].16RepVGGA deep learning model that is introduced in [22].17DenseNetA deep learning model that is introduced in [23].18VGG16A deep learning model that is introduced in [24].19ResNetA deep learning model that is introduced in [25].20CDThe accuracy of predicting correct domains21CDCLThe accuracy of predicting correct domains and correct labels22CDWLThe percentage of predicting correct domains and wrong labels23WDThe percentage of predicting wrong domains24WDCLThe percentage of predicting wrong domains and correct labels25WDWLThe percentage of predicting wrong domains and wrong labels


## 4. Conclusions

In this article, we have introduced a novel framework that achieves the scalability of classification by using multiple models. Different from the existing single super model methods, our framework lowers the consumption of computational resource and achieves good scalability at the same time. Furthermore, we solve the problem of existing fusion methods. Our framework can be a good solution for the applications, which has to classify more domains of samples.In the future work, we will do research about how to solve the problem of similarity between domains and labels. In some cases, the similarity is caused by the similar labels of different domains like the “fox” in CIFAR-10 and “white fox” in CIFAR-100. In other cases, this may be caused by the similar features between labels of the same domain, which is related to the accuracy of models. We believe that these factors are the key of increasing the accuracy of classification.



---

*Source: 5578043-2022-09-25.xml*