Change Detection for bi-temporal images classification based on Siamese Variational AutoEncoder and Transfer Learning
Authors: Chouikhi Farah
Ali Ben Abbes
Imed Riadh Farah
Venue: Submitted to ICLR 2023 (2023)
Keywords: Feature extraction
Variational Auto-Encoder
Change Detection
Siamese structure
Transfer Learning
Desertification

Abstract: Siamese structures empower Deep Learning (DL) models to increase their efficiency by learning how to extract the relevant temporal features from the input data.  In this paper, a Siamese Variational Auto-Encoder (VAE) model based on transfer learning (TL) is applied for change detection (CD) using bi-temporal images. The introduced method is trained in a supervised strategy for classification tasks. Firstly, the suggested generative method utilizes two VAEs to extract features from bi-temporal images. Subsequently, concatenates them into a feature vector. To get a classification map of the source scene, the classifier receives this vector and the ground truth data as input. The source model is fine-tuned to be applied to the target scene with less ground truth data using a TL strategy. Experiments were carried out in two study areas in the arid regions of southern Tunisia. The obtained results reveal that the proposed method outperformed the Siamese Convolution Neural Network (SCNN) by achieving an accuracy of more than 98%, in the source scene, and increased the accuracy in the target scene by 1.25% by applying the TL strategy.

Under review as a conference paper at ICLR 2023
CHANGE DETECTION FOR BI-TEMPORAL IMAGES CLASSIFICATION BASED ON SIAMESE VARIATIONAL AUTOENCODER AND TRANSFER LEARNING
Anonymous authors Paper under double-blind review
ABSTRACT
Siamese structures empower Deep Learning (DL) models to increase their effi- ciency by learning how to extract the relevant temporal features from the input data. In this paper, a Siamese Variational Auto-Encoder (VAE) model based on transfer learning (TL) is applied for change detection (CD) using bi-temporal im- ages. The introduced method is trained in a supervised strategy for classification tasks. Firstly, the suggested generative method utilizes two VAEs to extract fea- tures from bi-temporal images. Subsequently, concatenates them into a feature vector. To get classification map of the source scene, the classifier receives this vector and the ground truth data as input. The source model is fine-tuned be applied to the target scene with less ground truth data using TL strategy. Experiments were carried out in two study areas in the arid regions of southern Tunisia. The obtained results reveal that the proposed method outperformed the Siamese Convolution Neural Network (SCNN) by achieving an accuracy of more than 98%, in the source scene, and increased the accuracy in the target scene by 1.25% by applying the TL strategy.
INTRODUCTION
The feature extraction step in the classification process allows improving DL model performance in
several fields (Hakak al.] 2021 Islam & Nahiduzzaman 2022 Xiong & Zuo 2022). In fact, Con-
volutional neural network (CNN) has been efficiently employed to solve computer vision problems in variety of fields including industry, environment, and healtheare (Alzubaidi et al. 2021 Huang let al. 2022). Nevertheless, the performance of the algorithms depends of the used datasets. Fur- thermore, CNN has shown low performance in the classification task thanks to the high similarity and non-dispersity of the input data. Recently, with these challenging, the VAE has demonstrated its good performance the classification tasks as based on distribution-free assumptions and nonlinear approximation (Zerrouki et al., 2020) Ran et al. 2022). However, the periodicity of the input data reduces its efficiency and, therefore, makes it unable to ensure the temporal consistency of the extracted features (Zhao & Peng (022). Moreover, traditional DL models (c.g. CNN, VAE, etc.) cannot capture the temporal information. Thus, they have limited capability to extract the temporal features. To overcome this shortcoming, the Siamese structure, which is one of the best approaches for CD in bi-temporal images, can be a good solution. Siamese networks were first utilized for sig- nature verification. Subsequently, they were applied in feature matching, particularly between pairs of images (Ghosh et al., [202T] Zhang et al., 2022). Recent studies focusing on classification tasks have employed bi-temporal images for CD (Lee et al. 2021, Zheng al.] 2022). The CD process consists in identifying the differences between bi-temporal images of the same geographic location undergoing anthropic and climatic factors Exploring the generalization of Siamese DL. models is a key challenge. Discussing its TL capabilities is one of the most popular analyses (Krishnamurthy ct al. 2021 Abou Baker et al. (022). The TL aims at gaining knowledge by solving a problem and applying it to another related problem. The use of TL in practice is to apply knowledge from one context with several labeled data to another situation with limited labels. In application, TL consists in re-using the weight values of the trained model with source data, while applying fine-tuning approach to provide a model adapted to the target data (Rafeel et al., 2020] [hhabiir et al. 2021 Toseef et al. 2022). By employing the pre-trained model source the target scene adapter instead

---

Under review as a conference paper at ICLR 2023
of starting from the scratch, the fine-tuning technique reinforces learning and considerably reduces
the model overfitting (Tan et al. 2018 Cao et al., 2022). The contributions of the present work are presented below:
Proposing a new method for bi temporl images classification based on Siamese VAE, in order to extract the relevant temporal features.
Using a TL strategy to transfer the pre-traine pre-trained Siamese VAE from the source to the target scene.
Evaluating the introduced method w.r.t. SCNN, in two study areas, using bi- bi-temporal multispectral images aequired with Landsat.
The rest of this manuscript is organized follows. Some related works are described in section 2 The developed technique and the background of the Siamese VAE and the TL strategy used in this study are presented in section [ Section[4eepictt the experimental settings, the implementation details and the applied evaluation metrics. The obtained results are provided and discussed in section while Section [ concludes the paper and gives some future perspectives.
5
2 RELATED WORK
Recently, numerous studies have focused on Siamese structure using bi-temporal images to enhance feature extraction-based classification models for CD. For example, Zhu et al. (2022) have designed a Siamese global learning (Siam-Gl)) GL) framework for high spatial resolution (HSR) remote sensing images. The Siamese structure has been used to improve the feature extraction of bi-temporal HSR remote sensing images. Researchers have concluded that the Siam-GI. framework outperformed the advanced semantic CD methods as it provided more pixel data and ensured high precision classifica- tion. Besides Zhao & Peng (2022) have presented a semi-supervised technique relying on VAE with the Siamese structure to detect changes in Synthetic Aperture Radar (SAR) images by concatenating the extracted features of bi-temporal images in a single latent space vector to extract the pixel change characteristics. Moreover, Daudtt et al. (2018) proposed CD framework based on Siamese CNN for CD. In the suggested method, Sentinel-2 multispectral pair images were encoded via a Siamese network to extract new data representation. Then, the extracted bi-temporal representations were combined to produce an urban CD map. The designed network was trained in a fully supervised manner and showed excellent test performance.
Indeed, the performance of a DL model can be enhanced applying TL strategy, especially CD models based on Siamese structures. We list, in this paragraph, some research works relying on Siamese CD and TL. For instance, Yang et al. (2019) proposed DL-based CD framework with TL strategy to apply a learned source CD model training and fine-tuning steps, while the target domain is used to fit the source domain modification concept. method of image difference the target domain was utilized to pick pixels having a high probability of being correctly classified by an unsupervised technique, which improved the change detection network (CDN) of the target domain. The provided findings showed that the developed method outperforms the state-of-the-an art CD techniques and offers an excellent ability to transfer the concept of change from the source domain to the target domain. Heidari & Fouladi-Ghaal (2020) presented face recognition platform based on TL in Siamese network made up of two identical CNNs. On this platform, the Siamese network extracts the features from the input pair of images and determines whether they belong to the same person or not. The experimental results revealed that the accuracy of the proposed model (5.62%) is better than that of the state-of-the-arn methods of face recognition. [aandara & Patel (2022) designed a transformer-batt based Siamese network architecture (ChangeFormer) for CD by considering pair of co-registered remote sensing images. In the proposed Siamese network, convolutional networks (ConvNets) were used to transform the multiscale information into long-range data needed for CD. The experimental findings demonstrate that the developed end-to-end trainable architecture outperforms the existing ones as enhances the CD performance. Andresini al.) (2022) suggested Siamese network trained utilizing labeled imagery data of the same land scene acquired by Sentinel-2 at various times to detect changes in land cover in bi-temporal images. The trained Siamese network in labeled scene was transferred to a new unlabeled scene applying a fine-tuned TL strategy. The lack of change labels in the new scene was addressed by estimating pseudo-change labels in unsupervised manner. The experiment
target domain. The introduced framework includes pre-
2

---

Under review as a conference paper at ICLR 2023
conducted in Cupertino and Las Vegas showed that the proposed strategy performs better than the standard Siamese networks trained either a supervised or unsupervised manner.
METHOD
consider two bi-temporal image scenes with two different pixel grids having different spatial locations: X, the source and X₁ the target. We also take into account the matrices C, and Ca containing information about the ground truth classes available for pixels in the scenes X, and X,, respectively. Note that the size of C₁ is always smaller than that of C,- The Siamese VAE method classifies the detected changes in two steps a supervised manner. Siamese network trained, in the first step, using supervised classification task depending temporal image source scene X, and ground truth C.. Then, the second step consists in deploying the trained Siamese VAE the bi-temporal image target scene X, and using the labeled data of C employing fine-tuning approach. The schema of the proposed methodology is illustrated in Figure
the type of CD method for bi-
Sinmese VAE Learning (I)
Source Scene X₃
Targef Scene X4
Image
Image
Image
Ground Truth
Source Map
Ground Truth
Target Map Ct
Con cntcuation
Classification
Image 2
3.
30
Target Map
Prediction
Transfer Learning (II)
Figure 1: Flowchart of the proposed method: (I) Siamese VAE used to extract the latent spaces from the source scene of bi-temporal images. These latent spaces are then concatenated to create feature vector, which is used input for the classifier trained strategy was employed to improve the trained Siamese VAE on the bi-temporal source images, (III) The classification map of the target scene was predicted using the target Siamese VAE.
ground truth data. (II) A TL
3.1 VARIATIONAL
BACKGROUND
The VAE is made up of two components (Kingma & Welling 2013), as indicated Figure 3.1 The first one, called the encoder, is clustering part that projects data input X into latent space Z according to a Gaussian probability P(Z]X), with a regularization trick provided by a vector of means and vector of standard deviations However, the second component, named decoder part, generates the original data X as X' from the latent space Z according to a probability distribution Q()

---

Under review as a conference paper at ICLR 2023
Sample
Z
Encoder
Decoder
Figure 2: Variational Auto-Encode model.
The VAE model can be presented by the expressions:
f(x)
(X)
X' g(()
= 1 A o + +E
E NO,
2 ~ NN,,
where is the vector of means, denotes the vector of the standard deviations, and designates small constant equal to the value of the reduced normal distribution N),, Z corresponds to the latent space, X refers to the input image and X' is the reconstructed image. The encoder is f. while the decoder is y. The VAE model aims to minimize the loss function by optimizing the two functions: f (encoder) and (decoder). 7, comprises the regularization loss (K (K),), which is the Bullback-Leible divergence loss, and reconstruction loss (RL).
= RL KL
The explanation for the reconstruction loss RL is:
RT 1/2|X X'
(3)
The explanation for the kullback-Leibler divergence loss K L is:
1 Σ (log (-of)+ +og n²² + log 0² + 1)
KL D)L(((()))())()) = DKL(N(,,0
2
(4)
And by incorporating (Eq3) and (Eq4) into (Eq2), our model's loss function L. becomes:
= 1/2
x -
2 Σ (log (-o²) log μ² + log 0² + 1)
(5)
The term n represents the hidden space dimension n=dim(Z).
3.2 STAMESE VAE BACKGROUND
A Siamese network is an architecture containing two parallel neural networks having identical con- figurations with identical parameters and shared weights. The suggested architecture is made up of two VAEs connected by vector of features created by concatenating the two latent spaces of the
4

---

Under review as a conference paper at ICLR 2023
pair VAE and classification component trained with source ground truth map (C₂). The loss
function LS of the developed Siamese VAE is composed of the loss function of VAE1 (L1), the loss function of V VA22 (L2) and the cross-entropy loss (CE) of the classification part.
LS = LI + L2 + CE
(6)
The explanation for the cross-entropy loss CE is:
CE = - E Ti log Y;
Where Y is the result, e is the number of classes, and C' is the ground truth input to the classification component. The loss of our model becomes.
LV AEDeert = - - 2 Σ (log (-o + log 1 + log of + 1)
Σ
log Y;
(8)
3.3 TRANSFER LEARNING
We consider souree domain D, consisting of the source scene X,, containing the ground truth C., and the target domain D. made up of the target scene X, containing the ground truth C, and the unknown knowledge be learned. The knowledge learned in the source domain to the target domain through TL. TL applies the knowledge in D, to help learn the knowledge in D.. In this study, the condition D. * Dr and C. are always smaller, compared to C., the learning process, the classification knowledge of the source scene X, is employed to detect the classification of the target scene X₁. Besides, TL is used to address the classification issue by utilizing the fine- tuning technique that can be divided into freezing and retraining, shown in Figure [3. Firstly, feature extraction and classification are applied using pre-trained Siamese VAE on the source scene. Then, the target Siamese VAE model is initialized with the saved weights of the source model which is the freezing part. Finally, the ground truth the target scene is used to retrain the target model and predict the classification map.
be transferred
Model
Fine-tuned model
Copy
Source
Target
Figure 3: TL with model fine-tuning adapted (Wang al. 2021)
EXPERIMENTAL EVALUATION
Recently, computer vision techniques have been widely employed in environmental monitoring due to the significant improvements in machine learning algorithms and the increasing availability of free remote sensing data with moderate spatial and temporal resolution. Enhancing the DL models ability to monitor the spatial and temporal distribution of desertification is highly recommended to
5

---

Under review as a conference paper at ICLR 2023
ameliorate combating desertification operations and ensure ecosystem security. In the present work, the introduced Siamese VAE method is applied with TL strategy to detect desertification risk two study areas.
4.1 STUDY AREA AND DATASET
Each of the considered scenes of Landsat images is composed of three bands of the Landsat visible spectrum (RGB) and the normalized vegetation index NDVT. To calculate the Normalized Dif- ference Vegetation Index NDVI, the reflectance of the red (R) and near-infrared (PTR) channels, measured in the visible band, were used. The NDVI calculation formula is written below.
PIR R
NDVI-
PIR + R
Bi-temporal Landsat images were acquired respectively from Menzel Habib, Gabes located in the south of Tunisia, as the source scene, and from Jeffara, Medenine, in the south of Tunisia, as the larget scene from the ClimateEngine website. Figure 4.1 iillustrates the two study areas. Each pair of images was acquired from the Landsat project 2001 and 2004 with spatial resolution of 30 meters. Desertification risk information of both image pairs was utilized to verify the accuracy of produced maps.
$38'0
N-021.35
NUSI
Km
0
5
10
20
N.G0z.ct
W
Source Scene
Target Scene
Figure 4: Study Areas: Source and Target Scenes.
4.2 IMPLEMENTATION DETAILS
The proposed method was implemented using Keras (Ketkar 2017). The categorical cross-entropy loss was used the loss function for the classification (Kingma & Ba 2014), while stochastic gradient descent (Adam) (Kingma & Ba 2014) was employed as the optimizer. The batch size was set to 256 and the number of the training data in the source scene 90 000 in the target scene. The introduced model was applied to train 100 epochs. The learning rate was set 0.001 and the latent space dimension equal to 24, demonstrated in Table
more than 500 000 and about

---

Under review as a conference paper at ICLR 2023
Table 1: Specifics of the grid search considered by each method. The ideal setting for each hyper- parameter is indicated with a bold font.
Method
Searching Range of hyperparameters
Siamese VAE source Scene
learning rate: [0.0001, 0.001, 0.01), latent space dim [2,4,8.16.22432), batch size [64, 244,12,,566,12,100244 epoch [10,50,100,200]
Siamese CNN source Scene
learning rate: [0.0001, 0.001, 0.01]. hidden space dim 12,4,8,16224.3, batch size [64, 244,1,22556,,12,10444 epoch [10,50, 100,200]
4.3 EVALUATION METRICS
Desertification detection results were compared with the reference data per pixel to assess quanti- tatively the efficacy of the developed method in terms of Accuracy, recall (R), precision (P) and F1 score. The classification Accuracy was obtained by dividing the total number of correct pre- dictions by the total number of predictions. However, recall measured the number of positive class predictions made out of all positive examples, as demonstrated in 10.
TP
R
(10)
TP
where true- sru-poostivee (TP) and true-negative (TN) denote the correct number of changed and un- changed correctly detected pixels. In fact, false-positive (FP) and false-negative (FNN denote the number of the changed and unchanged incorrectly detected pixels, respectively. On the other hand, precision quantifies the number of positive class predictions, as revealed in Eq.
TP
P FP
(11)
The F1 score is a comprehensive evaluation index expressed by the average harmonic value of precision and recall, illustrated in E2.
Fl - score = 2 * Precision * Recall
(12)
Precision + Recall
Generally speaking, method is considered more efficient if it provides higher Accuracy, P. R. and F1 score. Since the Accuracy and the F1 score are the most important metrics the multi- classification task, they were used in the experiments select better deep learning-based methods.
RESULTS AND DISCUSSION
5.1 COMPARATIVE METHOD EVALUATION
The Siamese VAE was compared to SCNN, with and without the application of TL strategy, to demonstrate better efficiency. In the conducted experiments, the same number of training samples was used. More than 500000 labeled pixels from the source scene and around 90000 labeled pixels were extracted from the target scene to make up the ground truth data. of the Siamese VAE and SCNN applications on the ground truth data. The obtained results show that the two methods (Siamese VAE and SCNN) perform well in desertification detection, the standard deviations were stable and only varied within small range. More precisely, the Siamese VAE provided the best Accuracy R, and Fl-scores in source and target scenes. It also ensured better feature extraction thanks to: 1) the normal distribution (Gaussian) of the latent space by the VAE, and 2) the spatial consistency conservation of the extracted features obtained by the Siamese structure.
the outcomes
5.2 DESERTIFICATION DETECTION OUTCOMES
The distribution of desertification (Figure 5.2 along the two study areas during the period from 2001 to 2004 demonstrates that the desertification areas (high and very high risk) were primarily located
7

---

Under review as a conference paper at ICLR 2023
Table 2:
Quantitative results the Gabes dataset.
Model
Accuracy
Precision
Recall
Fl-score
Siamese VAE Source
98,16
98,15
98,12
98,14
Siamese VAE Target with TL
92,41
91,29
91,06
90,84
Siamese VAE Target without TL
91,16
89,64
89,60
89,62
SCNN Source
96,98
96,85
96,55
96,70
SCNN target with TL
90,07
88,41
88,82
88,61
SCNN target without TL
89,11
88,14
89,83
88,90
in the lowland regions characterized by intensive agricultural activities, In fact, the latter together with unfavorable meteorological conditions have intensified the desertification risk in the two study areas. The most important agricultural activity in the two studied regions was rainfed and irrigated olive. The source scene's 's desertification could be more effectively detected using the Siamese VEA approach. The desertification detection was enhanced and refined in the target scene by applying the TI. strategy.
N..0.1..2
N.0.2..08
(a)
9'360"E
(c)
10'30'0"E
(e)
(d)
(f)
Km
Low
High Risk
Very High Risk
Figure 5: Desertification detection outcomes: (a): Siamese VAE prediction in source Scene, (b):
SCNN prediction in Source Scene, (c): Siamese VAE prediction in Target Scene with TI,(d): SCNN prediction in Target Scene with TL (e): Siamese VAE prediction in Target Scene without TL, (f): SCNN prediction in Target Scene without TL.
5.3 THE ADVANTAGES AND LIMITATIONS
In two dry areas of southern Tunisia, computer vision based on the Siamese DL model and TL. strategy were used to create desertification risk map with high spatial and temporal resolution from bi-temporal Landsat images. The obtained findings demonstrate that the developed method was efficiently employed to detect the land surface change and monitor desertification. Obviously, TL was more sophisticated than a single method Siamese VAE or SCNN and it showed higher performance in the target the accuracy of both Siamese methods was less than 93%, depending on the quantity and quality of the labeled data. These results may impede the identification of the desertified areas and alter
with less ground truth data. It is also clear that, in the target scene,
8

---

Under review as a conference paper at ICLR 2023
the outcomes of desertification monitoring. To produce more reliable results, In future studies, the labeled data in the target scene should be improved in quantity and quality.
CONCLUSION AND OUTLOOK
This work introduced classification method based on Siamese VAE and TL for CD in bi-temporal landsat images. The Siamese network trained in a labeled source scene was transferred to the bi-temporal image of the target scenes with less labeled data using fine-tuning technique. Two datasets, including Landsat bi-temporal images collected in Tunisia's southern, arid regions of Gabes and Medenine, were processed. The experimental results revealed that the proposed method is more efficient than SCNN in terms of CD and classification. As future research perspectives, the appro- priate classification the target scene can be investigated by applying an active learning approach for the optimal selection of labeled data. Additionally, we intend to test the performance of the suggested method when using more input data and applying an active learning strategy improve the quality of the ground truth data in the target scene.
ACKNOWLEDGMENTS
We want thank the Institute of Arid Regions of Medenine, Medenine, Tunisia, laboratory LESOR (laboratory of economics and rural societies), for providing the ground truth data.
REFERENCES
Nermeen Abou Baker, Nico Zengeler, and Uwe Handmann. transfer learning evaluation of deep neural networks for image classification. Machine Learning and Knowledge Extraction, 4(1): 22-41, 2022.
Laith Alzubaidi, Jinglan Zhang, Amjad Humaidi, Ayad AI-Dujaili, Ye Duan, Omran Al-Shamma, José Santamaría, Mohammed A Fadhel, Muthana Al-Amidie, and Laith Farhan. Review of deep learning: Concepts, enn architectures, challenges, applications, future directions. Journal of big Data, 8(1):1-74, 2021.
Giuseppina Andresini, Annalisa Appice, Domenico Dell'Olio, and Donato Malerba. Siamese net- works with transfer learning for change detection in sentinel-2 images. In International Confer- ence of the Italian Association for Artificial Intelligence, pp. 478-489. Springer, 2022.
Wele Gedara Chaminda Bandara and Vishal M Patel. A transformer-basee siamese network for change detection. arXiv preprint arXiw:2201.01293 2022.
Hailong Cao, Xianjun Xie, Jianbo Shi, Guibin Jiang, and Yanxin Wang. Siamese network-based transfer learning model to predict geogenic contaminated groundwaters. Environmental Science & Technology, 56(15): 11115))111711110077 2022.
Rodrigo Caye Daudt, Bertr Le Saux, Alexandre Boulch, and Yann Gousseau. Urban change detec- tion for multispectral earth observation using convolutional neural networks. In IGARSS 2018- 2018 IEEE International Geoscience and Remote Sensing Symposium, pp. 2115-21121 Icee, 2018.
Souvik Ghosh, Spandan Ghosh, Pradeep Kumar, Erik Scheme, and Partha Pratim Roy. A novel spatio-temporal siamese network for 3d signature recognition. Pattern Recognition Letters, 144: 13-20, 2021.
Saqib Hakak, Mamoun Alazab, Suleman Khan, Thippa Reddy Gadekallu, Praveen Kumar Reddy Maddikunta, and Wazir Zada Khan. An ensemble machine learning approach through effective feature extraction to classify fake news. Future Generation Computer Systems, 117:47-58, 2021.
Mohsen Heidari and Kazim Souladi-Ghaleh Using siamese networks with transfer learning for face recognition on small-samples datasets. In 2020 International Conference on Machine Vision and Image Processing (MVIP), pp. 1-4. IEFF, 2020.

---

Under review as a conference paper at ICLR 2023
Ting Huang, Qiang Zhang, Xiaoan Tang, Shuangyao Zhao, and Xiaonong Lu. A novel fault diag- nosis method based on cmn and 1stm and its application in fault diagnosis for complex systems. Artificial Intelligence Review, 55(2): 1315, 2022.
Md Robiul Islam and Md Nahiduzzaman. Complex features extraction with deep learning model for the detection of covid19 from sean images using ensemble based machine learning approach. Expert Systems with Applications, 195:1 16554, 2022.
Nikhil Ketkar. Introduction to keras. In Deep learning with Python, pp. 97-111. Springer, 2017.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprini arXiv:1412.6980, 2014.
Diederik Kingma and Max Welling. arXiv:1312.6114,0000 2013.
Auto-encoding variational bayes.
arXiv preprint
Surya Krishnamurthy, Kathiravan Srinivasan, Saeed Mian Qaisar, PM Durai Raj Vincent, and Chuan- Yu Chang. Evaluating deep neural network architectures with transfer learning for pneu- monitis diagnosis. Computational and Mathematical Methods in Medicine, 2021:1-12, 2021.
Haeyun Lee, Kyungsu Lee, Jun Hee Kim, Younghwan Na, Juhum Park, Jihwan P Choi, and Jae Youn Hwang. Local similarity siamese network for urban land change detection on remote sensing images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 4139-4149, 2021.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu, al. Exploring the limits of transfer learning with unified text-to-text transformer. J. Mach. Learn. Res., (.r,211(140) (140):1-67, 2020.
Xuming Ran, Mingkun Xu, Lingrui Mei. Qi Xu, and Quanying Liu. Detecting -itor--distrbbution samples via variational auto-encoder with reliable uncertainty estimation. Neural Networks, 145: 199-208. 2022.
Amsa Shabbir, Nouman Ali, Jameel Ahmed, Bushra Zafar, Aqsa Rasheed, Muhammad Sajid, Afzal Ahmed, and Saadat Hanif Dar. Satellite and scene image classification based on transfer learning and fine tuning of resnet50. Mathematical Problems in Engineering, 2021, 2021.
Chuanqi Tan, Fuchun Sun, Tao Kong, Wenchang Zhang, Chao Yang, and Chunfang Liu. A survey on deep transfer learning. In International conference on artificial neural networks, pp. 270-279. Springer, 2018.
Muhammad Toseef, Xiangtao Li, and Ka-Chun Wong. Reducing healthcare disparities using multi- ple multiethnic data distributions with fine-tuning of transfer learning. Briefings in Bioinformatics, 23(3)cba 2022.
Weiping Wang, Zhaorong Wang Zhanfan Zhou, Haixia Deng. Weiliang Zhao, Chunyang Wang, and Yongzhen Guo. Anomaly detection of industrial control systems based on transfer learning. Tsinghua Science and Technology, 26(6):821 883,, 2021.
Yihui Xiong and Renguang Zuo. Robust feature extraction for geochemical anomaly recognition using a stacked convolutional denoising autoencoder. Mathematical Geosciences, 54(3):623 644, 2022.
Meijuan Yang, Licheng Jiao, Fang Liu, Biao Hou, and Shuyuan Yang. Transferred deep learning- based change detection in remote sensing images. IEEE Transactions on Geoscience and Remote Sensing. 57(9):6960-699 2019.
Yacine Zerrouki, Fouzi Harrou, Nabil Zerrouki, Abdelkader Dairi, and Ying Sun. Desertification detection using improved variational autoencoder-basse approach through etm-landsat satellite data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 14: 202-213, 2020.
10

---

Under review as a conference paper at ICLR 2023
Kai Zhang, Shouliang Qi, Jiumei Cai, Dan Zhao, Tao Yu, Yong Yue, Yudong Yao, and Wei Qian. Content-basee image retrieval with convolutional siamese neural network: Distinguishing lung cancer and tuberculosis in ct images. Computers in biology and medicine, 140:105096 2022.
Guangwei Zhao and Yaxin Peng. Semisupervised sar image change detection based on siamese variational autoencoder. Information Processing & Management, 59(1):102726 2022.
Hanhong Zheng, Maoguo Gong, Tongfei Liu, Fenlong Jiang, Tao Zhan, Di Lu, and Mingyang Zhang. Hfa-net: High frequency attention siamese network for building change detection in vhr remote sensing images. Pattern Recognition, 129: 2022.
Qiqi Zhu, Xi Guo, Weihuan Deng, Qingfeng Guan, Yanfei Zhong, Liangpei Zhang, and Deren Li. Land-useeland-cover change detection based on a siamese global learning framework for high spatial resolution remote sensing imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 184:63-78, 2022.


Reviews:
[{"review_id": "qBSJmLgqnnD", "reviewer": "Reviewer_CkES", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Please I will provide the review on Wednesday.", "summary_of_the_review": "Please I will provide the review on Wednesday.", "strength_and_weaknesses": "Please I will provide the review on Wednesday.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "dppyptEVZmP", "reviewer": "Reviewer_G4jm", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper tackle problem of change detection from bi-temporal images by proposing to adapt a Siamese VAE method and Transfer Learning.\n\nThe performance of the proposed method on the bi-temporal image dataset collected in South Tunisia improves over a baseline of Siamese Convolutional Neural Network", "summary_of_the_review": "As there are quite many problems in the presentation of the paper, as well as limited, unclear novelty and reproducibility, I would not recommend this paper to be accepted.\n", "strength_and_weaknesses": "Strengths:\n\nThe paper proposes to adapt Siamese VAE and Transfer Learning on top of a CNN network, which shows improvement over the baseline on a dataset of Landsat visible spectrum and normalized vegetation index collected in Tunisia.\n\n\nWeaknesses:\n\nOverall, the paper presentation is tedious. The Introduction and the Method sections do not clearly present what problem the paper is trying to solve, which creates challenges for readers to understand the context of the paper.\nIn Related Works, the authors did not show the differences between their method and the previous works.\nI am concerned about the limited novelty of the proposed method. The author seems trying to collect several popular methods from related fields into the paper (e.g., Siamese VAE, transfer learning) without explicitly explaining the intuitions, the reasons, and the problem that leads authors to use those methods to fix them.\nAlthough there are improvements reported in Table 2, the gap seems to be small (<2%) on most of the metrics when comparing respective rows (prefixes of Siamese VAE vs. SCNN). Notably, the performances are very high even with target scenes (almost 90%).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "YcxBz1FyTz", "reviewer": "Reviewer_8FMz", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Authors proposed the use of a siamese variational autoencoder (SVAE) for change detection over bi-temporal imagery. The model receives as input to scenes of the same spatial location but collected during different times. The SVAE is optimized using a crossentropy classification loss computed using an image classifier on top of the concatanation of the latent representations of the two input images in addition to  standard reconstruction losses. The model was evaluated for the task of desertification detection using Sentinel 2 imagery in South Tunisia. The model shows better performance than siamese convolutional neural networks for this task.", "summary_of_the_review": "The paper is well written and easy to follow, but need further evaluation to better attest the performance of the proposed framework. Comparison to other approaches using change detection benchmark datasets would improve paper quality.", "strength_and_weaknesses": "-- Strength:\n- Proposed idea is technicaly sound.\n- This work has the potential of being impactfull since accurate change detection approaches are important for multiple earth Observation applications\n\n-- Weaknesses:\nThe work is missing exhaustive and proper evaluation. \n- The proposed approach does not include any wellknown benchmark change detection dataset beyond their own sentinel 2 scenes from South Tunisia. I list below a few baseline datasets to consider. Many more are available in the article by Shi et al in 7.\n- Their results are only compared to a Siamese CNN architecture ignoring multiple other proposed architecture like the one proposed in 1.\n\nChange detection dataset: \n- CDD dataset from 4.\n- Onera Satellite Change Detection dataset from 5\n- xBD dataset from 6\n\n*Related Work to consider:*\n1. Chen, J., Yuan, Z., Peng, J., Chen, L., Huang, H., Zhu, J., ... & Li, H. (2020). DASNet: Dual attentive fully convolutional Siamese networks for change detection in high-resolution satellite images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14, 1194-1206.\n2. Daudt, R. C., Le Saux, B., & Boulch, A. (2018, October). Fully convolutional siamese networks for change detection. In 2018 25th IEEE International Conference on Image Processing (ICIP) (pp. 4063-4067). IEEE.\n3. Mesquita, D. B., dos Santos, R. F., Macharet, D. G., Campos, M. F., & Nascimento, E. R. (2019). Fully convolutional siamese autoencoder for change detection in UAV aerial images. IEEE Geoscience and Remote Sensing Letters, 17(8), 1455-1459.\n4. Lebedev, M. A., Vizilter, Y. V., Vygolov, O. V., Knyaz, V. A., & Rubis, A. Y. (2018). CHANGE DETECTION IN REMOTE SENSING IMAGES USING CONDITIONAL ADVERSARIAL NETWORKS. International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences, 42(2).\n5. Rodrigo Caye Daudt, Bertrand Le Saux, Alexandre Boulch, and Yann Gousseau, “Urban change detection for multispectral earth observation using convolutional neural networks,” in International Geoscience and Remote Sensing Symposium (IGARSS). IEEE, 2018\n6. Gupta, R.; Goodman, B.; Patel, N.; Hosfelt, R.; Sajeev, S.; Heim, E.; Doshi, J.; Lucas, K.; Choset, H.; Gaston, M. Creating xBD: A dataset for assessing building damage from satellite imagery. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA, 16–20 June 2019; pp. 10–17. \n7. Shi, W., Zhang, M., Zhang, R., Chen, S., & Zhan, Z. (2020). Change detection based on artificial intelligence: State-of-the-art and challenges. Remote Sensing, 12(10), 1688.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "R58yeDYVviA", "reviewer": "Reviewer_Zx8V", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a new method for bi-temporal images classification based on Siamese variational autoencoder (VAE) and transfer learning (TL). First, the suggested generative method utilizes two VAEs to extract features from bi-temporal images. Then, the obtained features are concatenated into a feature vector. To get a classification map of the source scene, the classifier receives the concatenated vector and the ground truth data as input. The source model is fine-tuned to be applied to the target scene with less ground truth data using a TL strategy.", "summary_of_the_review": "Based on the comments displayed above, I do not recommend this work to be published in ICLR.", "strength_and_weaknesses": "Strength: The proposed method is reasonable and feasible.\nWeaknesses: However, the novelty of this work is limited. Siamese networks and VAE are common techniques for many applications. The authors only combine them simply to deal with their tasks. I cannot any innovative ideas in the current manuscript. Also, the testing data are remote sensing (RS) images. Nevertheless, the proposed method looks like a general-purpose model. Do the author sure that their method is suitable for RS images?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject