DEVELOPMENT OF DEEP LEARNING
APPROACHES TO EXTRACT INFORMATION
FROM SATELLITE IMAGES

Gopal Singh Phartiyal

<figure><img src="image_1.png" /></figure>

To cite this version:

Gopal Singh Phartiyal. DEVELOPMENT OF DEEP LEARNING APPROACHES TO EXTRACT INFORMATION FROM SATELLITE IMAGES. Engineering Sciences [physics]. IIT Roorkee, 2020.

English. NNT: . tel-04080228

HAL Id: tel-04080228

https://inria.hal.science/tel-04080228

Submitted on 18 Aug 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

Public Domain
DEVELOPMENT OF DEEP LEARNING APPROACHES TO
EXTRACT INFORMATION FROM SATELLITE IMAGES

Ph.D. THESIS

by

GOPAL SINGH PHARTIYAL

<figure><img src="image_2.png" /></figure>

DEPARTMENT OF ELECTRONICS AND COMMUNICATION ENGINEERING
INDIAN INSTITUTE OF TECHNOLOGY ROORKEE
ROORKEE, UTTARAKHAND -247 667 (INDIA)
DECEMBER, 2020
<figure><img src="image_3.png" /></figure>
DEVELOPMENT OF DEEP LEARNING APPROACHES TO
EXTRACT INFORMATION FROM SATELLITE IMAGES

A THESIS

Submitted in the partial fulfilment of the
requirements for the award of the degree

of

DOCTOR OF PHILOSOPHY

in

ELECTRONICS AND COMMUNICATION ENGINEERING

by

GOPAL SINGH PHARTIYAL

<figure><img src="image_4.png" /></figure>
<figure><img src="image_5.png" /></figure>
INDIAN INSTITUTE OF TECHNOLOGY ROORKEE, ROORKEE-247667
ALL RIGHTS RESERVED
<figure><img src="image_6.png" /></figure>
<figure><img src="image_7.png" /></figure>

INDIAN INSTITUTE OF TECHNOLOGY
ROORKEE

STUDENT'S DECLARATION

I hereby certify that the work which is being presented in the thesis entitled "DEVELOPMENT OF DEEP LEARNING APPROACHES TO EXTRACT INFORMATION FROM SATELLITE IMAGES" in partial fulfilment of the requirements for the award of the Degree of Doctor of Philosophy and submitted in the Department of Electronics And Communication Engineering of Indian Institute of Technology Roorkee, Roorkee is an authentic record of my own work carried out during a period from **July, 2015** to **December, 2020** under the supervision of **Dr Dharmendra Singh**, Professor, Department of Electronics and Communication Engineering of Indian Institute of Technology Roorkee, Roorkee, India and **Dr Hussein Yahia**, Professor, Head, project team "GeoStat", INRIA (Institut National de Recherche en Informatique et en Automatique), Bordeaux, France.

The matter presented in this thesis has not been submitted for the award of any
other degree of this or any other Institute.

Dated: December 31, 2020

<figure><img src="image_8.png" /><figcaption>Gopal Singh Phartiyal</figcaption></figure>

This is to certify that the above statement made by the candidate is correct to the
best of my knowledge.

Dr Dharmendra Singh
(Supervisor)

Dated: December 31, 2020

Dr Hussein Yahia
(Supervisor)

<figure><img src="image_9.png" /></figure>
<figure><img src="image_10.png" /></figure>
Jo

My Parents, God and, Friends
<figure><img src="image_11.png" /></figure>
This thesis titled ‘Development of Deep Learning Approaches to Extract Information from Satellite Images’ incorporates the study and critical analysis of the performance of conventional convolutional neural network (CNN) and recurrent neural network (RNN) models, and further development of novel CNN and RNN models to exploit the spectral and/or spatial and/or temporal information present in satellite images for applications such as land cover classification, disaster monitoring and mitigation, and agriculture crop monitoring. The main objective of this thesis is to pursue these studies with unimodal or multimodal satellite images. Special focus during these studies is to develop novel deep neural network (DNN) models (common term for CNN and RNN based models) that can achieve better classification accuracy and generalization performance in mixed land cover class scenarios.

Four tasks are set to pursue these studies. The tasks set are based on: type of satellite image used, application for which the task is realized, multi-modality images involvement, and presence of time series information. The tasks to be carried out are; i) Study and assessment of polarization signatures as potential features for PolSAR image based land cover classification and further development of novel polarization signatures and CNNs based approaches to achieve improved classification performances with PolSAR satellite images, ii) Development of a novel approach using polarization signatures and CNNs for detection of human settlements affected during floods, iii) Development of novel spectral and spectral-spatial CNN models for multisensor (PolSAR and multispectral) images based land cover classification, and iv) Development of novel CNN, RNN, and CNN-RNN models for multisensor (PolSAR and multispectral) satellite time series images based on crop classification.

The satellite image modalities used in these studies are PolSAR and multispectral images because these two satellite image modalities are most common and used popularly. Therefore, development of novel DNN models on these modalities is motivating and beneficial. ALOS PALSAR-2 and Sentinel-1 satellite PolSAR images and Landsat-8 and Sentinel-2 satellite multispectral images are considered in these studies. Since, SAR images are all weather images and are recently the popular satellite image modality, special emphasis is given on study and development of CNN and RNN models to extract information from PolSAR images, in standalone or in combination with multispectral images. Multiple study areas are also considered for testing of the developed models or approaches. The study areas are selected based on presence of diversity in land cover (mixed land cover) and ease of accessibility to ground truth data. Extensive ground truth collection is carried out in the study areas for the development and testing
of the novel models or approaches. Thousands of ground truth samples are collected for each
task throughout the study period. The ground truth samples are either collected directly on the
terrain using GPS or via visual inspection of the satellite images used in different studies and
corresponding Google Earth images.

The thesis consists of seven chapters. **Chapter 1** provides; a brief introduction to satellite images, major challenges involved with traditional satellite image processing techniques, motivation to develop deep learning and DNN based models for satellite image processing, major objectives to be carried out during the research, and framework & organisation of the thesis.

**Chapter 2** provides a brief literature review on the several state-of-the art deep learning and DNN based models and techniques developed for various satellite image based applications. The review includes literature on DNN based techniques developed for single senor/unimodal satellite images, multisensor/multimodal satellite images, and satellite time series images.

The study presented in **Chapter 3** involves a critical analysis of polarization signatures (PSs) as potential features for PolSAR data based land cover classification and development of novel approaches which utilize polarization signatures to provide improved classification performance. The study is divided into three sub tasks i.e. **sub task 1**: critical analysis of PSs as potential PolSAR features, **sub task 2**: development of an optimal class boundary estimator algorithm to fully exploit PSs, and **sub task 3**: development of CNN model to exploit PSs. The first sub task of this study explores PSs and utilizes it for the computation of PSs based features from quad PolSAR data. A land cover (e.g. bare soil) behaves as a complex mixture of different standard scatterers or standard targets and therefore its PS is difficult to interpret. Therefore, there is a need to find correlations between PSs of land covers and PSs of standard targets. These correlations indicate the degree of similarity between the two scatterers. In the direction, polarization signature correlation features (PSCFs) are obtained from PSs. Further, the PSCFs are critically analysed as potential features for PolSAR image based LC classification. The analysis indicated to the significant potential of polarization signatures or PSCFs as features for PolSAR image based land cover classification. In the second and third sub tasks of the study, two different approaches are developed. In particular, both classification techniques are developed to improve classification between tall vegetation and built-up areas because the built-up areas also produce volume scattering response along with double bounce which makes segregation of the two classes challenging in a PolSAR image. In the second sub task of the study, i.e. **sub task 2**, a novel adaptive and optimal class boundary estimator based decision tree classification approach is developed. Image statistics are used to obtain class boundary values
which make the approach robust and adaptive. Genetic algorithms are used to optimize the class
boundary values to achieve maximum overall classification accuracy. Finally, a decision tree
(DT) classifier, which utilizes the optimal class boundaries obtained by using this novel estimator
on PSCFs, is realized. At last, in the third sub task of the study, i.e. *sub task 3*, a second novel
classification approach namely PSs-CNN which utilizes 2D-CNNs as classifier directly with
polarization signatures to perform land cover classification is developed. Since, the PSs are two
dimensional representation of target backscatter response, they can be treated as images of target
response and thus can intuitively be used in 2D-CNN classification framework. The CNNs
captures the spatial context present in a pixel PSs which helps in its classification. For
experimentation, PALSAR-2 fully polarimetric SAR data over Roorkee (Uttarakhand, India)
study area is considered. Five land cover classes are considered namely built-up, dense tall
vegetation such as forests, bare soil, short vegetation such as agriculture crops, and water. Both
novel approaches are compared with other popular methods. The optimal class boundary
estimator based DT classifier reports highest classification accuracy of 75.3% whereas the PSs-
CNN classifier reports 73.2.8% classification accuracy. Classification results from the novel PSs-
CNN model are comparable to the optimal class boundary estimator based DT classifier.
However, the CNN technique is better than the optimal class boundary estimator based DT
classifier because the PSs-CNN classifier is simple, less hand-crafted, and easily scalable.

Satellite PolSAR images are the most reliable, useful, and consistent form of land cover
information during a flood disaster because of its weather independent monitoring capabilities.

**Chapter 4** presents an efficient approach to map human settlements, especially villages in the
rural parts of India that are affected during flood using PolSAR imagery, CNNs, *apriori*
knowledge, and image morphology. Monitoring human settlements in the rural parts of India
during floods is difficult even with PolSAR data. The built-up structures (e.g. houses) in these
human settlements, especially villages of rural India are closely packed and randomly oriented.
Therefore, these villages show volume scattering response along with double bounce scattering
response in a PolSAR image. The mixed scattering response from these villages hinders their
segregation from tall vegetation in a PolSAR image. This challenge is also discussed in chapter
3. Therefore, the detection and mapping of these villages during a flood using PolSAR image is
challenging as there is high risk of false alarms, or missing out a flood affected village during
rescue operations.

The approach involves three stages: -first, built-up area extraction from a PolSAR image,
-second, detection of villages in a built-up area image, and -third, identification and mapping
villages that are affected by flood and masking out villages that are not. In the first stage, a CNN
based novel classifier which utilizes PolSAR polarization signatures (PSs) as features (similar to
as in chapter 3), is proposed to extract built-up areas from a PolSAR image. This novel classifier
uses a window (rather than a pixel) of PSs as input image because the villages are usually large
in area and therefore cover several pixels in a PolSAR image. The window approach helps
suppress the volume scattering present in the backscatter response of the villages and thereby
improves the detection of these villages. However, there is presence of false alarms and true
negatives in the built-up areas extracted and there is a need to mask out these artefacts. Therefore,
the second stage proposes a novel village detection methodology. The methodology detects
whether a particular object in the built-up area image (obtained in the first stage) is a village or
not. A series of morphological operations are developed based on *apriori* knowledge are
employed to suppress false alarms and true negatives. Next, villages that are affected by flood
are required to be mapped and villages that are not affected by flood are masked out. In the third
stage, flood affected villages are identified and mapped and non-flooded villages are masked out
using another image morphology based methodology.

Quantitative and qualitative assessment of the performance of the novel approach over
two different study areas is carried out. Different artificial flood scenarios are created to evaluate
the performance of developed approach. The developed approach is evaluated under these flood
scenarios and flood affected villages are identified and mapped. Finally, number of villages
successfully detected and false alarms are computed with the help of ground truth information
 (actual number of villages or towns in the study regions are known). Results indicate to high
detection accuracy (above 90%), low false alarms and true negatives. The approach is scalable
and potentially satisfactory for real time usage.

**Chapter 5** presents study and analysis of conventional, and further development of novel multisensor satellite image based land cover classification approaches that are based on CNNs. First, the conventional CNN models are analysed and evaluated for multisensor satellite image based land cover classification. It is observed from this analysis that since earth observation satellite images have usually larger spectral depths than normal camera images, conventional CNN models are not sufficient to exploit the high spectral information present in these images. However, exploiting the spectral information is crucial. In fact, similar to spatial domain, convolutions in the spectral domain are intuitive with high spectral dimension data. Convolutions in the spectral domain might also be an alternative to the process of feature selection. Based on this idea, chapter 5 presents two novel multisensor satellite image classification approaches that utilize CNNs in both spatial and spectral domains. The frameworks employ localized spectral convolutions along with localized spatial convolutions on permuted and stacked high
dimensional multisensor images. The study establishes the significance of performing localized spectral convolutions on permuted input in comparison to non-localized or non-permuted localized spectral convolutions. Two models, namely, a permuted local spectral CNN (Perm-LS-CNN) and a permuted local spectral-spatial CNN (Perm-LSS-CNN) are obtained.

For assessment of the performance of models (conventional and novel), PALSAR-2 PolSAR and Landsat-8 multispectral multisensor images over the Roorkee and Haridwar (Uttarakhand, India) study areas are considered. Since, PolSAR and multispectral images provide complementary information and they are widely used together in numerous applications. Five land cover classes are considered namely built-up, tall vegetation such as forests, bare soil, short vegetation such as agriculture crops, and water. The models are trained and tested on ground truth class data points measured directly on the terrain. Classification accuracy and generalization performance of key models are evaluated. Generalization performance is evaluated via visual inspection of classified images on selected well known regions in the study areas. In an attempt to quantize the generalization performance of classifiers, another parameter, AD, which the difference between precision and recall, averaged over all the land cover classes is computed. Comparison with conventional CNN models and other popular machine learning classifiers shows that the Perm-LSS-CNN model although has classification accuracies in the order of 90% (similar to other classifiers), but provides better generalization performance. Both indicators i.e. visual inspection and AD support to the better generalization performance of the Perm-LSS-CNN.

**Chapter 6** presents a study in which an attempt is made; to analyse the conventional CNN and RNN models, and to develop novel CNN and RNN (in standalone as well as synergistic modes) models for agriculture crop classification with multisensor/multimodal satellite time series (MSTS) imagery. For this purpose, two sub tasks are set in this study. In the first sub task i.e. *sub task 1*, conventional CNN and RNN models are analysed and evaluated for the task of multisensor satellite time series (MSTS) images based crop classification. 1D, 2D, and 3D CNNs, RNNs, and combinations of these are considered for analysis. Then, based on the critical analysis of the working and performance of these conventional models, few novel models are proposed and developed in *sub task 2*.

The novel models involve permuted spectral and/or spatial convolutions and/or bi-directional recurrent layers. The Bidirectional RNNs capture the temporal profile and the permuted spectral and/or spatial CNNs capture the spectral and/or spatial profile of a particular crop. Bidirectional strategy in RNNs enables the influence of the past, present, and future inputs on the current output. Bidirectional RNNs are beneficial in time series images based crop
classification because crop classification can utilize the bidirectional process i.e. classification result can be observed from any time node. Also, the permuted spectral band stacking strategy developed in chapter 5 is also used in this study and presumed to improve the significance of the spectral information present in multisensor satellite time series images. In particular, a synergistic utilization of permuted spectral-spatial CNNs and Bidirectional RNNs helps in better exploitation of the multisensor satellite time series images.

Overall, 12 models, 6 conventional models namely; 1D-CNN-v1, 3D-CNN-v1, RNN-v1, 1D-CRNN-v1, 2D-CRNN-v1, and 3D-CRNN-v1, and 6 novel models, namely; Perm-1D-CNN-v1, Perm-3D-CNN-v1, Perm-RNN-v1, Perm-1D-CRNN-v1, Perm-2D-CRNN-v1 (also termed as Perm-LS-Bi-LSTM), and Perm-3D-CRNN-v1 (also termed as Perm-LSS-Bi-LSTM) are developed. For analysis and performance evaluation of the conventional and novel models, multimodal satellite (Sentinel-1 dual-PolSAR and Sentinel-2 multispectral) time series images over the Roorkee study area is considered. Wheat and sugarcane are the major crops grown in the area during the study period. Multisensor time series satellite images over a period of one crop (wheat) cycle (based on heuristic knowledge) are used. Also, to evaluate the robustness of the models under mixed land cover scenarios, land cover classes other than crop types such as forests and built-up areas are also included as output classes. Addition of these classes in the classification process resembles the actual scenario on the ground. Training and testing samples are once again measured and collected directly from the ground. Conventional and novel DNN based crop classification models are evaluated on the basis of classification accuracy and generalization performance. In terms of classification accuracy, Perm-1D-CNN-v1 achieves an accuracy of 99.70% which is the best among the models considered for comparison. In contrast, the generalization performance evaluated via visual inspection, indicated to a superior performance of the proposed Perm-3D-CRNN-v1 and Perm-1D-CRNN-v1 models as these models are able to capture the spectral, spatial, and temporal identity of land cover features more efficiently which results to improved generalization performance of these models.

Finally, **chapter 7** concludes the different studies reported in this thesis. The conclusions drawn from each study, contributions made by the studies and, future scopes of the studies are also listed in this chapter. In summary, based on the studies presented in chapter 3, 4, 5, and 6, it can be concluded that the novel CNN and RNNs models developed in these studies are high performance and scalable and are potentially automatable. The models have shown good generalization performance which makes the models robust and less prone to overfitting and also these models perform well in mixed class scenarios. Further studies are however required to establish these models in the satellite image processing community.
ACKNOWLEDGEMENTS

Now, at this stage I am overwhelmed by the feeling of completing my Ph.D. and don't know how to describe my long journey. I began this journey back in July, 2015 with lots of aims, hopes in my mind. Today, I say, it was roller coaster ride. A plethora of events and emotions went through me that I think I cannot outline in these few paragraphs. Let me start by stating that I could not have reached at this destination or even couldn't have walked this far without the support of so many persons.

I wish to extend by warmest gratitude towards my supervisors, Dr Dharmendra Singh and Dr Hussein Yahia. Their immense knowledge and plentiful experience have encouraged me in all the time of my academic research and daily life. The one person to whom I admire, honour, acclaim, trust, rely and cherish is my honourable supervisor Dr Dharmendra Singh. He has been always there to help me both technically, personally, emotionally, and morally. He cares for everyone in his own way and by all his means. This is his forte that makes him unique in all scores. I am very much thankful for his guidance and inculcating research and personal skills in me. His intuitions and ability to resolve complex problems has always helped me excel my research findings. It is his hunger for research that I could have been able to accomplish my research work in an advanced topic of deep learning for satellite image processing with limited resources in hand. Without his untiring efforts, stimulating discussions, and motivational examples I would not have been writing this page for completion towards my thesis. In contrast, my honourable supervisor, Dr Hussein Yahia was a 'Polar Star' for me during my research journey. He always believed in my research intuitions & ideas and guided me through by appending his expertise, experience, and knowledge. He was as cool as mint irrelevant of the research outcome as he believed in learning rather than achieving. His unconditional support kept me going and timely inputs helped me breakthrough technical challenges. Let me assure you that he is a believer. I would like to extend my warmest gratitude and regards to Dr Dharmendra Singh and Dr Hussein Yahia for a synergistic (fire & ice) guidance during my Ph.D.

I would like to take a minute to admire the efforts of Dr Nicolas Brodu, who played the role of a high speed wheels during my research journey. If not for him, I would have struggled a lot my way here. He is an all-rounder, may it be fundamentals of machine learning or programming languages or it be the state-of-the-art algorithms, he had notion of either. He advocated and recommended me to go for open-source scientific programming such as Python based development of satellite image processing deep learning algorithms. His vision helped me sail through my research goals and overcome conceptual and non-conceptual challenges. I will
always be thankful for his all-round support. I would also like to forward my thanks to Dr Khalid Daoudi for his wonderful company at times and his virtues as a critic which helped me a lot to improve the understanding and representation of my work.

This acknowledgement will be incomplete without mentioning the names of my research committee members; Dr M.V. Kartikeyan (chairman), Dr R. Balasubramanian (external), Dr A. Panaik (internal) for their valuable suggestions and healthy criticism during my research work presentations. I would also like to extend my thanks to all the faculty members and staff members of ECE department, IIT Roorkee for their support and suggestions.

My deepest regards extend to Department of Electronics & Communication Engineering, for providing me the working platform and well equipped laboratory to carry out my research tasks. I want to convey my heartfelt obligations to Dr Dharmendra Singh for providing me a space in his research lab with an ideal research environment.

I will never forget the company I had from my fellow researchers and friends. In particular, I am thankful to my research colleagues from RailTel-IITR Centre of Excellence in Telecom (RICET), Remote Sensing Lab (RSL), Microwave Imaging and Space Technology Applications Lab (MISTAL), and INRIA (Bordeaux, France) labs; Dr Ankita Jain, Dr Shruti Gupta, Dr Neeraj Gill, Dr Varsha Mishra, Dr Akanksha Garg, Dr Bambam Kumar, Dr Prabhat Sharma, Dr Naina Narang, Deepak, Ankush, Arun, Jaydeep, Arun Kant, Ajay, Mandar, Ardhya, Rakesh, Jyothi, Ekta, Vyomika, Priyanka, Anshika, Udeshwari, Kundan, Rohit, Tushar, Kangana, Rishikant and Anas (INRIA, Bordeaux) for the congenial cheering work environment and for the round of discussions altogether sometimes regarding research, sometime social, political and cinematic reviews.

I would like to thank Shrawan and Satendra in the Remote Sensing Lab for his helping hand during the experimental work and his serving tea, snacks which comes as an energy drink during long, tiring study hours.

Nobody has been more important to me in the pursuit of this research than the members of my family. I extend my deepest gratitude to my father Mr. J. S. Phartiyal and my mother Ms. Manju Phartiyal for going through immense hardships to see me fulfil my dreams. I will be indebted to them forever. Their blessings, prayers, and values light my life. They made me believe that education is not only measured by academic titles but by ethics, integrity, kindness and diversity of interests.

I wish to acknowledge my profound appreciation to my brother Mr. Himanshu Phartiyal, my sister Ms. Seema Phartiyal, my in-laws Mr. U. S. Trivedi, Ms. Anita Trivedi, Ms. Annapurna Dixit, and Mr. Shubham Trivedi for their anytime assistance and full support in various ways. It was their patience, encouragement and faith in me that I could complete this thesis. My journey
would not be possible or even instigated without the firm support of my lovely wife Ms. Deepmala Phartiyal who had been my friend and invisible guide, always caring and nurturing when I was stressed or tensed. I admire her for unconditional love and patience during all these years.

Writing these few lines give me utmost pleasure as these lines are dedicated to my friends; Akhil, Nikhil, Vishal, Deepmala, Prateek, Lokesh, Pankaj, Ganesh, Tarun, Avantika, Navin, Eshan, Rakesh, Alaknanda, Shilpi, Hitesh, Sachin, Ashutosh, Bharat, Ayush, Mandakini, Purnima, shambhaivi, Ruchi, Ayushi, Avantika. Without these guys, I would not have been able to withstand the pressure of this journey. Akhil, Nikhil, Vishal, Avantika, Deepmala were the sparkles who kept me cheered up all along. I would like to thank everyone whom I met and was part of journey during this period.

I would like to thank Prabhu, Bhagwaan, Ishwar, and God for keeping a blessing eye on me. This journey would not have been smooth if not for his showers of wisdom.

It would be impossible to acknowledge all the people who contributed either technically or by offering their friendship towards the completion of this work. I am thankful to all those who were associated with my research journey directly or indirectly and to whom I forgot to mention here unknowingly. My thesis has been a wonderful experience and I cherish the educational and personal growth it has provided to me for my future endeavors.

Gopal Singh Phartiyal
TABLE OF CONTENTS

<table><tr><td>ABSTRACT</td><td>XI</td></tr><tr><td>ACKNOWLEDGEMENTS</td><td>XIX</td></tr><tr><td>TABLE OF CONTENTS</td><td>XXIII</td></tr><tr><td>LIST OF FIGURES</td><td>XXVII</td></tr><tr><td>LIST OF TABLES</td><td>XXXIII</td></tr><tr><td>LIST OF ACRONYMS</td><td>XXXVII</td></tr><tr><td>CHAPTER 1 INTRODUCTION</td><td>1</td></tr><tr><td>1.1. BACKGROUND: SATELLITE IMAGES</td><td>2</td></tr><tr><td>1.1.1. Satellite image: Characteristics</td><td>3</td></tr><tr><td>1.1.2. Satellite images: Acquisition types and sensors</td><td>7</td></tr><tr><td>1.2. SATELLITE IMAGES: SUMMARY AND CONTEXT TO THE THESIS</td><td>22</td></tr><tr><td>1.3. SATELLITE IMAGE PROCESSING: RECENT CHALLENGES AND EXPECTATIONS</td><td>23</td></tr><tr><td>1.4. MOTIVATION: DEEP LEARNING IN SATELLITE IMAGE PROCESSING</td><td>26</td></tr><tr><td>1.5. RESEARCH GAPS IN DEEP LEARNING BASED SATELLITE IMAGE PROCESSING</td><td>29</td></tr><tr><td>1.6. THESIS OBJECTIVE</td><td>30</td></tr><tr><td>1.7. FRAMEWORK OF THE RESEARCH</td><td>31</td></tr><tr><td>1.8. ORGANIZATION OF THE THESIS</td><td>34</td></tr></table>

<table><tr><td>CHAPTER 2 BRIEF LITERATURE REVIEW: RECENT TRENDS AND CHALLENGES IN DEEP LEARNING AND DNNS BASED SATELLITE IMAGE PROCESSING</td><td>37</td></tr><tr><td>2.1. BRIEF REVIEW OF RECENT DEVELOPMENTS IN DNNS BASED UNIMODAL SATELLITE IMAGE PROCESSING</td><td>38</td></tr><tr><td>2.2. BRIEF REVIEW OF RECENT DEVELOPMENTS IN DNNS BASED MULTISENSOR SATELLITE IMAGE PROCESSING</td><td>42</td></tr><tr><td>2.3. BRIEF REVIEW OF RECENT DEVELOPMENTS IN DNNS BASED SATELLITE TIME SERIES IMAGE PROCESSING</td><td>45</td></tr><tr><td>2.4. CONCLUSIONS</td><td>47</td></tr></table>

<table><tr><td>CHAPTER 3 CNNS AND POLARIZATION SIGNATURES BASED NOVEL APPROACH FOR LAND COVER CLASSIFICATION WITH POLSAR SATELLITE IMAGES</td><td>49</td></tr></table>
3.1. INTRODUCTION ..................................................................................................................49

3.2. THEORETICAL BACKGROUND .................................................................................................53

3.2.1. Polarization signatures................................................................................................................53

3.2.2. Convolutional neural networks....................................................................................................54

3.3. EXPERIMENT SETUP ....................................................................................................................55

3.3.1. Study area........................................................................................................................................55

3.3.2. Dataset........................................................................................................................................55

3.3.3. Ground truth........................................................................................................................................55

3.3.4. Preprocessing ................................................................................................................................56

3.4. SUB TASK 1: POLARIZATION SIGNATURES: CRITICAL ANALYSIS AS POTENTIAL POLSAR FEATURES FOR LC CLASSIFICATION ................................................................................................................57

3.4.1. Polarization signatures computation: Observed and standard ....................................................57

3.4.2. Polarization signature correlation features or PSCFs .................................................................59

3.4.3. Assessment and critical analysis of PSCFs....................................................................................60

3.5. SUB TASK 2: OPTIMAL CLASS BOUNDARY ESTIMATOR BASED DECISION TREE APPROACH FOR LC CLASSIFICATION................................................................................................................65

3.5.1. PSCF statistics........................................................................................................................................65

3.5.2. Relationship between class boundaries and corresponding PSCF statistics....................................66

3.5.3. Relating classification accuracy and class boundaries: Empirical modelling ....................................67

3.5.4. Optimizing the empirical model........................................................................................................68

3.5.5. Optimal class boundaries based decision tree classifier ................................................................69

3.5.6. Results and discussion ................................................................................................................................71

3.6. SUB TASK 3: DIRECT UTILIZATION OF POLSAR POLARIZATION SIGNATURES VIA CONVOLUTIONAL NEURAL NETWORKS FOR LC CLASSIFICATION................................................................72

3.6.1. Two dimensional convolution over polarization signatures: The concept....................................72

3.6.2. PSs-CNN model architecture design and training........................................................................73

3.6.3. Results and discussion ................................................................................................................................74

3.7. OVERALL RESULTS AND DISCUSSION.................................................................................................76

3.8. CONCLUSIONS .................................................................................................................................................76

CHAPTER 4 NOVEL APPROACH FOR DETECTION OF HUMAN SETTLEMENTS AFFECTED DURING FLOODS USING POLARIZATION SIGNATURES AND CNNS ................................................................................................................................................................................79

4.1. INTRODUCTION....................................................................................................................79

4.2. EXPERIMENT SETUP ....................................................................................................................82

4.2.1. Study area........................................................................................................................................82

4.2.2. Satellite data........................................................................................................................................82

4.2.3. Ground truth........................................................................................................................................82

4.3. PROPOSED APPROACH, IMPLEMENTATION, AND DISCUSSION ................................................................85
<table><tr><td>4.3.1. Land cover polarization signatures</td><td>85</td></tr><tr><td>4.3.2. Stage 1: CNN and polarization signatures based classification model for built-up area extraction</td><td>87</td></tr><tr><td>4.3.3. Detection and mapping of flood affected villages using apriori knowledge and image morphology</td><td>91</td></tr><tr><td>4.4. CONCLUSION</td><td>99</td></tr></table>

CHAPTER 5 PERMUTED SPECTRAL AND PERMUTED SPECTRAL-SPATIAL CNN MODELS FOR POLSAR-MULTISPECTRAL DATA BASED LAND COVER CLASSIFICATION 101

<table><tr><td>5.1. INTRODUCTION</td><td>102</td></tr><tr><td>5.2. THEORETICAL BACKGROUND</td><td>105</td></tr><tr><td>5.2.1. Convolutional neural networks</td><td>105</td></tr><tr><td>5.3. EXPERIMENT SETUP</td><td>105</td></tr><tr><td>5.3.1. Study area</td><td>105</td></tr><tr><td>5.3.2. Dataset</td><td>106</td></tr><tr><td>5.3.3. Ground truth</td><td>107</td></tr><tr><td>5.4. 1D, 2D, AND 3D CNNs FOR LAND COVER CLASSIFICATION WITH POLSAR-MS DATA</td><td>108</td></tr><tr><td>5.4.1. 1D-CNNs for PolSAR-MS data</td><td>108</td></tr><tr><td>5.4.2. 2D-CNNs for PolSAR-MS data</td><td>110</td></tr><tr><td>5.4.3. 3D-CNNs for PolSAR-MS data</td><td>112</td></tr><tr><td>5.5. PROPOSED PERMUTED LOCAL SPECTRAL CNN (PERM-LS-CNN) AND PERMUTED LOCAL SPECTRAL-SPATIAL CNN (PERM-LSS-CNN)</td><td>113</td></tr><tr><td>5.6. RESULTS AND DISCUSSIONS</td><td>116</td></tr><tr><td>5.7. CONCLUSION</td><td>122</td></tr></table>

CHAPTER 6 NOVEL CNN, RNN, AND CNN-RNN MODELS FOR MULTISENSOR SATELLITE TIME SERIES IMAGE BASED CROP CLASSIFICATION 123

<table><tr><td>6.1. INTRODUCTION</td><td>124</td></tr><tr><td>6.2. THEORETICAL BACKGROUND</td><td>127</td></tr><tr><td>6.2.1. Recurrent neural networks</td><td>127</td></tr><tr><td>6.3. EXPERIMENT SETUP</td><td>131</td></tr><tr><td>6.3.1. Study area</td><td>131</td></tr><tr><td>6.3.2. Dataset</td><td>132</td></tr><tr><td>6.3.3. Ground truth</td><td>133</td></tr><tr><td>6.3.4. Preprocessing</td><td>135</td></tr><tr><td>6.4. GENERALIZED DNN FRAMEWORK FOR SATELLITE TIME SERIES IMAGES BASED CROP CLASSIFICATION</td><td>135</td></tr><tr><td>6.5. SUB TASK 1: CONVENTIONAL CNN AND RNN MODELS FOR MULTISENSOR SATELLITE TIME SERIES IMAGE BASED CROP CLASSIFICATION</td><td>137</td></tr></table>
<table><tr><td>6.5.1. 1D-CNN model for MSTS images based crop classification</td><td>137</td></tr><tr><td>6.5.2. 2D-CNN model for MSTS images based crop classification</td><td>137</td></tr><tr><td>6.5.3. 3D-CNN model for MSTS images based crop classification</td><td>138</td></tr><tr><td>6.5.4. RNN model for MSTS images based crop classification</td><td>138</td></tr><tr><td>6.5.5. CNN-RNN models for MSTS images based crop classification</td><td>139</td></tr><tr><td>6.6. THE CONCEPT OF PERMUTED LOCAL SPECTRAL CONVOLUTIONS AND ITS SYNERGISTIC USE WITH RNNs FOR MSTS IMAGES BASED CROP CLASSIFICATION</td><td>142</td></tr><tr><td>6.7. SUB TASK 2: NOVEL CNN AND RNN MODELS WITH PERMUTED LOCAL SPECTRAL CONVOLUTIONS FOR MSTS IMAGE BASED CROP CLASSIFICATION</td><td>143</td></tr><tr><td>6.7.1. 1D-CNN model with permuted local spectral convolution for MSTS Images based crop classification</td><td>143</td></tr><tr><td>6.7.2. 2D-CNN model with permuted input for MSTS images based crop classification</td><td>144</td></tr><tr><td>6.7.3. 3D-CNN model with permuted local spectral convolution for MSTS images based crop classification</td><td>144</td></tr><tr><td>6.7.4. RNN model with permuted input for MSTS images based crop classification</td><td>146</td></tr><tr><td>6.7.5. CNN-RNN models with permuted local spectral convolution for MSTS images based crop classification</td><td>147</td></tr><tr><td>6.8. RESULTS AND DISCUSSION</td><td>150</td></tr><tr><td>6.9. CONCLUSION</td><td>155</td></tr><tr><td>CHAPTER 7 CONCLUSIONS AND FUTURE SCOPE</td><td>157</td></tr><tr><td>APPENDIX-A</td><td>163</td></tr><tr><td>AUTHOR'S PUBLICATIONS</td><td>165</td></tr><tr><td>REFERENCES</td><td>167</td></tr></table>
LIST OF FIGURES

<table>
  <tbody>
    <tr>
      <td>Figure 1.1 Landsat-8 pixel of 30m spatial resolution.</td>
      <td>4</td>
    </tr>
    <tr>
      <td>Figure 1.2 Improvements in image spatial resolutions over time.</td>
      <td>4</td>
    </tr>
    <tr>
      <td>Figure 1.3 Demonstration of the concept of radiometric resolution. The larger the bit depth, the higher the radiometric resolution (more number of intensity levels).</td>
      <td>7</td>
    </tr>
    <tr>
      <td>Figure 1.4 The temporal resolution of a satellite imaging system refers to how often it records imagery of a particular area. This example depicts the systematic collection of data every 30 days, presumably at approximately the same time of day.</td>
      <td>7</td>
    </tr>
    <tr>
      <td>Figure 1.5 Illustration of the propagation of an electromagnetic plane wave. The electric field vector has horizontal (green) and vertical (blue) components, which combine to yield the net electric field vector (red) [27].</td>
      <td>14</td>
    </tr>
    <tr>
      <td>Figure 1.6 Comparison of performances of traditional machine learning and deep learning techniques with respect to amount of training data availability.</td>
      <td>27</td>
    </tr>
    <tr>
      <td>Figure 1.7 Figure illustrates that deep learning models can directly learn from raw data. The feature selection/extraction process is therefore redundant and sub-optimal in these models [71].</td>
      <td>27</td>
    </tr>
    <tr>
      <td>Figure 1.8 A general framework in DNN based models for satellite image analysis and processing.</td>
      <td>28</td>
    </tr>
    <tr>
      <td>Figure 1.9 Framework of the research.</td>
      <td>33</td>
    </tr>
    <tr>
      <td>Figure 3.1 (a) False colour composite generated from PALSAR-2 data. Colour scheme: red = HH, green = HV, and blue = VV. (b) Panchromatic image collected from Google Earth dated Feb 19, 2015. Symbol ‘A’ marks built-up area in both the images and symbol ‘B’ marks the dense vegetation area in both the images. (c) The enlarged segments are shown in middle in the outlined boxes. The built-up area is showing volume scattering (high backscatter response in HV, refer) along with double bounce scattering.</td>
      <td>51</td>
    </tr>
    <tr>
      <td>Figure 3.2 Co-polarized polarization signature of an arbitrary land cover.</td>
      <td>53</td>
    </tr>
    <tr>
      <td>Figure 3.3 Sample images of different land covers collected during ground truth survey. (a) Tall vegetation, (b) bare soil, (c) agriculture cropland, and (d) sugarcane field.</td>
      <td>56</td>
    </tr>
    <tr>
      <td>Figure 3.4 Flowchart showing polarization signature correlation features (PSCFs) computation using fully polarimetric SAR data.</td>
      <td>58</td>
    </tr>
    <tr>
      <td>Figure 3.5 Geometrical structures, scattering matrices, and co-polarization signatures of different standard targets.</td>
      <td>58</td>
    </tr>
  </tbody>
</table>
Figure 3.6 Figure illustrates the computation of polarization signature correlation features (PSCFs) between observed (image pixel) and standard targets PSs. 60

Figure 3.7: Classified Image; (a) M-Y4D-SVM, (b) GFD-SVM, (c) Cloude-SVM, (d) H-A-Alpha-SVM, (e) PSCF-SVM (proposed), and (f) Google Earth image for reference purpose. Colour scheme; red = built-up, light green = agriculture or SV, green = TV., blue = water, and yellow = bare soil. Marker A indicates built-up area and marker B indicates tall vegetation area. 64

Figure 3.8 Correlation images (a) corr_co_HD and (b) corr_co_VD. 65

Figure 3.9 Flowchart illustrates the development process of the optimal class boundary estimator based decision tree classifier. 70

Figure 3.10 Decision tree based on optimal class boundaries values. 70

Figure 3.11: Classified image: (a) Optimal class boundary estimator based DT classifier, (b) RBF-SVM, and (c) Google Earth image for reference purpose. Colour scheme; red = built-up, light green = SV, green = TV, blue= water, and yellow = BS. Marker A indicates built-up area and marker B indicates tall vegetation area. 72

Figure 3.12: Two dimensional convolution over polarization signature. (a) Polarization Signature of a PolSAR image pixel. (b) Top view. (c) Polarization signature subset. (d) Convolutional filter. 73

Figure 3.13 Architecture of the proposed PSs-CNN model. 74

Figure 3.14 Classified Image: (a) PSs-CNN, (b) NSCM, (c) NED, and (d) Google earth image. Colour scheme; red = built-up, light green = SV, green = TV, blue= water, and yellow = BS. Marker A indicates built-up area and marker B indicates tall vegetation area. 75

Figure 4.1 (a) Google Earth imagery of the Roorkee with markers indicating major land cover classes. Markers highlighted in dots: red = built-up, green = TV, blue = water, light green = SV, and yellow = BS (Courtesy: Google Earth). (b) False colour representation of preprocessed SAR data. 83

Figure 4.2 (a) Google Earth imagery of the Haridwar with markers indicating major land cover classes. Markers highlighted in dots: red = built-up, green = TV, blue = water, light green = SV and yellow = BS (Courtesy: Google Earth). (b) False colour representation of preprocessed SAR data. 84

Figure 4.3 Areas highlighted in white are villages in the (a) Roorkee, and (b) Haridwar study areas. Village areas are extracted from Google Earth imagery of the corresponding regions. (Courtesy: Google Earth). 84

Figure 4.4 Flowchart of the proposed approach. 86
Figure 4.5 Sample co-polarized PSs of land cover classes mentioned in Table 4.1. (a) bare soil, (b) TV, (c) SV, (d) built-up, and (e) water. 86

Figure 4.6 Architecture of the proposed PSs-CNN. 88

Figure 4.7 Roorkee classified image obtained using: (a) the PSs-CNN classifier, (e) the NED classifier, and (i) the NSCM classifier. Areas classified as built-up in Roorkee study area using: (b) the PSs-CNN classifier, (f) the NED classifier, and (j) the NSCM classifier. Haridwar classified image obtained using: (c) the PSs-CNN classifier, (g) the NED classifier, and (k) the NSCM classifier. Areas classified as built-up in Haridwar study area using: (d) the PSs-CNN classifier, (h) the NED classifier, and (l) the NSCM classifier. Colour scheme: red = built-up, green = TV, and blue = rest (Water + BS + SV). 90

Figure 4.8 Results from different morphological operations considered during village detection: (a) Segment of image shown in Figure 4.7(a). Corresponding segments of: (b) Image after the closing operation, (c) Image after the holes filling operation, and (d) image after the filtering operation. 94

Figure 4.9 (a) False colour composites for Roorkee using reference-DV<sub>PSs-CNN</sub>-DV<sub>NED</sub>, (b) False colour composites for Haridwar using reference-DV<sub>PSs-CNN</sub>-DV<sub>NED</sub>, (c) False colour composites for Roorkee using reference-DV<sub>PSs-CNN</sub>-DV<sub>NSCM</sub>, and (d) False colour composites for Haridwar using reference-DV<sub>PSs-CNN</sub>-DV<sub>NSCM</sub>. Colour scheme: red = reference, green = DV<sub>PSs-CNN</sub>, blue = DV<sub>NED</sub> or DV<sub>NSCM</sub> 94

Figure 4.10 Methodology flowchart for flood affected village mapping. 97

Figure 4.11 Different scenarios: (a) Scenario 1, (b) Scenario 2, and (c) Scenario 3. Colour scheme: blue = flooded region. 98

Figure 4.12 Flood maps: (a) Affected villages under flood condition 1. (b) Affected villages under flood condition 2. (c) Affected villages under flood condition 3. Colour scheme: red = villages not affected by flood, yellow = villages affected by flood, and blue = flooded region. 98

Figure 5.1 Datasets for Roorkee study area: (a) False colour composite (red = HH, green = HV, and blue = VV) of ALOS PALSAR-2 PolSAR data, and (b) False colour composite (red = NIR, green = Red, and blue = Green) of Lansat-8 multispectral data. 107

Figure 5.2 Datasets for Haridwar study area: (a) False colour composite (red = HH, green = HV, and blue = VV) of ALOS PALSAR-2 PolSAR data, and (b) False colour composite (red = NIR, green = Red, and blue = Green) of Lansat-8 multispectral data. 107

Figure 5.3 Flowchart of the experiment setup and CNN based land cover classification methodologies proposed with PolSAR-MS data. 108
Figure 5.4 Different convolution strategies: (a) Data cuboid with M by N pixels and P bands, (b) Convolution with 1 × 1 × P convolutional filter where 1 × 1 represents to the spatial extent and P represents total number of bands in the data. Each filter convolves with all the bands as all times. This strategy is employed in 1D-CNNs during per pixel classification of satellite images. (c) Convolution with 1 × 1 × 3 convolutional filters where 1 × 1 represents the spatial extent and 3 represents the number of adjacent bands to be convolved simultaneously. One filter can convolve with three adjacent bands only. This strategy is also employed in 1D-CNNs. (d) Convolution with 1 × 1 × 3 convolutional filters where 1 × 1 represents to the spatial extent and 3 represents number of arbitrarily selected bands to be involved. This strategy is employed in the proposed Perm-LS-CNN framework. 110

Figure 5.5: Different convolution strategies: (a) Data cuboid with M by N pixels and P bands, (b) Convolution with 3 × 3 × P convolutional filter where 3 × 3 represents the spatial extent and P represents the total number of bands in the data. Each filter convolves with all the bands. This strategy is employed in 2D CNNs, (c) Convolution with 3 × 3 × 3 convolutional filters where 3 × 3 represents to the spatial extent and 3 represents number of adjacent bands to be convolved simultaneously. One filter can convolve with three adjacent bands only. This strategy is employed in 3D-CNNs [96], and (d) Convolution with 3 × 3 × 3 convolutional filters where 3 × 3 represents the spatial extent and 3 represents the number of arbitrarily selected bands to be involved. This strategy is employed in the proposed Perm-LSS-CNN framework. 112

Figure 5.6 Classification image: (a) Google Earth image of the Roorkee study area, India. Classified images are shown for different classifiers: (b) 1D-CNN-v1, (c) 1D-CNN-v2, (d) Perm-LS-CNN, (e) 2D-CNN-v1, (f) 3D-CNN-v1, (g) Perm-LSS-CNN, (h) RBF-SVM, and (i) RF. Colour scheme; red = built-up, light green = short vegetation, green = tall vegetation, blue = Water, and yellow = bare soil. 118

Figure 5.7 Classification image: (a) Google Earth image of the Haridwar study area, India. Classified images are shown for different classifiers: (b) 1D-CNN-v1, (c) 1D-CNN-v2, (d) Perm-LS-CNN, (e) 2D-CNN-v1, (f) 3D-CNN-v1, (g) Perm-LSS-CNN, (h) RBF-SVM, and (i) RF. Colour scheme; red = built-up, light green = short vegetation, green = tall vegetation, blue = water, and yellow = bare soil. 119

Figure 5.8 Results from different classification techniques applied on the Roorkee data set. The areas include data points collected as ground truth. 121

Figure 6.1 Typical architecture of a recurrent neural network [250]. 127

Figure 6.2 Signal flow graph in a simple recurrent cell [250]. 128

Figure 6.3 A LSTM recurrent cell [250]. 130
<table>
  <tbody>
    <tr>
      <td>Figure 6.4 A GRU recurrent cell [250]</td>
      <td>130</td>
    </tr>
    <tr>
      <td>Figure 6.5 RNN architectures used popularly in satellite time series image based crop classification: (a) Many to one, and (b) Many to many RNN architectures [250]</td>
      <td>131</td>
    </tr>
    <tr>
      <td>Figure 6.6 Google Earth image of the study area (GE Image Copyright 2018). Subset areas selected for ground truth via visual inspection. 1: GTI-1, 2: GTI-2, 3: GTI-3, and 4: GTI-4 (highlighted in yellow boxes)</td>
      <td>132</td>
    </tr>
    <tr>
      <td>Figure 6.7 False colour composite from the Sentinel-2 time series dataset. Colour scheme; red: Band 4 (November 21, 2017), green: Band 4 (February 9, 2018), blue: Band 4 (March 31, 2018). Image subset areas selected for ground truth via visual inspection (highlighted in black boxes). 1 = GTI-1, 2 = GTI-2, 3 = GTI-3, and 4 = GTI-4.</td>
      <td>133</td>
    </tr>
    <tr>
      <td>Figure 6.8 Sample images of different land covers collected during ground truth survey. (a) Sugarcane cropland, (b) Wheat cropland, (c) Grasslands, (d) Popular plantations.</td>
      <td>134</td>
    </tr>
    <tr>
      <td>Figure 6.9 A generic framework for DNN models for satellite time series images based crop classification.</td>
      <td>136</td>
    </tr>
    <tr>
      <td>Figure 6.10 Classified image; (a) 1D-CNN-v1. Markers; 1: GTI-1, 2: GTI-2, 3: GTI-3, and 4: GTI-4. (b) 3D-CNN-v1, (c) RNN-v1, (d) 1D-CRNN-v1, (e) 2D-CRNN-v1, (f) 3D-CRNN-v1, (g) Perm-1D-CNN-v1, (h) Perm-3D-CNN-v1, (i) Perm-RNN-v1, (j) Perm-1D-CRNN-v1, (k) Perm-2D-CRNN-v1, and (l) Perm-3D-CRNN-v1.</td>
      <td>151</td>
    </tr>
    <tr>
      <td>Figure 6.11 Visual interpretation and evaluation of classification performance of the compared classifiers for GTI-1</td>
      <td>152</td>
    </tr>
    <tr>
      <td>Figure 6.12 Visual interpretation and evaluation of classification performance of the compared classifiers for GTI-2</td>
      <td>153</td>
    </tr>
    <tr>
      <td>Figure 6.13 Visual interpretation and evaluation of classification performance of the compared classifiers for GTI-3</td>
      <td>154</td>
    </tr>
    <tr>
      <td>Figure 6.14 Visual interpretation and evaluation of classification performance of the compared classifiers for GTI-4</td>
      <td>154</td>
    </tr>
  </tbody>
</table>
LIST OF TABLES

<table><tr><td>Table 1.1 Broad categorization of the electromagnetic spectrum in terms of spectral bands and their spectral bandwidths [21]</td><td>6</td></tr><tr><td>Table 1.2 Landsat-8 (multispectral) OLI and TIRS band specifications [22]</td><td>10</td></tr><tr><td>Table 1.3 Sentinel-2 band specifications [23]</td><td>10</td></tr><tr><td>Table 1.4 MODIS sensor band specifications and primary usage [25]</td><td>11</td></tr><tr><td>Table 1.5 Hyperion sensor specifications [26]</td><td>12</td></tr><tr><td>Table 1.6 Polarimetric parameters</td><td>15</td></tr><tr><td>Table 1.7 Designation of microwave bands [30]</td><td>18</td></tr><tr><td>Table 1.8 Brief list of few past, current and upcoming spaceborne SAR sensors with their properties [30]</td><td>18</td></tr><tr><td>Table 3.1 Summary of ground truth sample points collected for study</td><td>56</td></tr><tr><td>Table 3.2 Separability index (SI) values for various class pairs for all features</td><td>63</td></tr><tr><td>Table 3.3 Separability index (SI) values for various class pairs for popular features</td><td>63</td></tr><tr><td>Table 3.4 Classification performance of PS based and decomposition based PolSAR features. Classes: built-up (BU), water (W), bare soil (BS), short vegetation (SV), and tall vegetation (TV)</td><td>64</td></tr><tr><td>Table 3.5 List of selected feature corresponding to land cover classes</td><td>65</td></tr><tr><td>Table 3.6 Updated list of selected feature corresponding to land cover classes</td><td>66</td></tr><tr><td>Table 3.7 Polarization signature correlation feature statistics</td><td>66</td></tr><tr><td>Table 3.8 Empirical relations between individual overall accuracies (IOA) and n<sub>i</sub> values</td><td>68</td></tr><tr><td>Table 3.9 Optimal n<sub>i</sub> values obtained using the genetic algorithm</td><td>69</td></tr><tr><td>Table 3.10: Classification performance indicators for the proposed method and RBF-SVM classifier with PSCFs as features</td><td>71</td></tr><tr><td>Table 3.11 Summary of PSs-CNN model configuration, hyperparameters and training</td><td>74</td></tr><tr><td>Table 3.12 Classification performance indicators for the PSs-CNN, NSCM, and NED classifiers with PSs as features. BS = bare soil, TV = Tall vegetation, SV = Short vegetation, BU = Built-up</td><td>75</td></tr><tr><td>Table 3.13 Classification performance indicators for the optimal class boundary estimator based DT, PSs-CNN, and RBF-SVM classifier</td><td>76</td></tr><tr><td>Table 4.1. Summary of ground truth samples collected for the study. *BS = bare soil, TV = dense tall vegetation, SV = short vegetation. All points are acquired directly on the field by visual</td><td></td></tr></table>
<table>
  <tr>
    <td>inspection at selected GPS coordinates. Also, the total number of villages in both the study regions are listed here.</td>
    <td>84</td>
  </tr>
  <tr>
    <td>Table 4.2 Summary of PSs-CNN model configuration, hyperparameters and training</td>
    <td>88</td>
  </tr>
  <tr>
    <td>Table 4.3 Summary of the classification performance of the considered classifiers considered for Roorkee and Haridwar datasets. OA = overall accuracy. Rest = BS + Water + SV.</td>
    <td>91</td>
  </tr>
  <tr>
    <td>Table 4.4 summary of quantitative assessment of performance of PSS-CNN, NED, and NSCM</td>
    <td>96</td>
  </tr>
  <tr>
    <td>Table 5.1 Summary of the ground truth collected for study. *BS = Bare soil, TV = Tall vegetation, SV = Short vegetation. All points are acquired directly on the field by visual inspection at selected GPS coordinates.</td>
    <td>108</td>
  </tr>
  <tr>
    <td>Table 5.2 1D-CNN-v1, 1D-CNN-v2, 2D-CNN-v1, and 3D-CNN-v1 CNN model configurations, hyperparameter and training settings.</td>
    <td>111</td>
  </tr>
  <tr>
    <td>Table 5.3 Perm-LS-CNN and Perm-LSS-CNN model configurations, hyperparameters and training settings.</td>
    <td>115</td>
  </tr>
  <tr>
    <td>Table 5.4 Separability index (SI) values for all the possible class pairs for the 2D CNN-v1, 3D CNN-v1, and 'Perm-LSS-CNN filters.</td>
    <td>116</td>
  </tr>
  <tr>
    <td>Table 5.5 Classification performance of various classifiers considered for PolSAR-MS Roorkee dataset. *AD is the difference between precision and recall, averaged over all the land cover classes considered. BS = bare soil, TV = tall vegetation, SV = short vegetation, BU = built-up, W = water.</td>
    <td>120</td>
  </tr>
  <tr>
    <td>Table 5.6 Classification performance of various classifiers considered for PolSAR-MS Haridwar dataset. *AD is the difference between precision and recall, averaged over all the land cover classes considered. BS = bare soil, TV = tall vegetation, SV = short vegetation, BU = built-up, W = water.</td>
    <td>120</td>
  </tr>
  <tr>
    <td>Table 6.1 Geographical extent of study area.</td>
    <td>131</td>
  </tr>
  <tr>
    <td>Table 6.2 Sentinel-1 and Sentinel-2 data acquisition dates.</td>
    <td>132</td>
  </tr>
  <tr>
    <td>Table 6.3 Band specifications of Sentinel-1 and Sentinel-2.</td>
    <td>133</td>
  </tr>
  <tr>
    <td>Table 6.4 Summary of ground truth samples collected during ground truth survey.</td>
    <td>134</td>
  </tr>
  <tr>
    <td>Table 6.5 Architecture and hyperparameter settings of the 1D-CNN-v1, 2D-CNN-v1, and 3D-CNN-v1 models for MSTS images based crop classification.</td>
    <td>139</td>
  </tr>
  <tr>
    <td>Table 6.6 Architecture and hyperparameter settings of the RNN-v1 model for MSTS images based crop classification.</td>
    <td>139</td>
  </tr>
  <tr>
    <td>Table 6.7 Architecture and hyperparameter settings of CNN-RNN based models for MSTS images based crop classification.</td>
    <td>141</td>
  </tr>
</table>
<table><tr><td>Table 6.8 Architecture and hyperparameter settings of CNN models with permuted spectral convolution for MSTS images based crop classification.</td><td>145</td></tr><tr><td>Table 6.9 Architecture and hyperparameter settings of RNN model with permuted input for MSTS images based crop classification.</td><td>146</td></tr><tr><td>Table 6.10 Architecture and hyperparameter settings of CNN-RNN with permuted spectral convolution based models for MSTS images based crop classification.</td><td>149</td></tr><tr><td>Table 6.11 Summary of the classification performance of the compared classifiers.</td><td>150</td></tr></table>
LIST OF ACRONYMS

<table><tr><td>AL</td><td>Area limit</td></tr><tr><td>ALOS</td><td>Advanced land observing satellite</td></tr><tr><td>AMSR-E</td><td>Advanced microwave scanning radiometer for earth observation</td></tr><tr><td>ANNs</td><td>Artificial neural networks</td></tr><tr><td>BBOR</td><td>Bounding box overlap ratio</td></tr><tr><td>Bi-LSTM</td><td>Bidirectional long short term memory</td></tr><tr><td>BOA</td><td>Bottom-of-Atmosphere</td></tr><tr><td>BS</td><td>Bare Soil</td></tr><tr><td>CERES</td><td>Clouds and the earth's radiant energy system</td></tr><tr><td>CNN</td><td>Convolutional neural network</td></tr><tr><td>Corr_co</td><td>Co-polarized correlation</td></tr><tr><td>Corr cross</td><td>Cross-polarized correlation</td></tr><tr><td>DBNs</td><td>Deep belief networks</td></tr><tr><td>DCAE</td><td>Deep conventional auto encoder</td></tr><tr><td>DEM</td><td>Digital elevation models</td></tr><tr><td>DESIS</td><td>DLR earth sensing imaging spectrometer</td></tr><tr><td>Di</td><td>Dihedral</td></tr><tr><td>DL</td><td>Deep learning</td></tr><tr><td>DNN</td><td>Deep neural network</td></tr><tr><td>DT</td><td>Decision tree</td></tr><tr><td>DV</td><td>Detected village</td></tr><tr><td>EM</td><td>Electromagnetic</td></tr><tr><td>EnMAP</td><td>Environmental mapping and analysis program</td></tr><tr><td>FC</td><td>Fully Connected</td></tr><tr><td>FIR</td><td>Far infrared</td></tr><tr><td>FP</td><td>Flat plate</td></tr><tr><td>FP</td><td>False positive</td></tr><tr><td>FWHM</td><td>Full width and half maximum</td></tr><tr><td>GA</td><td>Genetic algorithm</td></tr><tr><td>GFD</td><td>Generalized Freeman Durden decomposition</td></tr></table>
<table>
   <tr>
    <td>GOES</td>
    <td>Geostationary operational environmental satellite</td>
   </tr>
   <tr>
    <td>GPS</td>
    <td>Global positioning system</td>
   </tr>
   <tr>
    <td>GRU</td>
    <td>Gradient recurrent unit</td>
   </tr>
   <tr>
    <td>GTI</td>
    <td>Ground truth image</td>
   </tr>
   <tr>
    <td>HD</td>
    <td>Horizontal dipole</td>
   </tr>
   <tr>
    <td>HH</td>
    <td>Horizontal transmit and horizontal receive</td>
   </tr>
   <tr>
    <td>HSI</td>
    <td>Hyperspectral imagery</td>
   </tr>
   <tr>
    <td>HV</td>
    <td>Horizontal transmit and vertical receive</td>
   </tr>
   <tr>
    <td>IAEG-SDG</td>
    <td>Inter-agency and expert group on SDG indicator</td>
   </tr>
   <tr>
    <td>IFOV</td>
    <td>Instantaneous field of view</td>
   </tr>
   <tr>
    <td>InSAR</td>
    <td>Interferometric synthetic aperture radar</td>
   </tr>
   <tr>
    <td>IOA</td>
    <td>Individual overall accuracy</td>
   </tr>
   <tr>
    <td>ISRO</td>
    <td>Indian space research organization</td>
   </tr>
   <tr>
    <td>JAXA</td>
    <td>Japan aerospace exploration agency</td>
   </tr>
   <tr>
    <td>LandSat</td>
    <td>Land remote-sensing satellite</td>
   </tr>
   <tr>
    <td>LC</td>
    <td>Land Cover</td>
   </tr>
   <tr>
    <td>LIDAR</td>
    <td>Light detection and ranging</td>
   </tr>
   <tr>
    <td>LSTM</td>
    <td>Long short term memory</td>
   </tr>
   <tr>
    <td>LWIR</td>
    <td>Long wave infrared</td>
   </tr>
   <tr>
    <td>M</td>
    <td>Mean</td>
   </tr>
   <tr>
    <td>MAP</td>
    <td>Maximum a posterior</td>
   </tr>
   <tr>
    <td>MLP</td>
    <td>Multilayer perceptron</td>
   </tr>
   <tr>
    <td>MODIS</td>
    <td>Moderate-resolution imaging spectroradiometer</td>
   </tr>
   <tr>
    <td>MOPITT</td>
    <td>Measurements of Pollution in the Troposphere</td>
   </tr>
   <tr>
    <td>MRI</td>
    <td>Magnetic resonance imaging</td>
   </tr>
   <tr>
    <td>MS</td>
    <td>Multispectral</td>
   </tr>
   <tr>
    <td>MSI</td>
    <td>Multispectral instrument</td>
   </tr>
   <tr>
    <td>MSS</td>
    <td>Multispectral scanner</td>
   </tr>
   <tr>
    <td>MTS</td>
    <td>Multisensor time series</td>
   </tr>
   <tr>
    <td>MWIR</td>
    <td>Mid wave infrared</td>
   </tr>
   <tr>
    <td>m-Y4D</td>
    <td>Modified Yamaguchi four component decomposition</td>
   </tr>
   <tr>
    <td>NAIP</td>
    <td>National agricultural innovation project</td>
   </tr>
   <tr>
    <td>NASA</td>
    <td>National aeronautics and space administration</td>
   </tr>
  </table>
<table>
   <tr>
    <td>NDVI</td>
    <td>Normalized difference vegetation index</td>
   </tr>
   <tr>
    <td>NED</td>
    <td>Normalized Euclidean distance</td>
   </tr>
   <tr>
    <td>NIR</td>
    <td>Near infrared</td>
   </tr>
   <tr>
    <td>NISAR</td>
    <td>NASA ISRO SAR</td>
   </tr>
   <tr>
    <td>NSCM</td>
    <td>Normalized signature correlation mapper</td>
   </tr>
   <tr>
    <td>OA</td>
    <td>Overall accuracy</td>
   </tr>
   <tr>
    <td>OLI</td>
    <td>Operational land imager</td>
   </tr>
   <tr>
    <td>PALSAR</td>
    <td>Phased array type L-band synthetic aperture radar</td>
   </tr>
   <tr>
    <td>PCA</td>
    <td>Principal component analysis</td>
   </tr>
   <tr>
    <td>Perm-LS-CNN</td>
    <td>Permuted local spectral convolutional neural network</td>
   </tr>
   <tr>
    <td>Perm-LSS-CNN</td>
    <td>Permuted local spectral-spatial convolutional neural network</td>
   </tr>
   <tr>
    <td>PolSAR</td>
    <td>Polarimetric synthetic aperture radar</td>
   </tr>
   <tr>
    <td>ppi</td>
    <td>Pixels per inch</td>
   </tr>
   <tr>
    <td>PS</td>
    <td>Polarization signature</td>
   </tr>
   <tr>
    <td>PSCF</td>
    <td>Polarization signature correlation features</td>
   </tr>
   <tr>
    <td>RBF</td>
    <td>Radial basis function</td>
   </tr>
   <tr>
    <td>RBMs</td>
    <td>Restricted Boltzmann machine</td>
   </tr>
   <tr>
    <td>RCS</td>
    <td>Radar cross section</td>
   </tr>
   <tr>
    <td>ReLU</td>
    <td>Rectified linear unit</td>
   </tr>
   <tr>
    <td>RF</td>
    <td>Random forest</td>
   </tr>
   <tr>
    <td>RNN</td>
    <td>Recurrent neural network</td>
   </tr>
   <tr>
    <td>ROI</td>
    <td>Regions of interest</td>
   </tr>
   <tr>
    <td>RS</td>
    <td>Remote sensing</td>
   </tr>
   <tr>
    <td>S</td>
    <td>Standard deviation</td>
   </tr>
   <tr>
    <td>SAE</td>
    <td>Stacked auto encoder</td>
   </tr>
   <tr>
    <td>SAR</td>
    <td>Synthetic aperture radar</td>
   </tr>
   <tr>
    <td>SCM</td>
    <td>Signature correlation mapper</td>
   </tr>
   <tr>
    <td>SI</td>
    <td>Separability index</td>
   </tr>
   <tr>
    <td>SLC</td>
    <td>Single look complex</td>
   </tr>
   <tr>
    <td>SNR</td>
    <td>Signal to noise ratio</td>
   </tr>
   <tr>
    <td>SONAR</td>
    <td>Sound navigation ranging</td>
   </tr>
   <tr>
    <td>SRTM</td>
    <td>Shuttle radar topography mission</td>
   </tr>
   <tr>
    <td>SV</td>
    <td>Short vegetation</td>
   </tr>
  </table>
<table><tr><td>SVM</td><td>Support vector machine</td></tr><tr><td>SWIR</td><td>Short wave infrared</td></tr><tr><td>TIRS</td><td>Thermal infrared sensor</td></tr><tr><td>TN</td><td>True negative</td></tr><tr><td>TP</td><td>True positive</td></tr><tr><td>TRMM</td><td>Tropical rainfall measuring mission</td></tr><tr><td>TV</td><td>Tall vegetation</td></tr><tr><td>UN</td><td>United Nations</td></tr><tr><td>VD</td><td>Vertical dipole</td></tr><tr><td>VH</td><td>Vertical transmit and horizontal receive</td></tr><tr><td>VV</td><td>Vertical transmit and vertical receive</td></tr></table>
<figure><img src="image_12.png" /></figure>

INTRODUCTION

The planet is continually being observed and imaged by satellites. Before 1972, satellites weren't designed to study or monitor earth's surface. Instead, they were mainly used for military missions. Satellite based earth imaging was commercialized in 1984, but faced many funding issues. This led to the passing of the land remote sensing policy act of 1992. The new law embraced the value of satellite based earth imaging technology, promoted research and public sector applications, and allowed commercial companies to launch satellites, acquire images of earth's surface, and utilize it or sell it. As a result, earth observation satellites specifically intended for non-military applications were designed for long-term global observations of the land surface, oceans, biosphere, and atmosphere.

In the last decade, there has been a surge in the availability of high quality (spatial, spectral, and temporal), open-source/economical earth observation satellite images. Today, more than 600 satellites provide earth observation data, mostly in the form of images. Satellite images are available at different spatial, spectral, radiometric, and temporal resolutions. Also, satellite images are available in different modalities such as surface reflectance (multispectral, superspectral, or hyperspectral), backscatter (active microwave), emission (passive microwave, thermal), polarization (linear, circular, or elliptical), and others. These satellite images are used for numerous civil and defence applications. Popular categories of satellite image applications include land cover monitoring, agriculture crop monitoring, disaster monitoring, environment health and earth surface dynamics monitoring, and mineral exploration. These applications vary from being performance-critical such as land cover classification to being time-critical such as disaster management. Therefore, different applications require focus on different characteristics (spatial, spectral or temporal) of satellite images. The different characteristics, modalities, and applications of satellite images are further discussed in section 1.1.

Processing satellite images for utilization in different applications is challenging. Literature reports hundreds of approaches or techniques available for satellite image processing for various applications. The popular satellite image processing techniques are decision tree, random forest, maximum likelihood, support vector machines, artificial neural networks, k-nearest neighbours [1], logistic regression [2], distance measure [3], K-means, ISODATA, and more. These techniques are briefly discussed in section 1.2. However, it is observed that development of techniques, that can explore different properties (different resolutions or different
modalities) offered by different satellite images, is tedious and crafty let alone, keeping track of
more resourceful satellite images every day and improvising the techniques accordingly is not an
easy task. Therefore, it is expected that new techniques should possess universal applicability,
scalability, automatability, and generalization. Generalization is the measure of the ability of
machine learning techniques on how well these techniques perform on ‘out of sample’ data. The
challenges with currently popular satellite image processing techniques and new expectations are
discussed in section 1.3.

Deep learning (DL) with deep neural networks (DNNs) have shown substantial potential
in satellite image processing applications [4]. The deep learning theory is a universally applicable
approach which can map multiple levels of information to represent complex relationships
between data [5]. Various aspects of deep learning favourable to satellite image processing
application are discussed in section 1.4.

However, the deep neural networks are still young, promising, and growing techniques
in the satellite image processing community and further exploration of the potential of these
models is motivating. The present chapter attempts to establish the need of a comprehensive
study on the potential, significance, advantage and drawbacks with using deep neural network
models for extraction of information from satellite images. This chapter also highlights some
major research gaps in the development of more efficient and generalized deep neural network
models. Based on the research gaps, new objectives are resolved and are discussed in this chapter.

## 1.1. Background: Satellite Images

Satellite imagery depicts the earth's surface at various spectral, temporal, radiometric,
and spatial resolutions, as determined by the sensing device (sensor) on board each collection
system, and the orbital path of its reconnaissance platform [6]. Satellite sensors acquire
information reflected or emitted from an object in the form of electromagnetic radiation and then
the information about the state and condition of an object is derived by interpreting this received
electromagnetic signal. The collected information (in the form of images) indicates about the
physical, chemical, and biological properties of observed object which may be soil [7], [8], ice,
snow [9], [10], water [11], wind [12], vegetation [13] or any other. The goal of collecting the
information is to understand how electromagnetic energy interacts with the earth's surface
objects. The information can further be used for crucial military and civil applications such as
monitoring; vegetation coverage [14], soil moisture [15], disasters [16], sea-ice fluctuation, sea
level and sea surface temperatures [17], [18], ships in sea [19], concentrations of atmospheric
gases, and many other parameters. These observations are applied to meteorological forecasting, environmental monitoring, disaster monitoring, and cartographic purposes.

Satellite images can be gathered using different ways of “sensing” the object: (i) by reflection, (ii) by emission, and (iii) by combined emission-reflection.

### 1.1.1. Satellite image: Characteristics

A satellite image has four different characteristics namely spatial resolution, spectral resolution, radiometric resolution, and temporal resolution. The concept of resolution embraces several aspects and most common of these is spatial resolution, but other aspects are also very important for image interpretation.

#### A. Spatial Resolution

Spatial resolution is the fineness of detail in the image and is measured in pixel per inch (ppi). The more pixels per inch, better the resolution. Generally, an image with a higher resolution produces a better printed image quality. Spatial resolution helps to identify the minimum separation at which objects appear independent and isolated. It is measured in millimetres on the photograph or in meters on the ground, and it depends on the focal length of the camera and its height above the ground. The most common definition of sensor spatial resolution is the size of the projected instantaneous field of view (IFOV) on the ground (d), computed as in equation (1.1).

$$d = 2H \tan\left(\frac{\text{IFOV}}{2}\right) \qquad (1.1)$$

Where d is the distance on the ground for each information unit (pixel) and H the height of the observing sensor. IFOV is defined as the angular section observed by the sensor, in radians, at a given moment in time.

The spatial resolution of optical-electronic sensors depends on orbit height, speed of data collection, and the number of detectors. As for the antenna sensors such as SAR, their resolution depends on the aperture angle, height of the platform, and the wavelength at which they work. The larger the aperture angle, the lower the height and the wavelength, and the more detailed the resolution. Figure 1.1 shows a subset of image acquired via the Landsat-8 sensor which has 30 m spatial resolution. It means that a single pixel represents an area of 900 square meters (30m ×30m) on the ground. The spatial resolution of imaging sensors improved significantly over time as shown in Figure 1.2 and is usually inversely related to acquisition period and coverage area.
<figure><img src="image_13.png" /><figcaption>Figure 1.1 Landsat-8 pixel of 30m spatial resolution.</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>Figure 1.2 Improvements in image spatial resolutions over time.</figcaption></figure>

## B. Spectral Resolution

Spectral resolution refers to the number of bands provided by the sensor, as well as their spectral bandwidths. Generally speaking, a sensor will provide better discrimination capacity as more bands are acquired. Multispectral satellite imaging systems record energy in multiple bands of the electromagnetic spectrum. Table 1.1. briefly summarizes the categorization of the electromagnetic (EM) spectrum in terms of spectral bands and spectral bandwidths. The nominal size of a band may be large (i.e. coarse), as with the Landsat MSS near-infrared band 4 (800 – 1,100 nm) or relatively smaller (i.e. fine), as with the Landsat MSS band 3 (700 – 800 nm). Thus, Landsat MSS band 4 detectors recorded a relatively large range of reflected near-infrared radiant flux (300 nm wide) while the MSS band 3 detectors recorded a much reduced range of near-infrared radiant flux (100 nm wide). A hyperspectral satellite imaging sensor typically acquires images in hundreds of spectral bands. For example, the earth observing-1 Hyperion has 220
bands in the region from 356 to 2,577 nm spaced just 10 nm apart based on the full width and
half maximum (FWHM) criteria.

C. **Radiometric Resolution**

The radiometric resolution of an imaging system (including satellite sensors) is described as its ability to discriminate very slight differences in energy. The finer the radiometric resolution of a sensor, the more sensitive it is to detecting small differences in reflected, scattered, or emitted energy. Figure 1.3 demonstrates the concept of radiometric resolution. In Figure 1.3, the image represented with 1 bit has only 2 levels of intensities whereas the image represented with 16 bits has 65,536 levels of intensities. While the arrangement of pixels describes the spatial distribution of a structures present in an image, the radiometric characteristics represent information about the interaction of the structure with electromagnetic energy. Every time an image is acquired on film or sensor, its sensitivity to the change in magnitude of the electromagnetic energy determines its radiometric resolution.

D. **Temporal Resolution**

Temporal resolution is defined as the amount of time needed to revisit and acquire data for the exact same location. When applied to satellite imaging based earth observation, temporal resolution is controlled by a sensor's swath width alongside the orbital parameters of its sensing platform; together these parameters determine the periodicity (return interval) at which a fixed area on the ground is successively imaged by a space borne sensor. It is also important to bear in mind that the effective temporal resolution depends on the atmospheric conditions because optical and thermal sensors cannot detect surfaces under cloud cover. That is why in some areas the availability of useful images can be notably lower that the acquisition cycle of the sensor (tropical wet regions, for instance). To somehow alleviate this problem, modern sensors have the capacity to make observations pointing the sensor at different angles, facilitating the collection of data over a zone of interest from an adjacent orbit.

Temporal resolution is a primary consideration when selecting an image product for researchers studying time series behaviour. Sensors with higher temporal resolution or more extensive imaging archives are however, most likely to afford cloud-free and seasonally favourable images, making them attractive options for all investigators. Figure 1.4 depicts the temporal revisit period of an arbitrary satellite as a typical example.
<table><caption>Table 1.1 Broad categorization of the electromagnetic spectrum in terms of spectral bands and their spectral bandwidths [21].</caption><thead><tr><th>EM Radiation Regions</th><th>Wavelength</th><th>Applications</th></tr></thead><tbody><tr><td>Gamma-ray</td><td>&lt;10<sup>-3</sup> nm</td><td>Food purification, medical hygienic and nuclear medicine</td></tr><tr><td>X-ray</td><td>0.1 to 10<sup>-3</sup> nm</td><td>Medical diagnostics and industry area</td></tr><tr><td>Ultra-violet</td><td>3 to 400 nm</td><td>For computer/lab imaging, fluorescence microscopy and food (meat) quality evaluation</td></tr><tr><td>Visible</td><td>Red: 610-700 nm<br>Orange: 590-610 nm<br>Yellow: 570-590 nm<br>Green: 500-570 nm<br>Blue: 450-500 nm</td><td>Environment monitoring, land surface dynamics assessment, mineral exploration</td></tr><tr><td>Near infrared</td><td>Near infrared (NIR): 0.7 to 1.5 µm<br>Short wave infrared (SWIR): 1.5 to 3 µm<br>Mid wave infrared (MWIR): 3 to 8 µm<br>Long wave infrared (LWIR): 8 to 15 µm<br>Far infrared (FIR): &gt;15 µm</td><td>Earth observation<br>Vegetation monitoring</td></tr><tr><td>Microwave</td><td>P band: 30-100 cm<br>L band: 15-30 cm<br>S band: 7.5-15 cm<br>C band: 3.8-7.5 cm<br>X band: 2.4-3.8 cm<br>Ku band: 1.7-2.4 cm<br>K band: 1.1-1.7 cm</td><td>Imaging for cloud free and weather independent acquisition.<br>Location tracking</td></tr><tr><td>Radio</td><td>Very short wave, VHF: 1-10 m<br>Short wave, HF: 10-100 m<br>Medium wave, MF: 0.1-1 km<br>Long wave, LF: 1-10 km<br>Very long wave, VLF: 10-100 km</td><td>Communication operations,<br>Astronomical applications, magnetic resonance imaging (MRI)</td></tr></tbody></table>
<figure><img src="image_15.png" /><figcaption>Figure 1.3 Demonstration of the concept of radiometric resolution. The larger the bit depth, the higher the radiometric resolution (more number of intensity levels).</figcaption></figure>

<figure><img src="image_16.png" /><figcaption>Figure 1.4 The temporal resolution of a satellite imaging system refers to how often it records imagery of a particular area. This example depicts the systematic collection of data every 30 days, presumably at approximately the same time of day.</figcaption></figure>

### 1.1.2. Satellite images: Acquisition types and sensors

One of the most common ways to classify sensor systems is on the basis of the mechanism they use to detect electromagnetic energy. In this regard, two broad groups of sensors: i) Passive, which simply collect radiation derived from external sources, and ii) Active, in which the sensor system is able to provide both, the energy source and detect reflection of the energy from the observed surfaces. Obviously, active sensors are more flexible because they are independent of the sensor-earth illumination conditions. A further distinction among passive sensors is to divide them according to the method used to record the incoming energy.

**Passive Imaging:** Passive remote imaging works in one of the following three ways: i) By observing how radiation changes as it passes through objects (such as sunlight passing through a thin clouds), ii) By observing radiation emitted by an object of interest (such as the emission of infrared radiation by earth), or iii) By observing radiation reflected or scattered by an object of
interest (such as a visible satellite image taken from space). These three methods take advantage of naturally occurring radiation (such as sunlight or infrared radiation emitted by objects) and as such, provide a wealth of information all around us. For example, human eyes serve as types of passive remote imaging devices - our eyes capture scattered and emitted visible light, which the human brain process into images. In contrast, satellite imaging devices such as the Landsat series satellites or the Sentinel-2 satellite constellation have aboard them sensors that can record electromagnetic radiation beyond visible light and provide a great deal of information about our environment.

**Active Imaging:** Active remote imaging devices work by, first emitting a signal, and then recording the "return" of the emitted signal from an object of interest. It includes, observing how long it took the signal to return, the magnitude of the returned signal, and changes in the signal properties such as frequency shifts or orientation (phase, polarization) of the signal. In nature, bats make extensive use of their natural active remote imaging system. They emit ultrasonic 'squeaks' which bounce off from objects around them including the 'moths'. The bats use this active imaging method to locate moths for food. A bat can 'see' (locate) a moth flying in the dark by processing to the changes in the returned ultrasonic signal from the moths. Whales and dolphins use a similar system to augment their other senses for communications and hunting. Their system is termed as sound navigation and ranging or more popularly as 'SONAR'. Satellite-based active remote imaging systems include RADARS (an acronym for radio detection and ranging) and LIDARS (an acronym for light detection and ranging) which work by emitting electromagnetic radiation (either radio waves, in the case of radar, or a laser beam for the LIDAR system) in the direction of the object to be sampled, and then utilizing a parabolic dish to collect the return (called 'backscatter') radiation for processing by computers.

## Popular satellite imaging modalities

### A. Optical imaging for earth observation

Imaging in the optical region of the electromagnetic (EM) spectrum implies use of visible, near infrared and short-wave infrared sensors to form images of the earth's surface by detecting the solar radiation reflected or scattered from objects on the earth's surface. Different earth surface objects reflect and absorb differently at different wavelengths. Thus, these objects can be identified, differentiated, or detected based on their spectral reflectance signatures recorded by satellite optical sensors. Optical imaging systems are mainly classified into four types namely panchromatic, multispectral, superspectral, and hyperspectral depending on the number of spectral bands used in the imaging process.
In panchromatic imaging systems, the imaging sensor is a single wide spectral band (channel) detector sensitive to radiation covering the visible light. A panchromatic image (grey scale image) is one band spread over a couple of hundred nanometres bandwidth. The bandwidth enables the panchromatic sensor to hold a high signal-noise, making the panchromatic image available at a higher spatial resolution. The physical quantity being measured is the apparent brightness of the targets. The spectral information or "colour" of the targets is lost. Recent examples of panchromatic imaging systems are: WorldView-PAN, IKONOS-PAN, GeoEye-PAN, SPOT high resolution visible (HRV-PAN) etc.

The image acquisition sensor in a multispectral imaging system is a multiband/multichannel detector. Each channel is sensitive to radiation within a narrow wavelength band. The resulting image is a multilayer image which contains both the brightness and spectral (colour) information of the objects being observed. Examples of multispectral systems are: Landsat MSS/TM/ETM/ETM+/OLI, Sentinel-2 A/B multispectral instrument (MSI), etc. The band specifications of few multispectral imaging sensors are discussed here.

For example, Landsat-8 operational land imager (OLI) and thermal infrared sensor (TIRS) images consist of nine spectral bands with a spatial resolution of 30 meters for Bands 1 to 7 and 9. Band 1 (ultra-blue) is useful for coastal and aerosol studies. Band 9 is useful for Cirrus cloud detection. The resolution for Band 8 (panchromatic) is 15 meters. Thermal bands 10 and 11 are useful in providing more accurate surface temperatures and are collected at 100 meters. Approximate scene size is 170 km north-south by 183 km east-west. Landsat-8 band specifications are listed in Table 1.2. The sensors on Landsat-9 (launch ready mid-2021) are being designed as improved copies of Landsat-8. Sentinel-2 MSI covering 13 spectral bands (443-2190 nm), with a swath width of 290 km and a spatial resolution of 10 m (four visible and near-infrared bands), 20 m (six red edge and shortwave infrared bands) and 60 m (three atmospheric correction bands). Sentinel-2 band specifications are listed in Table 1.3. The Sentinel-2A MSI data has spectral bands very similar to Landsat 8 (excluding the thermal bands of Landsat 8's thermal infrared sensor).
<table><caption>Table 1.2 Landsat-8 (multispectral) OLI and TIRS band specifications [22].</caption><thead><tr><th>Bands</th><th>Bandwidth (µm)</th><th>Resolution (m)</th><th>Potential Applications</th></tr></thead><tbody><tr><td>Band 1- Coastal aerosol</td><td>0.43-0.45</td><td>30</td><td>Coastal and aerosol studies</td></tr><tr><td>Band 2- Blue</td><td>0.45-0.51</td><td>30</td><td>Bathymetric mapping, distinguishing soil from vegetation and deciduous from coniferous vegetation</td></tr><tr><td>Band 3- Green</td><td>0.53-0.59</td><td>30</td><td>Emphasizes peak vegetation, which is useful for assessing plant vigour</td></tr><tr><td>Band 4- Red</td><td>0.64-0.67</td><td>30</td><td>Discriminates vegetation slopes</td></tr><tr><td>Band 5- Near Infrared (NIR)</td><td>0.85-0.88</td><td>30</td><td>Emphasizes biomass content and shorelines</td></tr><tr><td>Band 6- SWIR 1</td><td>1.57-1.65</td><td>30</td><td>Discriminates moisture content of soil and vegetation, penetrates thin clouds</td></tr><tr><td>Band 7- SWIR 2</td><td>2.11-2.29</td><td>30</td><td>Improved moisture content of soil and vegetation and thin cloud penetration</td></tr><tr><td>Band 8- Panchromatic</td><td>0.50-0.68</td><td>15</td><td>Sharper image definition</td></tr><tr><td>Band 9- Cirrus</td><td>1.36-1.38</td><td>30</td><td>Improved detection of cirrus cloud contamination</td></tr><tr><td>Band 10- Thermal Infrared (TIRS) 1</td><td>10.6-11.19</td><td>100</td><td>Thermal mapping and estimated soil moisture</td></tr><tr><td>Band 11- Thermal Infrared (TIRS) 2</td><td>11.50-12.51</td><td>100</td><td>Improved thermal mapping and estimated soil moisture</td></tr></tbody></table>

<table><caption>Table 1.3 Sentinel-2 band specifications [23].</caption><thead><tr><th>Bands</th><th>Wavelength (µm)</th><th>Resolution (m)</th><th>Potential Applications</th></tr></thead><tbody><tr><td>Band 1- Coastal aerosol</td><td>0.443</td><td>60</td><td>Aerosol scattering</td></tr><tr><td>Band 2- Blue</td><td>0.490</td><td>10</td><td>Bathymetric mapping</td></tr><tr><td>Band 3- Green</td><td>0.560</td><td>10</td><td>Bathymetric mapping</td></tr><tr><td>Band 4- Red</td><td>0.665</td><td>10</td><td>Bathymetric mapping</td></tr><tr><td>Band 5- Vegetation Red Edge</td><td>0.705</td><td>20</td><td>Vegetation classification</td></tr><tr><td>Band 6- Vegetation Red Edge</td><td>0.740</td><td>20</td><td>Vegetation classification</td></tr><tr><td>Band 7- Vegetation Red Edge</td><td>0.783</td><td>20</td><td>Vegetation classification</td></tr><tr><td>Band 8- NIR</td><td>0.842</td><td>10</td><td>chlorophyll, biomass</td></tr><tr><td>Band 8A- Vegetation Red Edge</td><td>0.865</td><td>20</td><td>Vegetation classification</td></tr><tr><td>Band 9- Water Vapour</td><td>0.945</td><td>60</td><td>Detection of thin cirrus</td></tr><tr><td>Band 10- SWIR Cirrus</td><td>1.375</td><td>60</td><td>Water vapour absorption</td></tr><tr><td>Band 11- SWIR</td><td>1.610</td><td>20</td><td>lignin, starch, and forest above biomass</td></tr><tr><td>Band 12- SWIR</td><td>2.190</td><td>20</td><td>Distention to live biomass</td></tr></tbody></table>

A superspectral imaging sensor has typically 15 to 50 spectral channels. The bands have narrower bandwidths, enabling the finer spectral characteristics of the targets to be captured by the sensor. An example of superspectral imaging system is: moderate-resolution imaging spectroradiometer (MODIS) which have 36 spectral bands and its band specifications are listed in Table 1.4.
<table><caption>Table 1.4 MODIS sensor band specifications and primary usage [25].</caption><thead><tr><th>Band</th><th>Bandwidth<sup>1</sup></th><th>Spectral Radiance<sup>2</sup></th><th>Required SNR<sup>3</sup></th><th>Potential Applications</th></tr></thead><tbody><tr><td>1</td><td>620-670</td><td>21.8</td><td>128</td><td>Land/Cloud/Aerosols Boundaries</td></tr><tr><td>2</td><td>841-876</td><td>24.7</td><td>201</td><td></td></tr><tr><td>3</td><td>459-479</td><td>35.3</td><td>243</td><td>Land/Cloud/Aerosols Properties</td></tr><tr><td>4</td><td>545-565</td><td>29.0</td><td>228</td><td></td></tr><tr><td>5</td><td>1230-1250</td><td>5.4</td><td>74</td><td></td></tr><tr><td>6</td><td>1628-1652</td><td>7.3</td><td>275</td><td></td></tr><tr><td>7</td><td>2105-2155</td><td>1.0</td><td>110</td><td></td></tr><tr><td>8</td><td>405-420</td><td>44.9</td><td>880</td><td>Ocean Colour/ Phytoplankton/ Biogeochemistry</td></tr><tr><td>9</td><td>438-448</td><td>41.9</td><td>838</td><td></td></tr><tr><td>10</td><td>483-493</td><td>32.1</td><td>802</td><td></td></tr><tr><td>11</td><td>526-536</td><td>27.9</td><td>754</td><td></td></tr><tr><td>12</td><td>546-556</td><td>21.0</td><td>750</td><td></td></tr><tr><td>13</td><td>662-672</td><td>9.5</td><td>910</td><td></td></tr><tr><td>14</td><td>673-683</td><td>8.7</td><td>1087</td><td></td></tr><tr><td>15</td><td>743-753</td><td>10.2</td><td>586</td><td></td></tr><tr><td>16</td><td>862-877</td><td>6.2</td><td>516</td><td></td></tr><tr><td>17</td><td>890-920</td><td>10.0</td><td>167</td><td>Atmospheric Water Vapour</td></tr><tr><td>18</td><td>931-941</td><td>3.6</td><td>57</td><td></td></tr><tr><td>19</td><td>915-965</td><td>15.0</td><td>250</td><td></td></tr><tr><td>20</td><td>3.660-3.840</td><td>0.45 (300 K)</td><td>0.05</td><td>Surface/ Cloud Temperature</td></tr><tr><td>21</td><td>3.929-3.989</td><td>2.83 (335 K)</td><td>2.00</td><td></td></tr><tr><td>22</td><td>3.929-3.989</td><td>0.67 (300 K)</td><td>0.07</td><td></td></tr><tr><td>23</td><td>4.020-4.080</td><td>0.79 (300 K)</td><td>0.07</td><td></td></tr><tr><td>24</td><td>4.433-4.498</td><td>0.17 (250 K)</td><td>0.25</td><td>Atmospheric Temperature</td></tr><tr><td>25</td><td>4.482-4.549</td><td>0.59 (275 K)</td><td>0.25</td><td></td></tr><tr><td>26</td><td>1.360-1.390</td><td>6.00</td><td>150</td><td>Cirrus Cloud Water vapour</td></tr><tr><td>27</td><td>6.535-6.895</td><td>1.16 (240 K)</td><td>0.25</td><td></td></tr><tr><td>28</td><td>7.175-7.475</td><td>2.18 (250 K)</td><td>0.25</td><td></td></tr><tr><td>29</td><td>8.400-8.700</td><td>9.58 (300 K)</td><td>0.05</td><td>Cloud Properties</td></tr><tr><td>30</td><td>9.580-9.880</td><td>3.69 (250 K)</td><td>0.25</td><td>Ozone</td></tr><tr><td>31</td><td>10.780-11.280</td><td>9.55 (300 K)</td><td>0.05</td><td>Surface/ Cloud Temperature</td></tr><tr><td>32</td><td>11.770-12.270</td><td>8.94 (300 K)</td><td>0.05</td><td></td></tr><tr><td>33</td><td>13.185-13.485</td><td>4.52 (260 K)</td><td>0.25</td><td>Cloud Top Altitude</td></tr><tr><td>34</td><td>13.485-13.785</td><td>3.76 (250 K)</td><td>0.25</td><td></td></tr><tr><td>35</td><td>13.785-14.085</td><td>3.11 (240 K)</td><td>0.25</td><td></td></tr><tr><td>36</td><td>14.085-14.385</td><td>2.08 (220 K)</td><td>0.35</td><td></td></tr></tbody></table>

<footnote n="1">¹Bands 1 to 19 are in nm; Bands 20 to 36 are in µm</footnote>

<footnote n="2">²Spectral Radiance values are (W/m2 – µm-sr)</footnote>

<footnote n="3">³SNR = Signal- to – Noise Ratio</footnote>

<footnote n="4">⁴NE (Delta)T = Noise-equivalent temperature difference</footnote>

<footnote>Note: Performance goal is 30-40 % better than required</footnote>
<table><caption>Table 1.5 Hyperion sensor specifications [26].</caption><thead><tr><td>Specification</td><td>Setting</td></tr></thead><tbody><tr><td>Spatial resolution</td><td>30 m</td></tr><tr><td>Swath width</td><td>7.75 km</td></tr><tr><td>Spectral channels</td><td>220 unique channels:<br>VNIR (70 channels, 356 nm-1058 nm)<br>SWIR (172 channels, 852 nm-2577 nm)</td></tr><tr><td>Spectral bandwidth</td><td>10nm (nominal)</td></tr><tr><td>Digitization</td><td>12 bits</td></tr><tr><td>Signal-to-noise ratio (SNR)</td><td>161 (550 nm)<br>147(700 nm)<br>110 (1125 nm)<br>40 (2125 nm)</td></tr></tbody></table>

A hyperspectral imaging system, also known as an "imaging spectrometer" collects information in about a hundred or more contiguous spectral bands. The goal is to collect precise spectral information about objects of interest such as minerals, oil, gas, forests or others. This information enables better characterisation and identification of objects. Hyperspectral images have potential applications in fields as coastal management (e.g. monitoring of phytoplankton, pollution, bathymetry changes) [24], forestry, water spectroscopy, mineral exploration, precision agriculture (e.g. monitoring the types, health, moisture status and maturity of crops). An example of a hyperspectral system is: Hyperion which has 220 spectral bands. Hyperion sensor specifications are listed in Table 1.5.

## B. Microwave imaging for earth observation

Electromagnetic radiation in the microwave region is also used in imaging to provide useful information about the earth's surface and objects on it. There are various types of microwave imaging systems developed to capture information in different forms.

A microwave radiometer is a passive sensing device which records the microwave emissions emitting naturally from the earth's surface. The sensing technology can be used in many applications such as measuring and monitoring sea surface temperature or land surface temperature etc.

A radar altimeter sends out pulses of microwave energy and records the energy scattered back from the object on the earth surface. The height of the object from a certain point on earth's surface can be measured based on the time taken by the energy signal to return.

A wind scatterometer can be used to measure wind speed and direction over the ocean surface. It sends out pulses of microwaves along several directions and records the magnitude of
the signals backscattered from the ocean surface. The magnitude of the backscattered signal is related to the ocean surface roughness, which in turns is dependent on the sea surface wind condition, and hence the wind speed and direction can be derived from platforms to generate high resolution images of the earth surface using microwave energy.

## Synthetic aperture radar

In synthetic aperture radar (SAR) imaging, microwave pulses are transmitted by an antenna towards the earth surface at an oblique angle and the microwave energy scattered back towards the antenna (backscatter) is measured and used to analyse earth surface features. The SAR imaging system form an image by utilising the magnitude and time delay of the backscattered signals. Mathematically, this backscattered signal is represented using radar cross section (RCS) $\sigma$, which is defined as the ratio between the incident and received signal intensity. The microwave energy beam sent out by the antenna towards the earth's surface illuminates an area on the ground. The illuminated area is termed as the antenna's "footprint". The recorded or received signal strength depends on the microwave energy backscattered from the objects inside this footprint. In real aperture radar imaging, the ground resolution is limited by the size of the microwave beam sent out from the antenna. Finer details on the ground can be resolved by using a narrower beam. The beam width is inversely proportional to the size of the antenna i.e. the shorter the antenna, the wider is the beam or footprint and the longer the antenna, the narrower the beam or footprint. Increasing the length of the antenna will decrease the width of the footprint. It is not feasible for a spacecraft to carry a very long antenna which is required for high resolution imaging of the earth surface. To overcome this limitation, a SAR capitalises on the motion of the space craft to emulate a large antenna from a small antenna it actually carries on board.

## SAR polarimetry

In general, when an EM wave is emitted from a source (e.g. a radar antenna), it propagates in all directions (a spherical wavefront), with a specific field strength and a phase in each direction. At a long distance from the antenna, it can be assumed that the wavefront of the emitted EM wave lies on a plane, rather than on the surface of a sphere. Polarization is an important property of an EM wave. Polarization refers to the alignment and regularity of the electric and magnetic field components of the EM wave, in a plane perpendicular to the direction of propagation. In particular, EM wave can be characterized by the behaviour of the electric field vector as a function of time. The propagation of an EM wave is illustrated in Figure 1.5. A scatterer (object on the surface of earth in this case) can change the polarization of the EM wave incident upon it. The polarization of the scattered EM wave can be different from the polarization of the incident
EM wave. Usually, a radar antenna is often designed to receive a predefined set of polarization components of the scattered EM wave simultaneously. For example, the horizontal (H) and vertical (V) parts of an antenna can receive the two orthogonal components of the incoming EM wave, and the system electronics keep these two signals separate.

<figure><img src="image_17.png" /><figcaption>Figure 1.5 Illustration of the propagation of an electromagnetic plane wave. The electric field vector has horizontal (green) and vertical (blue) components, which combine to yield the net electric field vector (red) [27].</figcaption></figure>

Denoting the transmit and receive polarizations by a pair of symbols, a radar system using H and V linear polarizations can thus have the following channels:

* HH - for horizontal transmit and horizontal receive,

* VV - for vertical transmit and vertical receive,

* HV - for horizontal transmit and vertical receive, and

* VH - for vertical transmit and horizontal receive.

The first two of these polarization combinations are referred to as like-polarized or co-polarized, because the transmit and receive polarizations are the same. The last two combinations are referred to as cross-polarized because the transmit and receive polarizations are orthogonal to one another [28].

A radar system can have different levels of polarization complexity. Popular polarization configurations are listed here (though not limited to).

* Single polarized (single channel) - HH or VV or HV or VH

* Dual polarized (dual channel) - HH and HV, VV and VH, or HH andVV

* Four or full polarized (four channels) - HH, VV, HV, and VH

A quadrature polarized (i.e. fully polarimetric) radar uses these four polarizations, and measures the phase difference between the channels as well as the magnitudes. Some dual polarized radars also measure the phase difference between channels, as this phase plays an important role in polarimetric information extraction.

When a horizontally polarized wave is incident upon a target, the backscattered wave can have contributions in both horizontal and vertical polarizations. The same applies to a vertically
polarized incident wave. As the horizontal and vertical components form a complete basis set to describe an electromagnetic wave, the backscattering properties of the target can be described by a scattering matrix, S as.

$$[S] = \begin{bmatrix} S_{HH} & S_{HV} \\ S_{VH} & S_{VV} \end{bmatrix} \qquad (1.2)$$

The four elements of the scattering matrix are complex, and can be obtained from the magnitudes and phases measured by the four channels of the receiving antenna of a polarimetric radar.

The different polarization channels record different scattering mechanisms, and hence offer different information of objects as mentioned in Table 1.6. HH, HV, VH, and VV polarizations provide double-bounce, volume, and surface scattering respectively. For example, HH polarization gives double-bounce scattering, which is usually obtained from buildings in a landscape consisting of a variety of land covers.

## Historical, current and future SAR sensors:

Spaceborne SAR sensors have been in use for more than 40 years. The first SAR sensor was launched on June 28, 1978, on board the NASA's Seasat satellite. The SAR sensor was developed and deployed, aimed at monitoring oceanographic phenomena. Seasat carried an HH-polarized L-band SAR sensor mounted at a fixed angle to observe the global 'surface wave fields' and polar sea ice conditions as part of its sensor suite. Even though the Seasat's SAR sensor operated for only 106 days, the mission was deemed an extensive success, demonstrating a SAR's capability to monitor both ocean and land surface.

Since then, SAR imaging has made a long and successful journey. Several SAR sensors with ever-improving imaging capabilities have been used by organizations, communities, nations collectively ensuring continuous coverage of the earth and availability of SAR data [29]. Unfortunately, this constellation of SAR systems comes with a downside. The SAR sensors launched by the various agencies vary widely in their sensor configurations. The data from the different sensors are not always directly comparable with each other.

<table><caption>Table 1.6 Polarimetric parameters.</caption><thead><tr><td>Parameter</td><td>Application (Discrimination)</td></tr></thead><tbody><tr><td>HH Polarization</td><td>Double bounce scattering</td></tr><tr><td>HV Polarization</td><td>Volume scattering</td></tr><tr><td>VH Polarization</td><td>Volume scattering</td></tr><tr><td>VV Polarization</td><td>Surface scattering</td></tr></tbody></table>
SAR sensors transmit energy in one of the microwave frequency bands shown in Table 1.7. Roughly speaking, radar systems use frequencies from 1 to 90 GHz. This frequency range is subdivided into multiple frequency bands, as shown in the first column of Table 1.7. These bands were defined initially according to the different equipment(s) needed to generate and detect these particular frequencies. However, now they are understood as the equivalent of colours in the visual range. Unfortunately, the latter-based naming scheme (Ka-band to P-band) was used during World war II and never deemed necessary to be modified since then. This nomenclature sometimes leads to confusion among new users of SAR.

Not all of the microwave bands shown in Table 1.7 used are for SAR imaging. While some experimental airborne Ka- and Ku- band SAR systems exist, civilian spaceborne sensors have been using the frequency bands ranging from P- to X-band.

The operating wavelength of a SAR sensor is linked intrinsically to its transmitted signal's penetration capabilities. It means that longer wavelength signals (e.g., signals at L- and P-band) penetrate deeper into land covers and earth surface compared to shorter wavelength signals i.e. at C- or X-band. Hence, the applications supported by a SAR sensor depends on the frequency band used.

Table 1.7 also summarizes typical applications of SAR as a function of the frequency band. It shows that sensors at X-band predominantly are used for built-up and infrastructure monitoring. Due to the X-band's higher resolution capabilities, SAR sensors at this frequency find broad applications in surveillance and tracking. These SAR sensors are also often used in the monitoring of industrial installations. Due to its limited penetration into vegetation covers, X-band is rarely used to characterise forest canopies or monitor activity underneath vegetation.

With the predominant number of SAR sensors operating at this frequency range, C-band sensors have been the workhorse of SAR based monitoring over the last 30 years. C-band data can be seen as a good compromise between X-band and the longer wavelength (e.g. L-band) sensor classes because of its moderate- to high-resolution capabilities and increased vegetation penetration. In comparison to X-band SARs in terms of swath widths, C-band SARs typically allow for wider swath imaging, lending its applicability to regional and global scale. While C-band has improved canopy penetration capabilities, its signals will typically not penetrate vegetation completely. C-band is of limited use for monitoring activity underneath vegetation canopies, especially in regions with dense vegetation.

While S-band SAR sensors were used rarely in earth observation in the past, this frequency will have increased usage soon. NovaSAR-S, a S-band SAR sensor, was launched in September 2018. NovaSAR-S will provide moderate-resolution SAR data. This data will help explore S-band data's performance for applications such as disaster monitoring, crop monitoring,
forest monitoring (temperate and rainforests), and land cover mapping. More interesting to most users will be the upcoming NASA-ISRO SAR satellite, NISAR. In addition to an L-band SAR, NISAR will carry a fully polarimetric S-band SAR. While NISAR's S-band coverage will likely not be global, all data will be freely and openly available to the SAR science and applications community.

While most historical SAR systems operated in C-band, future SAR sensors are focused on the L-band frequency range. Although L-band SARs do not provide the high-resolution capabilities of shorter wavelength SARs, their ability to penetrate vegetation holds several advantages for earth observation. With a higher likelihood of seeing the ground underneath vegetation cover, L-band SARs are useful for mapping activity underneath canopies such as flooding or troop movements. Due to the high penetration into vegetation covers, L-band SAR can also characterise canopy structure, especially in dense vegetation. The higher canopy penetration ability of L-band SARs is also advantageous to interferometric SAR (InSAR) users in terms of achieving higher interferometric coherence and better deformation tracking.

P-band SAR sensors are currently under development. The major issue with P-band SAR's data is 'ionospheric distortions'. These ionospheric distortions can adversely hamper the quality of data. However, recent developments in ionospheric correction strategies have allowed spaceborne P-band SAR missions to go forward. The first spaceborne P-band SAR—ESA's BIOMASS mission is planned to launch in 2021 and will focus on mapping the status and dynamics of earth's forests.

A list of the most relevant past, current, and future SAR platforms is provided in Table 1.8. The sensors are sorted by their period of functioning. For each instrument, sensor's operating wavelength, supported polarization modes, resolution and size of image products, repeat cycle, and data access means are listed.
<table><caption>Table 1.7 Designation of microwave bands [30].</caption><thead><tr><th>Band</th><th>Frequency (GHz)</th><th>Wavelength (cm)</th><th>Typical Application</th></tr></thead><tbody><tr><td>Ka</td><td>27-40</td><td>1.1-0.8</td><td>Rarely used for SAR (airport surveillance)</td></tr><tr><td>K</td><td>18-27</td><td>1.7-1.1</td><td>Rarely used (H<sub>2</sub>O absorption)</td></tr><tr><td>Ku</td><td>12-18</td><td>2.4-1.7</td><td>Rarely used for SAR (satellite altimetry)</td></tr><tr><td>X</td><td>8-12</td><td>3.8-2.4</td><td>High-resolution SAR (urban monitoring; ice and snow, little penetration into vegetation cover; fast coherence decay in vegetated areas)</td></tr><tr><td>C</td><td>4-8</td><td>7.5-3.8</td><td>SAR workhorse (global mapping; change detection; monitoring of areas with low to moderate vegetation; improved penetration; higher coherence); Ice, ocean, maritime navigation</td></tr><tr><td>S</td><td>2-4</td><td>15-7.5</td><td>Little but increasing use for SAR-based earth observation; agriculture monitoring (NISAR will carry an S-band channel; expands C-band applications to higher vegetation density)</td></tr><tr><td>L</td><td>1-2</td><td>30-15</td><td>Medium resolution SAR (Geophysical monitoring; biomass and vegetation mapping; high penetration; InSAR)</td></tr><tr><td>P</td><td>0.3-1</td><td>100-30</td><td>Biomass. First P-band spaceborne SAR will be launched ~2020; vegetation mapping and assessment. Experimental SAR.</td></tr></tbody></table>

<table><caption>Table 1.8 Brief list of few past, current and upcoming spaceborne SAR sensors with their properties [30].</caption><thead><tr><th>Sensor</th><th>Lifetime</th><th>Wavelength /Frequency</th><th>Polarization</th><th>Resolution</th><th>Frame Size</th><th>Repeat Cycle</th><th>Access</th></tr></thead><tbody><tr><td>Seasat</td><td>1978</td><td>L-band<br>λ=24.6 cm</td><td>HH</td><td>Az:25m<br>Rg:25m</td><td>100km</td><td>-</td><td>Free& Open</td></tr><tr><td>ERS-1</td><td>1991-2001</td><td>C-band<br>λ=05.6 cm</td><td>VV</td><td>Az:6-30m<br>Rg:26m</td><td>100km</td><td>35days</td><td>Restrained</td></tr><tr><td>JERS-1</td><td>1995-1998</td><td>L-band<br>λ=24.6 cm</td><td>HH</td><td>Az:18m<br>Rg:18m</td><td>75km</td><td>44days</td><td>Restrained</td></tr><tr><td>ERS-2</td><td>1995-2011</td><td>C-band<br>λ=05.6 cm</td><td>VV</td><td>Az:6-30m<br>Rg:26m</td><td>100km</td><td>35days</td><td>Restrained</td></tr><tr><td>ENVISAT</td><td>2002-2012</td><td>C-band<br>λ=05.6 cm</td><td>HH, VV, VV/VH</td><td>Az:28m<br>Rg:28m</td><td>100km</td><td>35days</td><td>Restrained</td></tr></tbody></table>
<table><tr><td>ALOS-1</td><td>2006-2011</td><td>L-band<br/>λ =24.6 cm</td><td>FBS: HH, VV<br/>FBD: HH/HV, HH/VH<br/>PLR: HH/HV/VH/VV<br/>ScanSAR: HH, HV</td><td>FBS:10*10m<br/>FBD:10*10m<br/>PLR:30*10m<br/>ScanSAR:100m</td><td>FBS:70km<br/>FBD:70km<br/>PLR:30km<br/>ScanSAR:250-350km</td><td>46days</td><td>Free &amp; Open</td></tr><tr><td>Radarsat-1</td><td>1995-2013</td><td>C-band<br/>λ =05.6 cm</td><td>HH</td><td>Standard: 25x28m<br/>Fine: 9x9m<br/>Wide1: 35x28m<br/>Wide2: 35x28m<br/>ScanSAR: 50x50-100x100m</td><td>Standard: 100km<br/>Fine: 45km<br/>Wide1: 165km<br/>Wide2: 150km<br/>ScanSAR: 305-510km</td><td>24 days</td><td>1995-2008:<br/>Restrained<br/>2008-2013:<br/>Commercial</td></tr><tr><td>TerraSAR-X<br/>TanDEM-X</td><td>2007-2010</td><td>X-band<br/>λ = 03.5cm</td><td>Single: HH, VV<br/>Dual: HH/VV,<br/>HH/HV, VV/VH<br/>Twin: HH/VV,<br/>HH/VH, VV/VH</td><td>Spotlight: 0.2x1.0-1.7x3.5m<br/>Stripmap: 3x3m<br/>ScanSAR: 18-40m</td><td>Spotlight: 3-10km<br/>Stripmap: 50x30km<br/>ScanSAR: 150x100-200x200km</td><td>11 days</td><td>Application-dependent;<br/>restrained<br/>scientific,<br/>commercial</td></tr><tr><td>Radarsat-2</td><td>2007-</td><td>C-band<br/>λ = 05.6cm</td><td>Single: HH, VV, HV,<br/>VH<br/>Dual: HH/HV, VV/VH<br/>Quad: HH/HV/VH/VV</td><td>Spotlight: ~1.5m<br/>Stripmap: ~3x3-25x25m<br/>ScanSAR: 35x35-100x100m</td><td>Spotlight: 18x8km<br/>Stripmap: 20-170m<br/>ScanSAR: 300x300-500x500km</td><td>24 days</td><td>Commercial</td></tr><tr><td>COSMO-SkyMed</td><td>2007-</td><td>X-band<br/>λ = 03.5cm</td><td>Single: HH, VV, HV,<br/>VH<br/>Dual: HH/HV,<br/>HH/VV, VV/VH</td><td>Spotlight: ≤1m<br/>Stripmap: 3-15m<br/>ScanSAR: 30-100m</td><td>Spotlight: 10x10km<br/>Stripmap: 40x40km<br/>ScanSAR: 100x100-200x200km</td><td>Satellite: 16 days<br/>Constellation: ~hrs</td><td>Commercial;<br/>limited<br/>proposal-based<br/>scientific</td></tr><tr><td>XSAR<br/>RISAT-2</td><td>2009-</td><td>X-band<br/>λ = 03.1cm</td><td>Single: HH, VV, HV,<br/>VH<br/>Dual: HH/HV,<br/>HH/VV, VV/VH<br/>Quad: HH/HV/VH/VV</td><td>Spotlight: ≤1m<br/>Stripmap: 3m<br/>Mosaic:<br/>1.8m<br/>ScanSAR:<br/>8m</td><td>10x10-120x120km</td><td>14 days</td><td>Restrained<br/>scientific</td></tr></table>
<table><tr><td>ALOS-2<br>PALSAR-2</td><td>2014-</td><td>L-band<br>λ = 24.6cm</td><td>Single: HH, VV, HV,<br>VH<br>Dual: HH/HV, VV/VH<br>Quad: HH/HV/VH/VV</td><td>Spotlight: 1x3m<br>Stripmap: 3-10m<br>ScanSAR: 25-100m</td><td>Spotlight: 25x25km<br>Stripmap: 55x70-70x70km<br>ScanSAR: 355x355km</td><td>14 days</td><td>Commercial;<br>limited<br>proposal-based<br>scientific</td></tr><tr><td>Sentinel-1</td><td>2014-</td><td>C-band<br>λ = 05.6cm</td><td>Single: HH, VV<br>Dual: HH/HV, VV/VH</td><td>Stripmap: 5x5m<br>Interferometric Wide<br>Swath (IW):<br>5x20m<br>Extra Wide Swath<br>(EW): 20-40m</td><td>Stripmap: 375km<br>IW: 250km<br>EW: 400km</td><td>Satellite: 12<br>days<br>Constellation:<br>6 days</td><td>Free &amp; Open</td></tr><tr><td>SAOCOM</td><td>2018-</td><td>L-band<br>λ = 24.6cm</td><td>Single: HH, VV<br>Dual: HH/HV, VV/VH<br>Quad: HH/HV/VH/VV</td><td>Stripmap: 10x10m<br>Top SAR:<br>100x100m</td><td>Stripmap: &gt;65km<br>Top SAR: 320km</td><td>Satellite: 16<br>days<br>Constellation:<br>8 days</td><td>TBD</td></tr><tr><td>NovaSAR-S<br>S-SAR</td><td>2018-</td><td>S-band<br>λ = 11.75cm</td><td>Single: HH, VV<br>Dual: HH/HV, VV/VH<br>Quad: HH/HV/VH/VV</td><td>ScanSAR = 20m<br>Maritime<br>Surveillance = 30m<br>Stripmap = 6m<br>ScanSAR Wide =<br>30m</td><td>Stripmap = 15-20 km<br>ScanSAR Wide =<br>150km<br>Maritime Surveillance<br>= 750km</td><td></td><td>TBD</td></tr><tr><td>PAZ SAR</td><td>2018-</td><td>X-band<br>λ = 03.5cm</td><td>*See TerraSAR/Tan<br>DEM-x</td><td>*See TerraSAR/Tan<br>DEM-x</td><td>*See TerraSAR/Tan<br>DEM-x</td><td>11 days</td><td>Commercial</td></tr><tr><td>RCM</td><td>2019</td><td>C-band<br>λ = 05.6cm</td><td>Single: HH, VV, VH,<br>HV<br>Dual: HH/HV,<br>VV/VH, HH/VV<br>Compact<br>Quad</td><td>Very high, high,<br>medium, and<br>low-res modes (3-<br>100m)</td><td>20x20-500x500km</td><td>Satellite: 12<br>days<br>Constellation:<br>~hrs</td><td>TBD</td></tr><tr><td>NISAR</td><td>2021</td><td>L-band<br>λ = 24.6cm<br>S-band</td><td>Single: HH, VV, VH,<br>HV</td><td>3-20m (mode<br>dependent)</td><td>250km</td><td>12 days</td><td>Free &amp; Open</td></tr></table>
<table><tr><td>BIOMASS</td><td>2021</td><td>λ = 11.75cm<br>P-band<br>λ = 70.0cm</td><td>Dual: HH/HV,<br>VV/VH, HH/VV<br>Quad<br>Quad</td><td>≤60x50m</td><td>160km</td><td>17 days</td><td>Free &amp; Open</td></tr><tr><td>TanDEM-L</td><td>2023</td><td>L-band<br>λ = 24.6cm</td><td>Single, dual, quad<br>modes</td><td>12*12 m</td><td>350 km</td><td>Satellite: 16<br>days<br>Constellation:<br>8 days</td><td>Free &amp; Open</td></tr></table>
## 1.2. Satellite Images: Summary and Context to the Thesis

It is evident from section 1.1 that satellite images are critical to earth monitoring and are the most techno-economical means of doing so. Following are key takeaways from satellite images based earth monitoring.

Satellite images provide wide coverage, timely response, reliable and consistent information, and high quality [31] earth observation data. Satellite image are available at different characteristics i.e. spatial, spectral, and temporal resolutions. Also, satellite images are available in various modalities such as surface reflectance (multispectral, or hyperspectral), backscatter (active or passive microwave), brightness-temperature (microwave, thermal), polarization (linear, or circular), and others. These different satellite image characteristics and modalities are useful in standalone or synergistic fashion for various applications. This thesis focuses on efficiently utilizing these characteristics and modalities, in standalone or synergistic fashion, for different applications.

On top of this, satellite images are available in large amounts. Specifically, in the last decade, there has been a surge in the availability of high quality (spatial, spectral, and temporal), open-source or economical earth observation satellite images. Today, more than 600 satellites provide earth observation data, mostly in the form of images. Although there are various satellite image modalities available today, but PolSAR (backscatter) and multispectral (surface reflectance) are the most techno-economical, widely used satellite image modalities. This thesis focuses on studies based on these two image modalities for different applications.

Popular satellite image applications include land cover classification and mapping, agriculture crop monitoring, disaster monitoring, environment health and earth surface dynamics monitoring, and mineral exploration. These applications vary from being performance-critical such as land cover classification to being time-critical such as disaster management. Therefore, different applications require focus on one or more characteristics (spatial, spectral or temporal) or modality (spectral reflectance, backscatter, brightness-temperature) of satellite images. Although, *single sensor or unimodal* satellite images are able to provide satisfactory quality spatial, spectral, or temporal information but a trade-off is always at play. For example, if a satellite image has very good spatial resolution, then the satellite sensor is ought to take more time to gather this detailed spatial information than the sensor which provides satellite image with lower spatial information. In short, higher the spatial resolution, lower the temporal resolution, keeping all other aspects same. Similar trade-off holds for spectral and spatial
characteristics of satellite images. These trade-offs hinder the ability of a single sensor satellite image to be sufficient for every application.

Alternatively, use of multimodal and multisensor satellite images address the issues associated with the use of single sensor images. Use of multisensor images improves the spatial, spectral, and temporal characteristics simultaneously and therefore improving the application performance. Moreover, use of multisensor satellite images also address to some other issues associated with unimodal images. For example, multispectral images (modality: surface reflectance) are less noisy and easy to interpret but suffer from issues like obstruction due to cloud cover or no night-time imaging. On the other hand, PolSAR images (modality: backscatter) are weather-independent (can be captured at any time) but suffer from issues such as noise (speckle, layover, foreshortening etc.) and image interpretation. Use of multisensor images resolves these issues to some extent. Therefore, use of multisensor and/or multimodal satellite images for many applications is not uncommon. Also, with the availability of multiple sources of information for any particular application, it is important to exploit all the available sources efficiently and effectively. Therefore, this thesis also focuses to exploit multisensor satellite images for land cover classification, agriculture crop classification, and disaster monitoring.

Satellite time series images are set of images of the same area captured at different time stamps using the same satellite imaging sensor. Multisensor time series images include time series images from more than one satellite sensor. Both, single sensor and multisensor satellite time series images have proved potential in various applications such as change detection, agriculture crop monitoring, etc. This thesis focuses to exploit multisensor satellite time series images for agriculture crop classification.

In summary, unimodal or single sensor satellite images, multimodal or multisensor satellite images, and satellite time series images have advantages, disadvantages, and varied applicability for earth observation. This thesis focuses to exploit all the three categories for different applications.

**1.3. Satellite Image Processing: Recent Challenges and Expectations**

Processing satellite images for utilization in different application is challenging. In general, images acquired and utilized for earth observation are different from normal images because; - first, earth observation images contain significantly higher spatial information. A single image may contain spatial information about from a few kilometres to few hundred kilometres of the earth's surface, and -second, earth observation images also contain relatively higher spectral information. It means unlike the normal camera images which have three spectral bands i.e. red,
green, and blue, spectral bands in these images may vary from tens to hundreds. These unique
characteristics of earth observation images make them challenging to handle and require special
approaches to process. Satellite based earth observation images also possess these properties and
are equally challenging to be utilized in desired applications such as land cover classification or
agriculture crop monitoring. The challenges can be divided into three categories for the sake of
clarity i.e. challenges with single sensor satellite image processing, challenges with multisensor
satellite image processing, and challenges with satellite time series image processing. These
challenges are briefly discussed here.

Single sensor and unimodal satellite images have been used in numerous earth
observation applications. Among a wide variety of applications, land use land cover or land cover
(LC) classification is one of the most important and mundane task. Traditional techniques have
been successful in performing land cover classification with various unimodal satellite images
[32]. Few of the popular image classification techniques are decision trees, random forest [33],
maximum likelihood, support vector machines, artificial neural networks, k-nearest neighbours
[1], logistic regression [2], distance measure [3], K-means, ISODATA, and more. These
techniques have been widely used in satellite image based land cover classification for many
years and have performed well [32]. However, these techniques possess certain limitations such
as requirement of a feature extraction process, high-dimensional data inoperability, sub-optimal
classification, performance saturation with increase in availability of trainable data [5], [34] or
due to local minima, and low generalization performance [35]. Generalization is the ability of a
classifier or classification technique to perform well outside the training sample space.
Researchers in the satellite image processing community are always looking forward to
techniques that can process the high dimensional satellite images and possess abilities like
scalability, automatability, generalizability, and high-performance in mixed land cover scenarios.
For example, satellite polarimetric synthetic aperture radar (PolSAR) earth observation data is
weather independent data but it suffers from issues such as overlapping of class observables
(backscatter) due to weak sensor isolation or speckle noise and therefore often results in sub-
optimal land cover classification with traditional classifiers in mixed class scenarios. Mixed class
scenarios refer to study regions where land cover classes considered for classification are small,
unorganized, and are scattered randomly. Due to this landscape two or more different land covers
appear similar in a satellite image. For example, built-up areas having building structures close
to each other and are randomly oriented (with respect to SAR sensor flight path), might show
volume scattering along with double bounce scattering thus behaving as mixed class. Numerous
techniques are available to perform land cover classification with satellite PolSAR images.
Decomposition based [13], [36]–[41], factorization based [42], and machine learning based techniques [43]–[46] are most popular with satellite PolSAR image based land cover classification. Decomposition and factorization based methods mostly rely on exploring physical scattering properties [47]–[49] and texture information [50], [51] in SAR images. Also, these techniques are mainly based on specific problems and characteristics of SAR data sources. On the other hand, machine learning techniques are more generic in nature but they suffer from limitations such as overfitting and low generalization. Similar issues are observed with other modalities of satellite images. Therefore, more efficient (high performance), scalable and robust techniques are to be explored in order to achieve improved classification performances. Similar to land cover classification, other satellite image based applications such as crop and disaster monitoring can benefit from more advanced techniques. Moreover, traditional approaches with unimodal optical images also suffer from similar challenges.

Alternatively, synergistic exploitation of multimodal or multisensor satellite images is also a challenge for traditional techniques. Numerous multisensor satellite image processing techniques have been developed and have proved successful to some extent [52]–[56]. However, the popular multisensor satellite image processing techniques such as data fusion techniques are data and application intensive and tend to lose information during the fusion process [52], [57]. Machine learning techniques such as neural networks are also popular multisensor satellite image processing techniques but they suffer from overfitting and low generalization performance issues [58]. Therefore, there are still possibilities for improvements in the current multisensor satellite image processing techniques and there is scope for new ones.

Furthermore, efficient processing of satellite time series images is also critical for many applications [59]–[64]. For example, crop type identification and classification is a major step during agriculture crop monitoring [65], [66]. Accurate crop classification requires synergistic exploitation of spatial, spectral and temporal profiles of crops [67]. Numerous techniques have been reported so far, for satellite time series image based crop classification such as in [68], but certain limitations still exist such as presence of other land covers (built-up, forests, or grasslands) in and around the croplands under study. Therefore, it is still a challenge for existing techniques to synergistically exploit the spatial, spectral, and temporal information of a crop present in unimodal/single-sensor or multimodal/multisensor satellite time series images and provide accurate and reliable crop classification results [60].

In summary, literature reports plethora of techniques available for satellite image processing be it for unimodal or single sensor images, multimodal or multisensor satellite images, or satellite time series images. Overall, it is observed that development of techniques that can efficiently exploit different characteristics and/or different modalities offered by satellite images
is tedious and crafty. On top of it, keeping track of more resourceful satellite images every day
and improvising the techniques accordingly is not an easy task at all. Based on these observations,
it is safe to say that any novel technique, approach, method, or model should prefer to possess
qualities such as high performance, universal applicability framework, scalability,
automatability, and generalization. Presence of these attributes will help any technique, approach,
method, or model to keep up with the advancements in satellite images.

1.4. Motivation: Deep Learning in Satellite Image Processing

In this direction of improvements in satellite image processing techniques, deep learning with deep neural networks have shown substantial potential [4]. In the recent years, many satellite image based applications have used deep neural networks and their popularity is still growing [69], [70]. The deep learning theory is a universally applicable approach which can map multiple levels of information to represent complex relationships between data [5]. The deep neural network models are high performance, scalable, automatable, and can learn directly from raw data (less feature handling) [35]. These are the abilities researchers were looking towards from new techniques as discussed in section 1.3. For example, Figure 1.6 indicates the power of deep learning to learn directly from data. The deep learning based models improve their performance with increase in availability of training data in contrast to other machine learning techniques that are unable to learn beyond a certain point no matter how much training data is available. This means that, the more the training data, the better the deep learning based model performs. Another advantage of deep learning is its independency from hand-crafted features. Traditional machine learning techniques require hand-crafted features (not strictly) for learning. Figure 1.7 illustrates this advantage. These attributes or abilities of deep learning based models placed them in prominent position to be widely used for satellite image processing.

Although numerous techniques and methods are available that are based on deep learning
and DNNs, there is a general framework at play. Figure 1.8 illustrates the general framework of
DNN based models for satellite image analysis and processing. The framework includes three
main components; an input data, a core deep networks, and expected output. In practice, the
input–output pairs are dependent on a particular application. When the input–output pairs are
properly defined, the intrinsic and natural relationship between the input and output is then
constructed by a deep network composed of multiple levels of linear and nonlinear operations
and a deep learning algorithm. When the core deep network has been well trained, it can be
employed to predict the expected output of an input (test) whose output is never seen by the core
deep network.
<figure><img src="image_18.png" /><figcaption>Figure 1.6 Comparison of performances of traditional machine learning and deep learning techniques with respect to amount of training data availability.</figcaption></figure>

<figure><img src="image_19.png" /><figcaption>Figure 1.7 Figure illustrates that deep learning models can directly learn from raw data. The feature selection/extraction process is therefore redundant and sub-optimal in these models [71].</figcaption></figure>
<figure><img src="image_20.png" /><figcaption>Figure 1.8 A general framework in DNN based models for satellite image analysis and processing.</figcaption></figure>

For example, Convolutional neural networks or CNNs are recently the most widely used deep network models in satellite image based land cover classification [5]. CNNs have the ability to extract complex features directly from data [35]. Another advantage of focusing on developing CNNs for satellite image processing is that CNNs can directly utilize multi-sensor high dimensional satellite images with efficient performance [72]. Unlike the conventional data fusion methods, this reduces the need of fusing the multisensor images together before using it further. Furthermore, CNNs and recurrent neural networks or RNNs are the most recent and trending deep neural network frameworks for satellite time series image based agriculture crop monitoring applications. Since crop classification is the most challenging as well as important task in many crop monitoring applications, both CNN and RNN models have been developed in, standalone and synergistic modes for agriculture crop classification by many researchers [73], [74]. The aim of these models is to capture the spatial attributes of a particular crop type with CNNs and/or its temporal attributes with RNNs. These studies reported good classification results and promoted further studies. A detailed review on the recent trends and applications of CNNs and RNNs based satellite image processing is provided in chapter 2 here. The review involves major satellite image applications developed on uni- and multi-modal, single- and multi-sensor, single time stamp and time series satellite images with the help of deep learning and DNNs. The review indicates to the significant success and advantage of deep learning methods, especially CNNs and RNNs in the field of satellite image processing. However, the CNN and RNNs deep neural network models are still young, promising, and growing techniques in the satellite image
processing community and further exploration of the potential of these models, especially CNNs
is motivating. Studies such as, improving the generalization performance of CNN based land
cover classification models by focusing on the spectral attributes of satellite images or, exploring
synergistic use of bi-directional RNNs with spectrally focussed CNNs for agriculture crop
monitoring applications may be beneficial.

1.5. Research Gaps in Deep Learning based Satellite Image Processing

A brief literature review is carried out in the field of deep learning based satellite image
processing. Chapter 2 provides brief review, description and discussions on various novel deep
learning based satellite image processing techniques developed for various applications.
Although, these techniques have proved their potential and have gained popularity over the years,
there are areas that require special attention. There are areas of research in deep learning and
DNNs based satellite image processing where improvements are worth mentioning. Few of these
are:

* Although numerous DNN models have been developed so far for satellite image analysis and processing, we believe there is more to explore.

* Improving the performance of current DNN models or development of novel DNN models in mixed land cover scenarios is still a challenge.

* **Spectral information in satellite images is under exploited with current DNN techniques:** Earth observation satellite images contain relatively higher spectral information in comparison to normal camera images. This means unlike the normal camera images which have three spectral bands i.e. red, green, and blue, spectral bands in these images may vary from tens to hundreds. These characteristics of earth observation images makes them challenging to handle and require special attention. Current DNN techniques, especially CNNs and RNNs mostly focus on spatial and temporal characteristics of satellite images during satellite image processing for different applications. Focussing spectral information along with spatial/temporal information might be beneficial.

* **Need more attention on development of more generalized and robust DNN based satellite image processing techniques:** It is intuitive to say that similar to conventional neural networks, DNNs also suffer from overfitting and low generalizability. Special focus is required to improve the generalization ability of DNNs in mixed land cover scenarios. Ensemble training is key to achieve good generalization in DNN models. However, literature reveals lack in ensemble training based DNN models for satellite image processing. Also, visual inspection based evaluation of generalization ability of DNN models needs to be promoted.

* Multisensor satellite images, especially optical and PolSAR complement each other in various land cover applications. There is a lack of CNN and RNNs based models which can exploit multisensor, especially PolSAR and optical images
directly at various temporal, spectral and spatial resolutions for land cover applications. Special focus is required on the complementary spectral information present in high dimensional multisensor satellite images.

* Although, processing time series satellite imagery with CNNs and RNNs for crucial applications such as agriculture crop classification is gaining popularity, the weightage on the exploitation of temporal, spatial, and spectral information via CNNs and RNNs is not well investigated and needs attention. Also, the element of multisensor makes it more interesting and motivating.

* Since PolSAR and multispectral satellite images are the most abundant, economical, and popular modalities, special focus is required to develop CNN and RNN methodologies which can utilize these modalities for different land cover applications.

The above mentioned research gaps are addressed (to some extent) in this thesis and various novel DNN models are sought to be proposed that focus on exploiting the information present in satellite images more carefully.

## 1.6. Thesis Objective

The main objective of this thesis is; the study and critical analysis of the performance of conventional CNN and RNN models and further development of the novel CNN and RNN models to exploit the spectral and/or spatial and/or temporal information present in unimodal/single senor or multimodal/multisensor satellite images. Special focus during the studies is to develop novel deep neural network (DNN) models (common term for CNN and RNN based models) that can achieve better generalization performance in mixed class scenarios.

In the attempt, four tasks are set to forward the stated objective. The tasks are based on; type(s) of satellite image(s) used, application for which the task is realized, multi-modality images involvement, and presence of time series information. The tasks and sub-tasks set to be carried out are:

**Task 1:** Development of CNNs and polarization signatures based on novel approach for improved land cover classification with PolSAR satellite images.

### Sub-tasks:

* Critical analysis of polarization signatures as potential feature for PolSAR image based land cover classification

* Development of novel class boundary estimation algorithm for optimal utilization of polarization signature based features to improve land cover classification accuracy.

* Development of novel CNNs based approach that directly utilizes polarization signatures to achieve land cover classification.
**Task 2:** Development of a novel approach using polarization signatures and CNNs for detection of human settlements affected during floods.

**Sub-tasks:**

* Development of a polarization signatures and CNN based model to extract built-up areas from PolSAR image.

* Development of a detection approach to detect villages from built-up areas using image morphology and *apriori* information.

* Mapping flood affected villages using image morphology.

**Task 3:** Development of novel spectral and spectral-spatial CNN models for multisensor (PolSAR and multispectral) images based land cover classification.

**Sub-tasks:**

* Study of conventional CNN models for multisensor (PolSAR and multispectral) image based land cover classification.

* Development of novel CNN models to achieve better generalization performance by efficiently utilizing spectral and spatial information from multisensor image.

**Task 4:** Development of novel CNN, RNN, and CNN-RNN models for multisensor satellite time series images based crop classification.

**Sub-tasks:**

* Study of conventional CNN and RNN models for multisensor satellite time-series images based crop classification.

* Development of novel CNN and RNN models with improved performance in multisensor satellite time-series images based crop classification by synergistically exploiting spectral, spatial, and temporal profiles of crops.

## 1.7. Framework of the Research

Although each task follows a separate methodology, all the four tasks can be consolidated into a generic framework which is shown in Figure 1.9. The generic framework encompasses majorly five components namely: problem or objective identification, experiment setup, data preprocessing, model development and training, and model deployment, results, and discussion. The order might differ for different tasks based on whether the solution is empirical or analytical.

### Problem or objective identification

In this part of the research, either a problem at hand is explored or an objective is laid out. In the former, the challenges with current applications are critically analysed first, then new technologies, materials, and methods are searched in order to find motivation to solve the problem. Task 1, 3, and 4 listed in section 1.6 are set out in this manner. Whereas in the latter,
first an objective or vision is set and then the goal is to attain the objective with best efficiency.

Task 2 is set out in this manner. In both the cases, knowledge of the background and extensive literature review is a prerequisite. Various challenges and objectives identified during the present research are listed in section 1.4 and 1.5 respectively.

## Experiment setup

Once either the problem is identified or an objective is set, the next step is to set up an experiment.
The experiment setup includes materials such as acquiring datasets, selecting study areas, and
collecting ground truth information and set of hypothetical methods which are suitable candidates
for providing a solution are investigated. Since the present research is focussed on development
of DNN based models for satellite image (especially PolSAR and multispectral) based
applications, a set of satellite images (PALSAR-2, Landsat-8, Sentinel-1, and Sentinel-2) are
considered for study. Study areas which are easily accessible and potential for subject of interest
are identified. The study areas considered are mostly in the Haridwar district of Uttarakhand,
India. These study areas considered have mixed land cover scenario and are interesting to
evaluate any novel techniques. Extensive ground truth is collected in the study areas over the
period of research. Hypothetical techniques considered in the present research are CNNs and
RNNs.

## Data preprocessing

The data considered in the present research are PolSAR and multispectral satellite image.
Preprocessing PolSAR data involves calibration, speckle filtering, and terrain correction whereas
preprocessing multispectral data involves atmospheric correction, cloud removal, and
resampling. Both the datasets are preprocessed before to be used for various applications. Data
preparation such as rescaling and stacking of PolSAR and multispectral images for multisensor
data analysis is also carried out under this component. Further, the various CNNs and RNNs
models take input data in particular formats, the pre-processed data is restructured accordingly
under this component.

## Model development and training

The general approach in this component of the research framework involves development of
novel approach, algorithm, or model based on hypothetical methods and nature of data. In the
present research, various CNN and RNN models are either investigated and newly developed for
various applications. As per CNNs, one, two, and three dimensional convolution based novel
models are proposed in task 1, 2, 3, and 4. Also, LSTM and GRU recurrent cell based RNN
models are developed as well for task 4. Further for task 4, synergistic exploitation of both the
CNNs and RNNs is investigated and novel CNN-RNN models are proposed. The developed
models are trained with ground truth information and under different scenarios.

Model deployment, results, and discussion

The final component of any research is the deployment of developed model and evaluation of
results obtained. Accuracy and generalization are key evaluation parameters considered in the
present research. Results are evaluated on multiple study areas. Generalization performance is
evaluated via visual inspection of the results especially in task 3, and 4.

<figure><img src="image_21.png" /><figcaption>Figure 1.9 Framework of the research</figcaption></figure>
## 1.8. Organization of the Thesis

The thesis consists of seven chapters. **Chapter 1** includes introduction, major challenges, motivation, objective, and tasks presented in the thesis.

**Chapter 2** provides brief literature review on the several state-of-the art DNN based techniques reported for various satellite image based applications. The review includes literature on DNN based techniques developed for single sensor, multisensor, and time series satellite images.

The study presented in **Chapter 3** involves a critical analysis of polarization signatures (PSs) as potential features for PolSAR data based land cover classification, and development of novel approaches which utilize polarization signatures to provide improved classification performance. The study proposed in this chapter is divided into three sub tasks i.e. *sub task 1*: critical analysis of PSs as potential PolSAR features, *sub task 2*: development of an optimal class boundary estimator algorithm to fully exploit PSs, and *sub task 3*: development of CNN model to exploit PSs directly in order to achieve improved land cover classification.

**Chapter 4** presents a novel approach to detect human settlements affected by a flood from PolSAR imagery using a novel PSs-CNN based classifier, and image morphology. The object detection algorithm consists of a CNN framework for image classification and a set of morphological operators and *apriori* knowledge for post-classification object detection. The study proposed in this chapter comprises of three stages: —first, design and development of PSs based CNN model for extraction of built-up areas from PolSAR image which contains the flooded region, —second, development of novel methodology for design of village detection filters based on *apriori* knowledge and image morphology, and —third, development of novel methodology for identification and mapping of flood affected villages using image morphology.

**Chapter 5** presents study and analysis of conventional and further development of novel multisensor satellite image classification approaches that are based on CNNs. First, the conventional CNN models are analysed and evaluated for multisensor satellite image based land cover classification. Second, two novel multisensor satellite image classification approaches that utilize CNNs in both spatial and spectral domains are proposed and developed. The frameworks employ permuted localized spectral convolutions along with localized spatial convolutions on high dimensional multisensor images. The study establishes the significance of performing permuted localized spectral convolutions in comparison to non-localized or non-permuted localized spectral convolutions. Two models, namely a permuted local spectral convolutional network (Perm-LS-CNN) and a permuted local spectral-spatial convolutional network (Perm-LSS-CNN) are obtained.
**Chapter 6** presents a study in which an attempt is made; to analyse the conventional CNN and RNN models, and to develop novel CNN and RNN (in standalone as well as synergistic modes) models for agriculture crop classification with multisensor/multimodal satellite time series imagery. For this purpose, two sub tasks are set in this study. In the first sub task i.e. *sub task 1*, conventional CNN and RNN models are analysed and evaluated for the task of multisensor satellite time series (MSTS) images based crop classification. 1D, 2D, and 3D CNNs, RNNs, and combinations of these are considered for analysis. Then, based on the critical analysis of the working and performance of these conventional models over (MSTS) images based crop classification, 6 novel models are proposed and developed in *sub task 2*.

Finally, **chapter 7** concludes the different studies reported in the thesis. The conclusions drawn from each study, contributions made by the studies, and future scopes of the studies are also listed in this chapter.
# BRIEF LITERATURE REVIEW: RECENT TRENDS AND CHALLENGES IN DEEP LEARNING AND DNNs BASED SATELLITE IMAGE PROCESSING

Deep learning and deep neural networks have shown substantial potential as successful satellite image processing techniques. It has been established that DNNs and deep learning (DL) can be extended and generalized to satellite images from natural images [4]. In the recent years, many satellite image based applications such as land cover classification, crop monitoring, building extraction, etc. have been proposed and developed with the help of DNNs and their popularity is still growing [69], [70].

The deep neural network models are high performance, scalable, automatable, and can learn directly from raw data [35]. The deep learning models improve their performance with increase in data whereas other machine learning techniques, which depend on hand-crafted features for learning, are unable to learn beyond a certain amount of data. This means, more the training data, better the deep learning model performs. Another advantage of deep learning is its ability to learn directly from data and independency from hand-crafted features. Traditional machine learning techniques require hand-crafted features for learning. These attribute of deep learning models placed them in prominent position to be used for satellite image processing. This chapter presents a brief review of algorithms developed, challenges faced, and applications proposed in the field of deep learning and DNNs based satellite image processing.

The review involves the contribution and significance of deep learning and DNNs in uni- or multi-modal satellite image processing, single sensor or multi sensor image processing, different satellite image applications, single time stamp or multi-temporal satellite images, and any combination of the above. The review is divided into three categories; *-first*: brief review of recent developments in DNNs based unimodal satellite image processing analysis, and applications, *-second*: brief review of recent developments in DNNs based multisensor satellite image processing, analysis, and applications, and *-third*: brief review of recent developments in DNNs based models for satellite time series image processing, analysis, and applications. Since the studies on DNN based satellite image processing cover loads of fields, the current review is however focussed but not restricted to classification (e.g. land cover classification, crop classification).
## 2.1. Brief Review of Recent Developments in DNNs based Unimodal Satellite Image Processing

This section presents a brief review of the developments in DNNs based unimodal satellite image processing. Unimodal here refers to a single modality of satellite images i.e. either optical (surface reflectance), PolSAR (backscatter), thermal (brightness temperature) etc. This means that the studies reviewed in this section are developed or evaluated on a single modality of satellite images, may it be optical or PolSAR or any other. Studies on multimodal or multisensor satellite image processing with DNNs are not reviewed in this section. Studies on multimodal or multisensor satellite image processing with DNNs are reviewed in section 2.2 Also, this section does not include studies involving time series satellite images. Such studies are discussed later in section 2.3.

Among other applications, land cover classification is one of the most popular application in the geoscience and satellite image processing community. Numerous techniques have been successfully developed for land cover classification with unimodal satellite images over the years [44]. In context, DNN models have also been successful in satellite image based land cover classification applications.

Satellite SAR images are recently one of the most popular and most widely used modality among all modalities of satellite images. As first attempts in SAR, deep learning based methods have been adopted for a variety of tasks, including land cover or land cover classification [75], [76], object detection [77], parameter inversion [78], despeckling [79], and specific application in InSAR [80].

Land cover classification using PolSAR images is rapidly advancing with the help of deep learning. For land cover classification from SAR and Polarimetric SAR (PolSAR) images, effective feature extraction is essential. Deep learning feature extraction has proved to overcome this challenge, to some degree [75]. Regarding feature extraction, most conventional methods rely on exploring physical scattering properties and texture information in PolSAR images. Compared to conventional methods, deep learning is advantageous in land cover classification due to its capability of automatically learning discriminative features. Moreover, deep learning approaches such as CNNs can effectively extract not only polarimetric characteristics but also spatial patterns of PolSAR images [75]. Deep learning is applied to land cover classification using PolSAR images at first in [81]. They employed a stacked auto encoder (SAE) to automatically learn deep features from PolSAR data and then fed them to a softmax classifier. Remarkable improvements in both classification accuracy and visual effect proved that this method can effectively learn a comprehensive feature representation for classification purposes.
Geng et al. in [82] proposed a deep convolutional autoencoder (DCAE) for automatically extracting features and performing classification. The first layer of DCAE are a grey level co-occurrence matrices and Gabor filters. The second layer of the DCAE integrates correlated neighbour pixels to reduce speckle. Following these two hand-crafted layers, a trained SAE, which is similar to [81], is attached for learning more abstract features. The method achieved remarkable classification accuracy when tested on high-resolution single-polarization TerraSAR-X images. Similarly, Qin et al. in [83] applied adaptive boosting of RBMs to PolSAR image classification, stacked sparse AE to PolSAR image classification by taking into account local spatial information is developed in [84], Zhao et al. in [85] proposed a discriminant deep belief networks (DBNs) for land cover classification.

Moreover, taking into account that most current deep learning methods aim at exploiting features either from polarization information or spatial information of PolSAR images, Gao et al. in [86] proposed a dual-branch CNN to learn features from both perspectives for land cover classification.

Different variations of CNNs have been used for land cover classification with satellite PolSAR images. A 6-channel covariance matrix is first extracted in [87] and then, the 6-channel covariance matrix is fed into a trainable CNN model to achieve land cover classification. In [88], the authors focused on the computational efficiency of deep learning methods, proposing the use of lightweight three dimensional CNNs. They showed that classification accuracy comparable to other CNN methods was achievable while significantly reducing the number of learned parameters and therefore gaining computational efficiency. Wang et al. in [89] proposed a fully convolutional network integrated with sparse and low-rank subspace representations for classification of satellite PolSAR images. The knowledge of target scattering mechanism interpretation and polarimetric feature extraction is used to improve land cover classification performance in [90]. He et al. in [91] proposed the combination of features learned from nonlinear manifold embedding and applying a fully convolutional network on input PolSAR images; the final classification was carried out in an ensemble approach by SVM.

The techniques mentioned in the last paragraph are focussed to develop DNN models for either preprocessing SAR images or obtaining desirable features that can be learned with DNNs. Direct utilization of raw SAR or PolSAR data with DNNs could lead to simpler models which can extract abstract features directly from raw PolSAR data based on input-output relationship. In this direction, Zhang et al. in [92] proposed a novel complex-valued CNN specifically designed to process complex values in PolSAR data with CNNs. Other notable complex-valued deep learning approaches for classification using PolSAR images can be found in [93]–[95].
Satellite hyperspectral sensors capture images in hundreds of contiguous narrow spectral bands, producing three-dimensional hyperspectral imagery (HSI). The HSI images provide spectral and spatial information. With the launch of EnMAP, scheduled in 2020, and DESIS, which was planned originally for 2017, high-quality satellite hyperspectral data will be available. Such wealthy spectral information data (with appropriate spatial context) can potentially reveal interesting unknown content within the hyperspectral images. In particular, hyperspectral images have been used widely in a range of practical applications, such as land cover classification, change detection and agriculture monitoring. Indeed, applications of hyperspectral image data have been one of the most active research interests.

Deep learning methods have recently been introduced into hyperspectral image classification, achieving impressive results [96]–[99]. In hyperspectral images, spectral resolution is relatively more important as compared to spatial information. In deep learning (DL) based spectral feature classification methods, DNN models take raw spectral vectors as inputs and output deep spectral features, which are used then for classification. Hu et al. in [100] attempts to carry out HSI classification using 1D-CNN that contains four layers: one convolutional layer, one pooling layer and two fully connected layers. The study in [101] exploits a similar 1D-CNN to classify hyperspectral images but relies on using additional techniques such as batch normalization and dropout. Among the three typical DL models (deep belief networks, CNNs, and stacked autoencoders), stacked autoencoders and deep belief networks take the data represented in a vector form as input, thereby satisfying the need to extract deep spectral feature from spectral vectors. In fact, stacked autoencoders and deep belief networks were introduced prior to CNNs to perform HSI spectral feature classification.

Spectral feature classification methods just take account of spectral information and fail to make use of spatial information. Spatial information (also termed contextual information) helps reveal useful relationships between adjacent pixels. Making full use of spatial information such as shape or size can significantly improve classification accuracy and efficiency. The most significant advantage of 2D-CNNs is that they are suitable to effectively learn features from raw 2D images directly. However, as hyperspectral image data contains more than 100 spectral bands, direct utilization of a 2D-CNNs to extract deep features from raw hyperspectral image data require a mass of learnable kernels. The learnable kernels are hard to train and will increase computational overheads. A randomized principal components analysis (PCA) is employed in [102] to condense the spectral depth of hyperspectral image data in order to improve computation cost. They then applied a 2D-CNN to extract deep features from the reduced spectral space before classification. A similar study is proposed in [103] with the difference in consideration of principal components and window size. They used only 3 principal components and a window
of 42×42 for 2D convolution. Similar studies utilizing feature reduction prior to DL based feature extraction or classification are reported in [104]–[107].

Three-dimensional hyperspectral image simultaneously involves rich spectral information and spatial information. Neither spatial feature classification or spectral feature classification frameworks takes full advantage of this intertwined character of hyperspectral imagery. Deep learning based spectral-spatial classification strategies are more suitable for hyperspectral images and can lead to the improvement of classification accuracy. There are two main strategies to combine the spectral and spatial information present in hyperspectral images; one is to extract spectral and spatial features separately first, and then to combine these features; the other is to extract deep spectral-spatial features from hyperspectral image directly. The former strategy is employed in [103], [106], [108] whereas the later strategy is employed in [96] and [98].

In case of multispectral satellite images or satellite images with low spectral depth, per pixel classification does not yield significant results with deep learning techniques. However, such images, when used in patch mode yield new applications and results. One application of low spectral satellite images with deep learning is scene or patch classification. Scene classification aims to automatically assign a semantic label to each scene image [109]–[111]. Here, a scene image usually refers to a local image patch manually extracted from large-scale high-resolution aerial or satellite images that contain explicit semantic classes (e.g., residential area, commercial area, etc.). Due to high resolutions in such data, different scene images may contain the same types of objects or share similar spatial arrangements between objects. For example, residential and commercial areas may have buildings, roads, and trees, but they belong to two different build-up areas. Indeed, significant variations potentially existing in scene images in the spatial arrangements and structural patterns make scene classification a considerably challenging task [58]. Particular to land cover classification, a patch-based CNN which was more suitable than the pixel-based CNN for medium images is proposed in [112].

In summary, enough evidence suggests the significance of using DL and DNNs for unimodal satellite image processing. Most DL based studies focussed to either PolSAR, hyperspectral, and multispectral satellite images. DNN models have been developed and utilized with PolSAR images for land cover classification, sea ice monitoring, speckle filtering, and change detection applications. On the other hand, spectral and spectral-spatial characteristics based hyperspectral image classification seems to be popular recently. Initially, feature reduction is opted before the DNN model based classification process but later on, hyperspectral images are used directly for classification. 1D-, 2D-, and 3D-CNNs are popularly used with hyperspectral images in various applications including land cover classification. In case of
multispectral satellite images, spatial and temporal characteristics are more focussed and are exploited with the help of 2D-CNNs (spatial), and 3D-CNNs and RNNs (spatial, temporal). Multispectral images are mostly used in scene or patch classification.

However, certain aspects still need attention or are worth analysing. One aspect is, direct exploitation of PolSAR satellite images via DNN models for applications such as land cover classification. In particular, the use of 2D-CNNs to directly extract complex spatial feature from PolSAR data is motivating. More research gaps in DNN based unimodal satellite image processing is summarized in section 1.5.

## 2.2. Brief Review of Recent Developments in DNNs based Multisensor Satellite Image Processing

DNNs are not restricted to handling unimodal datasets. Instead, exploiting multisensor or multimodal data is one of the advantages of DNNs [60]. It is important to note here that various satellite image applications require to focus on one, or more characteristics (spatial, spectral or temporal) of satellite images. Usually, a single sensor or unimodal satellite images cannot provide high-quality information in all three aspects. Therefore, the use of multisensor or multimodal satellite images for many applications is not uncommon.

This section presents a brief review of the developments in DNNs based multisensor/multimodal satellite image processing and analysis. Multisensor here refers to a set of at least two modalities of satellite images. Multimodal satellite image processing falls under the umbrella of multisensor satellite image processing with image sources from two different modalities. The most popular multi-modal satellite image set is a stack of optical-PolSAR satellite images. Studies on multisensor satellite image processing with DNNs are reviewed in this section with a focus on land cover classification application.

Land cover classification with multisensor satellite images is achieved via two modes. In the first mode, the images from multiple sources are fused together via a fusion process (data fusion), and then the fused image is used as input during the classification process. In the other mode, images from multiple sources are directly used in a stacked manner during the classification process. The development in DNNs based methodologies in both these modes is briefly reviewed here.

Data fusion is one of the fast-moving areas of satellite image processing. Due to recent increases in multisensor data availability, using heterogeneous and big data for earth observation has become more tangible. In this regard, DNNs are natural candidates to tackle the challenges
of modern satellite image data fusion. Pansharpening, feature level, and decision level information fusion are the three major strategies of data fusion.

Pansharpening is the task of improving the spatial resolution of multispectral images by fusing these with high spatial resolution image such as panchromatic image. Traditionally, the field was dominated by works fusing multispectral data with panchromatic bands, but more recently it has been extended to hyperspectral images in [113] and thermal images in [114]. A first attempt in DNN based pansharpening is reported in [115] where the authors used a shallow network to upsample the intensity component obtained after the intensity, hue, and saturation transformation of the red, green, and blue images. Once the multispectral bands have been upsampled with the CNN, a traditional Gram-Schmidt transform is used to perform the pansharpening. The authors simply performed the interpolation operation with a CNN instead of using a nearest neighbour or bicubic interpolation function. Around the same time, authors in [116] used a three hidden layered CNN framework to learn the pansharpening operation end to end. In short, the CNN performs the complete pansharpening process. In this model, authors stacked the panchromatic band with the upsampled spectral bands and then, for each patch, the high resolution values of the central pixel are learned. This method eliminated the element of external intervention in the process and the requirement of alternative methods in the pansharpening process. They used the architecture on GeoEye-1 (panchromatic-multispectral pair), IKONOS-HRV (panchromatic-multispectral pair), and WorldView-2 and achieved satisfactory results. Moreover, the authors in [117] fine-tuned a superresolution CNN model pretrained on natural images for a set of hyperspectral images. They attempted transfer learning [118] between the spectral domains of natural images (three bands, large bandwidths) and the spectral domains of hyperspectral images (many bands, narrow bandwidths).

Most of the current literature dealing with DNNs based satellite image processing, addresses the problem of image classification (e.g. land cover classification). In the case of multisensor or multimodal images, land cover classification is performed by fusing information from multiple sources at either feature level or at decision level. A very simple way of using multisensor images in DNN models for land cover classification or any other application for that matter is to stack or concatenate the multisensor images and process them further as one. The filters learned by the first layer of the DNN, in this case therefore, depends on the order of stack of different sources. Such models have been used in satellite image patch classification. Fully convolutional and de-convolutional models are good examples of such models and have been used in multisensor satellite image classification (patch level) [119]. In contrast to stacking the input images directly, intermediate features (using a feature extraction process) have also been stacked and been used for classification [120]. The models proposed in [120] has been used in
the classification of SAR images via learning through a stack of spatial features. Similarly, in [121], the authors present a hierarchical feature learning process that learns combinations of spatial filters extracted from hyperspectral image bands and digital surface model (DSM) image.

Another series of studies which include an explicit decision level fusion layer between land cover class label images. The studies either perform the decision fusion post classification or the decision fusion process is learned directly. If the classification is performed with two different classifiers one of which is a DNN based model, the different class labels are fused posterior to classification. Two studies are particularly notable in this regard. The authors in [122] fused classification maps obtained from CNNs with classification maps obtained from random forest (RF). The RF classifier is trained on handcrafted features. The two maps are fused by multiplication of the posterior probabilities. An edge-sensitive conditional random field is also learned on top to improve the quality of the final labelling. Whereas the authors in [123] learned an ensemble of CNNs and then averaged their predictions. Their proposed processing framework has two main streams: -first, processing the colour infrared data from the Vaihingen dataset, and -second, processing the digital surface model. They trained an ensemble of CNNs and further used them as inputs for each layer of the primary model's activation maps. They obtained a series of land cover maps to nourish the ensemble. This strategy improved the performances by considering classifiers issued from different data sources and levels of abstraction. This model has the advantage of being entirely learned in an end-to-end fashion. However, it incurs a heavy computational load and involves a complex architecture with numerous fusion layers and skip connections. Data fusion is not only about fusing image data with the same viewpoint. Multimodal satellite image data that exceed these restrictive boundaries and approaches to tackle new, exciting problems with satellite image processing are beginning to appear in the literature.

In summary, DNNs based multisensor satellite image processing, analysis, and applications have significant potential and future. Notably, multisensor satellite image based land cover classification with DNNs provides higher accuracy, less fine-tuning, high dimensional data operability, and scalability. A model developed for a particular multisensor satellite image set can be used with other multisensor satellite image sets with slight modifications. Although, data fusion approach for multisensor satellite image applications seems to be popular right now but direct utilization of multisensor satellite images with DNNs is gaining attention. Few research gaps identified in multisensor satellite image processing with DNNs are discussed briefly in section 1.5.
## 2.3. Brief Review of Recent Developments in DNNs based Satellite Time Series Image Processing

This section presents a brief review of the developments in DNNs based satellite time series image processing techniques. Satellite time series images refer to a set of satellite images collected over time observing the same area of interest with the same sensor. If more than one sensor is considered, then the sensors may belong to same modality (unimodal) or different ones (multimodal). The review includes studies on both, unimodal and multisensor satellite time series images. However, in this section, the review is presented in the order of DNN technologies used in satellite time series image processing. The popular techniques include CNNs, RNNs and combined CNNs-RNNs.

CNNs are the most popular DNN technology in satellite time series image processing. In the recent years, numerous techniques have been developed for satellite time series image processing using 1D, 2D, and 3D CNNs. 1D CNNs are first utilized for Landsat-8 satellite time series image processing in [124]. They used 1D-CNNs with multilayer perceptron inspired from [125] to perform land cover classification. Zhong et al. in [126] developed two types of deep learning models; first, a 1D CNN based satellite time series crop classification model with inception blocks within the classification framework and second, a long short term memory (LSTM) RNN based crop classification model. The 1D-CNN with inception blocks model performed well in contrast to basic LSTM-RNN model for satellite time series image based crop classification. Pelletier et al. in [73] proposed a temporal CNN model to perform satellite time series image based crop classification. The model focuses the significance of temporal convolution in time series image classification. They also evaluated the effects of appending hand crafted features such as vegetation indices into the classification model. Their model performed satisfactorily over random forests and RNNs. Further, 3D-CNNs have also been utilized for satellite time series image processing in many studies. 3D CNNs have the ability to capture spectral, spatial, and temporal features simultaneously whereas 1D and 2D CNNs lack such ability. A satellite multi-temporal images based crop classification model is proposed and developed in [127] which uses 3D-CNNs as spatiotemporal feature extractors. It is validated from the study that 3D-CNNs are better spatiotemporal feature extractors than 2D or 1D CNNs for satellite time series images. A 3D CNN-transformer hybrid approach for crop classification using multitemporal Landsat-8-Sentinel-2 images is developed in [128]. The transformer architecture utilizes multi-head self-attention module to represent the sequence pattern which improves the classification performance in contrast to conventional RNN based crop classification modules.
RNNs are also popular DNN architectures to utilized with satellite time series images. Along with CNNs, RNNs have also gained interest in satellite time series image applications. The first study which utilized RNNs with satellite time series images is proposed in 2016 [129]. They developed a transferable change rule algorithm based on LSTM-RNNs for detecting change in land cover. Their comparison with SVM-CNN and decision tree signified the better performance of the proposed model. Later, a LSTM-RNN based model is developed in [74] for crop classification. The model is evaluated on Sentinel-2 multispectral time series images and performed well in comparison to CNNs and SVMs. A similar model is proposed in [130] for pixel as well as object based land cover classification. This model is evaluated over two different multispectral datasets and results indicated improved and stable performance. Moreover, RNNs have also been utilized for PolSAR time series image based crop classification in [131] and [132]. Both, LSTMs and gradient recurrent unit (GRU) based RNN models have been developed and tested with Sentinel-1 time series images. Both studies indicate to a slightly better performance of GRU over LSTMs. Sun et al. in [133] used ancillary time series cropland data layer along with Landsat-8 time series images and further used LSTM-RNN model over the combined dataset to perform land cover classification. Another study which uses a patch or window of pixel instead of per-pixel or object based in classification process is proposed in [134]. The study is developed on Landsat-8 time series images and LSTM-RNN architecture is used. The patch based land cover classification model benefits from the contextual information present in the patch which otherwise is absent in pixel based classifications.

CNNs and RNNs have also been used synergistically and successfully in satellite time
series image processing. A 2D-CNN-RNN architecture is developed in [135] to perform data
fusion with multimodal, multiscale, and multitemporal satellite images. The satellite time series
images are Sentinel-2 whereas a very high spatial resolution SPOT-6/7 image is also used. In
another study, a temporal attention network is developed for multisensor and multitemporal
satellite time series image based crop classification. Attention mechanism is a brain mechanism
of human signal processing. The study utilizes attention network in the temporal dimension to
reinforce the phenological differences in crops and restraining their phonological similarity.
Further, in [136], CNNs are used with RNNs in a parallel architecture to extract spectral-spatial-
temporal features from a combination of MODIS time series and NAIP high spatial resolution
dataset. Finally, a SVM and multilayer perceptron models are used to perform crop classification.
CNN-RNN models have also been used in change detection applications. In [137], authors used
a recurrent convolutional cascade model for learning spectral-spatial and temporal features from
Landsat -7 time series multispectral images. The learned features are then used to detect changes
with a binary classification module.
In summary, three major DNN model categories i.e. CNNs, RNNs, and CNNs-RNNs have been successful in analysing and processing satellite time series images for various applications. The applications include land cover classification, change detection and most importantly, agriculture crop classification. Although CNNs and RNNs have performed well with satellite time series images in standalone modes, synergistic utilization of both models either in parallel or in cascade fashion, results in superior performance. The use of CNN-RNN architecture in feature extraction and classification with multisensor satellite time series images for crop classification is very motivating. Research gaps in DNN based multisensor satellite time series image processing are discussed briefly in section 1.5.

## 2.4. Conclusions

This chapter presented a brief review on developments in DNN models and methodologies for satellite image processing, analysis, and applications. The review included unimodal satellite images, multisensor satellite images, and satellite time series images. In context to applications of satellite images, focus is kept on classification may it be land cover classification or crop classification. However, other applications are also touched upon. The review consolidates to certain observations

* Most of the novel DNN models and methodologies have been explored for SAR, hyperspectral, and multispectral satellite images.

* Spatial or contextual information present in satellite images is given preference in contrast to spectral information (in context to multisensor satellite images).

* Both parallel and cascade DNNs architectures have been developed. Parallel architectures extract spatial and spectral information simultaneously with two different DNN modules whereas the cascade architectures used the same module to extract the spatial and spectral information (one after the other). Critical analysis of these architectures is worth investigating and interesting.

* CNNs are more popular than other DNN models while handling spectral or spatial features of satellite images are whereas RNNs and 1D-CNNs are more popular while handling temporal features of satellite images. However, integration of CNN and RNN models for time series satellite image processing might yield high performance and consistency and also could lead to development of a general framework.

* The main focus of most studies is on classification tasks for e.g. land cover classification, scene classification or, object detection. However, several other applications are worth mentioning including fusion, segmentation, change detection, and registration.

Prominent research gaps are identified and are briefly listed in chapter 1 (refer section 1.5).
# CNNs AND POLARIZATION SIGNATURES BASED NOVEL APPROACH FOR LAND COVER CLASSIFICATION WITH POLSAR SATELLITE IMAGES

Use of CNNs with PolSAR satellite images for various earth observation applications is increasing at pace. One application where the use of CNNs has increased rapidly is land cover classification. However, the focus of new CNN based models for PolSAR image based land cover classification is to achieve higher classification performance in mixed land cover class scenarios. This chapter proposes a novel approach based on CNNs which exploits PolSAR images to improve classification performance in mixed class scenarios. This chapter highlights the challenges while extracting information from PolSAR images for land cover classification. One major challenge while extracting land cover information from PolSAR satellite images is to segregate or classify land covers efficiently in a mixed land cover scenario. Therefore, this chapter presents two different novel approaches (one of which is CNN based) to extract information from PolSAR images for improved land cover classification in a mixed land cover scenario.

## 3.1. Introduction

Use of polarimetry for radar applications has a long history. PolSAR target decomposition based techniques are used by various researchers for many earth observation applications such as land cover classification [39], [138]–[140]. A typical land cover scene includes both, natural (e.g. forests, water bodies) and man-made (e.g. buildings, bridges, roads) targets. These land covers show complex scattering response and poses a challenge in classification using satellite PolSAR data. Target decomposition based techniques aim at analysing the received scattering matrix from polarimetric data to extract information about the scattering processes exhibited by various land covers. These techniques compute particular features (e.g. decomposition features) from the backscattering response of land covers (recorded by the PolSAR sensor) and further use these features to detect, identify or, classify land covers. An extensive study on the performance of various decomposition based features in land cover classification is done in [141]. They observed an absence of dominant decomposition based features for different land covers in mixed land cover scenario. Mixed class scenarios refer to landscape (study regions) where land cover classes considered for classification are small, unorganized, and are scattered randomly. Due to this
landscape two or more different land covers appear similar in a satellite image. For example, built-up areas having building structures close to each other and are randomly oriented (with respect to SAR sensor flight path), might show volume scattering along with double bounce scattering thus behaving as mixed class. Researchers are also developing new multi-component decomposition and factorization based features that are able to segregate land covers more efficiently but the challenge still needs more attention [42], [142]–[144] Impact of various target decomposition base features on land cover/use classification performance is analysed in [145]. They merged land cover/use classes that are not separable with given dataset and features. For example, agriculture and bare soil are combined and treated as one class. Similarly, rural settlements and dense tall vegetation are combined and treated as one class. This indicates to the challenge to segregation various land covers in a mixed land cover scenario. An example of mixed land cover scenario is shown in Figure 3.1. For example, in Figure 3.1(b), marker A indicates rural settlements (built-up area) and marker B indicates dense tall vegetation patches. Observing the enlarged segments in Figure 3.1(c), it is evident that both the land cover classes are showing similar backscattering behaviour (high backscattering response in HV channel) and [145] found similar observation. Literature indicates few studies that aim to differentiate class pairs such as bare soli-agriculture or dense tall vegetation- rural settlements. Attempts are made in [42], [146], [147] and more recently in [42], [148] to segregate oriented built-up areas from tall vegetation using different decomposition and factorization features. However, the challenge still exists and new features needs to be explored which might have potential to segregate these class pairs and provide good land cover classification results.

In the context, PolSAR polarization signatures can be potential candidates. PolSAR polarization signatures (PSs) can characterize the scattering behaviour of target under study or in this case land cover under observation effectively and therefore can be considered for land cover classification applications. These signatures are 2D representations of scattering characteristics of a target. A method for generation of PSs with different polarization bases is proposed in [149]. They reported better classification accuracy than object based supervised Wishart classification method. Similar work proposed in [150] compared observed PSs to standard target or reference PSs based on norm and further, a pattern recognition using Kohonen networks was performed for LC classification in [150]. Results show that PSs not only allow to identify coherent scatters but also scattering mechanisms. Till date, few approaches have been developed for analysing PSs for land cover (LC) classification purposes [151]–[153]. Therefore, PSs are considered here to be analysed as potential PolSAR features for land cover classification in mixed land cover
scenario. In summary, this chapter presents a study in which PolSAR PSs are considered as potential PolSAR features for land cover classification.

However, the challenge with exploiting PSs for land cover analysis is its contextual nature. It means that PSs pixels are related with its neighbourhood. But, the current popular techniques that directly exploit PSs such as the normalized signature correlation mapper (NSCM) and normalized Euclidean distance (NED) based classifiers requires the transformation of the two dimensional PSs into a one dimensional vector before classification. This results in a loss of contextual information present in the PSs. There is a need to develop classifiers which can exploit the contextual information present in the PSs. In the direction of novel classifiers, authors in [146] and [148] utilized deep learning techniques for land cover (LC) classification with the help of synthetic databases for learning. Their attempt is motivating and points to the potential of deep learning techniques in extraction of information from PolSAR images. These studies motivate to explore the use of deep learning and DNNs with PolSAR data for various applications. The study presented in this chapter to propose a CNN based approach for PolSAR PSs based land cover classification.

<figure><img src="image_22.png" /><figcaption>Figure 3.1 (a) False colour composite generated from PALSAR-2 data. Colour scheme: red = HH, green = HV, and blue = VV. (b) Panchromatic image collected from Google Earth dated Feb 19, 2015. Symbol ‘A’ marks built-up area in both the images and symbol ‘B’ marks the dense vegetation area in both the images. (c) The enlarged segments are shown in middle in the outlined boxes. The built-up area is showing volume scattering (high backscatter response in HV, refer) along with double bounce scattering.</figcaption></figure>
The study presented in this chapter is divided into three sub tasks i.e. *sub task 1*: critical analysis of PSs as potential PolSAR features, *sub task 2*: development of an optimal class boundary estimator algorithm to fully exploit PSs, and *sub task 3*: development of CNN model to directly exploit PSs.

The first sub-task of the study covers the introduction to polarization signatures and further, the computation of polarization signatures based features from fully polarimetric SAR data. A land cover (e.g. bare soil) behaves as a complex mixture of different standard scatterers or standard targets and therefore its PS is difficult to interpret. Therefore, at first, there is a need to find correlations between polarimetric/ PSs of land covers and PSs of standard targets. These correlations indicate the degree of similarity between the two scatterers. Further, the PS correlation features or PSCFs needs to be critically analysed as potential features for PolSAR image based LC classification. Section 3.4 covers the details on the computation of PSCFs and their critical analysis as significant PolSAR features for LC classification.

In the second sub task of the study, a novel adaptive and optimal class boundary estimator is developed. Image statistics are used to obtain class boundary values which make the approach robust and adaptive. Genetic algorithms are used to optimize the class boundary values to achieve maximum overall classification accuracy. A set of optimal class boundaries are obtained using this estimator, and a decision tree (DT) classifier is realized using these class boundaries. Section 3.5 presents the development and implementation of the adaptive and optimal DT classification technique.

At last, in the third sub task of the study, a second novel classification approach namely PSs-CNN which utilizes CNNs as classifier directly with polarization signatures to perform land cover classification is developed. Since, the PSs are two dimensional representation of target backscatter response, they can be treated as images of target response and thus can intuitively be used in a 2D-CNN classification framework. The 2D-CNNs are able to capture the spatial context present in a pixel PSs which helps in the classification of the pixel. Section 3.6 presents the PSs and CNNs based novel classification technique.

Remaining of the chapter is outlined as follows. Section 3.2 covers the theoretical background required to understand the study. Section 3.3 provides description about the experiment setup which includes; the study areas and datasets considered in the study, and the ancillary information collected for algorithm development and training. Section 3.7 concludes the study.
## 3.2. Theoretical Background

### 3.2.1. Polarization signatures

Polarization signature (PS) is a two dimensional representation of backscattering behaviour of a target (in this case, a land cover). In PS representation, x-axis and y-axis represent ellipticity angle and orientation angle respectively and z-axis represent received backscattered power coefficient (σ°). Orientation or tilt angle (ψ) varies from -90 to 90° and ellipticity angle (χ) varies from -45° to 45°. The signatures are computed using equation (3.1) [154], [155].

$$ \sigma(\chi_i \psi_i \chi_j \psi_j) = \frac{4\pi}{k^2} \begin{pmatrix} 1 \\ \cos 2\chi_i \cos 2\psi_i \\ \cos 2\chi_i \sin 2\psi_i \\ \sin 2\chi_i \end{pmatrix} (K) \begin{pmatrix} 1 \\ \cos 2\chi_j \cos 2\psi_j \\ \cos 2\chi_j \sin 2\psi_j \\ \sin 2\chi_j \end{pmatrix} \quad (3.1) $$

Where σ represents backscattering coefficient, suffix i and j represent transmit and received combinations respectively. K is the Kennaugh matrix, which provides received power, given the polarization characteristics of the receiving antenna [155]. It is also known as the Stokes scattering matrix. Computation of K matrix parameters is presented in Appendix-A. k is the propagation constant. Co-polarized signature is obtained by transmit and received combination of $\psi_i = \psi_j$ and $\chi_i = \chi_j$, and cross polarized signature of $\psi_i = 90° + \psi_j$, and $\chi_i = -\chi_j$. Ellipticity angle defines the polarization behaviour (linear, circular or elliptical polarization), while orientation or tilt angle in combination with ellipticity angle, defines polarization states, for example, horizontal or vertical polarization [156]. For the sake of visualization, co-polarized PS of an arbitrary land cover is shown in Figure 3.2.

<figure><img src="image_23.png" /><figcaption>Figure 3.2 Co-polarized polarization signature of an arbitrary land cover.</figcaption></figure>
### 3.2.2. Convolutional neural networks

CNNs are a popular class of deep feedforward artificial neural networks proposed in various studies [157]–[160]. Building blocks of a conventional CNN are: a convolutional layer, a pooling layer, a dropout layer, a fully connected layer, and a loss layer. A generic arrangement of the layers is discussed in [161] and [162].

Imagine a CNN with $M$ layers, the output state vector of the $m^{th}$ layer is denoted by $\mathbf{O}^m$, where $m \in \{1, 2, \dots, M\}$ and, $\mathbf{O}^0$ represents the input data. The input layer is then connected to a convolutional layer. In the convolutional layer, a 1D, 2D or 3D convolution is performed with kernels. For the sake of understanding, a 2D convolution is explained here. The convolution plus pooling in the $m^{th}$ layer is represented in equation (3.2).

$$ \mathbf{O}^m = \text{pool}(g(\mathbf{O}^{m-1} \otimes \mathbf{W}_m + \mathbf{b}_m)) \quad (3.2) $$

Where, $\otimes$ indicates convolution operation and $\text{pool}$ denotes a spatial aggregation, as defined in [163]. The matrix $\mathbf{W}_m$, encodes the weights of the links connecting the $(m-1)^{th}$ convolutional layer to the $m^{th}$ layer. The vector $\mathbf{b}_m$ denotes a bias term. Subsequently, a non-linear pointwise activation operation $g(\cdot)$ is performed. Finally, a pooling layer is used which helps in selection of dominant features.

Several convolutional and pooling layers can be stacked to form a hierarchical feature extraction architecture. The resultant features are then combined to form a one-dimensional feature vector to be taken as input by the fully connected layer. The fully connected layer acts as a typical multi perceptron layer. The last layer is usually a linear (for regression scenarios) or softmax (for classification scenarios) layer, with the number of neurons matching the number of output variables or classes desired. This layer also specifies how training penalizes the deviation between the predicted and true labels and is normally the final layer.

The weights, $\{\mathbf{W}_1, \mathbf{W}_2, \dots, \mathbf{W}_M\}$ and the biases, $\{\mathbf{b}_1, \mathbf{b}_2, \dots, \mathbf{b}_M\}$ of the CNN compose the model parameters, which are iteratively and jointly optimized via overall accuracy maximization over the training set. Similar networks can be realized for 1D or 3D CNNs accordingly.

The whole network still expresses a single differentiable score function from the raw image pixels on one end to class scores at the other. 2D CNN architectures make the explicit assumption that the inputs are images. This allows the user to encode certain properties into the architecture. These make the following functions more efficient to implement and it also vastly reduce the amount of learnable parameters in the network. This property is explored and utilized
for PSs in this study. CNNs with deep learning have found their application in PolSAR data as well [58].

## 3.3. Experiment Setup

### 3.3.1. Study area

The area selected for study includes Roorkee city at the foothills of the great Himalayas of the northern India and its neighbouring region. The area extends from 29°57′4″N, 77°51′54″E to 29°48′10″N, 78°4′5″E. The area covers about 400 km² of landscape. The study area includes rivers and water canals, built-up areas include Roorkee city and villages nearby. Built-up areas are both sparse and dense. Structures in built-up areas are in patterns or at random. Area also includes sparse and dense tree plantations, cultivated and fallow agricultural lands, dry and water logged patches of land. Figure 3.1(b) depicts the Google Earth imagery of the study area. This topography makes the study area a mixed land cover scenario in terms of land cover classification.

### 3.3.2. Dataset

The dataset used for study is the ALOS PALSAR-2 L band synthetic aperture radar (SAR) data. PALSAR-2 is a polarimetric SAR sensor on-board the ALOS-2 spaceborne satellite. SAR data collected by PALSAR-2 is fully polarimetric with information in four polarization states i.e. HH, HV, VH, and VV. PALSAR-2 sensor and product specifications are listed in Table 1.8. The data acquisition date is March 13, 2015. The data is provided by JAXA, Japan for research purposes. A subset of the whole image, nearly 400 km² is considered for algorithm development. ALOS PALSAR-2 data has been reported to be utilized successfully for many land cover applications such as geology mapping [164], land deformation monitoring [165], and mostly in land cover classification [166], [167].

### 3.3.3. Ground truth

Ground truth sample points are collected for training and validation. Five land cover classes are considered namely bare soil (BS), tall vegetation (TV), short vegetation (SV), built-up (BU), and water. Ground truth class sample points are measured directly on the terrain based on visual inspection. Approximately 1200 data points are collected for both the study areas. Details of the ground truth collected for both study areas are shown in Table 3.1. Also, Figure 3.3 depicts samples of ground truth collected during the study.