Miracle-dz/UniFAR-FTUs huggingface dataset:
Remote Sensing Image Jitter Restoration Based on Deep Generative Adversarial Network

High stability of the observation satellite platform is increasingly demanded in recent years. However, attitude jitter of observation satellites is a problem that degenerates the development of imaging quality and resolution. In order to reduce the geo-positioning errors and improve the geometric accuracy of remote sensing images, satellite jitter have been studied in recent years. In this work, a generative adversarial network (GAN) architecture is proposed to automatically learn and correct the deformed scene features from a single remote sensing image. In the proposed GAN, a convolutional neural network (CNN) is designed to discriminate the inputs and another CNN is used to generate so-called fake inputs. In order to explore the usefulness and effectiveness of GAN for jitter detection, the proposed GAN are trained on part of PatternNet dataset and tested on three popular remote sensing datasets. Several experiments show that the proposed models provide competitive results compared to other methods. the proposed GAN reveals the huge potential of GAN-based methods for the analysis of attitude jitter from remote sensing images.
The Driving Factors of Global Land Surface Albedo: An Analysis with the Glass and Merra-2 Data

Land surface albedo is an important parameter in surface energy budget, which determines the reflected/absorbed fraction of shortwave solar radiation of land surfaces. As the land surface albedo is highly spatio-temporal dynamic, it is important to understanding the spatial patterns of the leading factors of land surface albedo. In this study, we analyzed the driving factors of global land surface albedo from 1982 to 2015 with the global land surface satellite (GLASS) and the modern-era retrospective analysis for research and applications version 2 (Merra-2) datasets based on the multilinear regression and partial correlation methods. The results show that the variations of land surface albedo can be largely explained by the three major factors: snow cover extent, vegetation dynamics, and surface soil moisture. The driving factors of land surface albedo vary with different area, and the spatial distribution patterns for driving factors of land surface albedo were also explored by this study.
A Survey on Motion Prediction of Pedestrians and Vehicles for Autonomous Driving

Autonomous vehicle (AV) industry has evolved rapidly during the past decade. Research and development in each sub-module (perception, state estimation, motion planning etc.) of AVs has seen a boost, both on the hardware (variety of new sensors) and the software sides (state-of-the-art algorithms). With recent advancements in achieving real-time performance using onboard computational hardware on an ego vehicle, one of the major challenges that AV industry faces today is modelling behaviour and predicting future intentions of road users. To make a self-driving car reason and execute the safest motion plan, it should be able to understand its interactions with other road users. Modelling such behaviour is not trivial and involves various factors e.g. demographics, number of traffic participants, environmental conditions, traffic rules, contextual cues etc. This comprehensive review summarizes the related literature. Specifically, we identify and classify motion prediction literature for two road user classes i.e. pedestrians and vehicles. The taxonomy proposed in this review gives a unified generic overview of the pedestrian and vehicle motion prediction literature and is built on three dimensions i.e. motion modelling approach, model output type, and situational awareness from the perspective of an AV.
Patch Size Selection for Analysis of Sub-Meter Resolution Hyperspectral Imagery of Forests

Very high resolution remote sensing data of forests, where individual tree crowns are separable, contains structural information on tree size and density. Such information is complementary to the spectral signatures currently used in forestry applications. Advanced machine learning methods, e.g. convolutional neural networks (CNNs), offer an automated and standardized way of retrieving both spectral and structural information from imagery. A key characteristic in CNNs is patch size, which should be large enough to include dominant structural scale, yet as small as possible to avoid unnecessary averaging. Our results show that the patch should be larger than one tree, but increasing it excessively reduces retrieval accuracy. Furthermore, large patch sizes can cause loss of independence between training and validation data, leading to overestimating model performance.
Potential of Automated Digital Hemispherical Photography and Wireless Quantum Sensors for Routine Canopy Monitoring and Satellite Product Validation

To better characterize the temporal dynamics of vegetation biophysical variables, a variety of automated in situ measurement techniques have been developed in recent years. In this study, we investigated automated digital hemispherical photography (DHP) and wireless quantum sensors, which were installed at two sites under the Copernicus Ground Based Observations for Validation (GBOV) project. Daily estimates of plant area index (PAI) and the fraction of absorbed photosynthetically active radiation (FAPAR) were obtained, which realistically described expected vegetation dynamics. Good correspondence with manual DHP and LAI-2000 data (RMSE = 0.39 to 0.90 for PAI, RMSE = 0.07 for FAPAR) provided confidence that the investigated approaches can deliver data of comparable quality to traditional in situ measurement techniques.
Megh Sansuchak: A Cloud Mask Algorithm for High Resolution Panchromatic Satellite Imagery

In the era of Remote Sensing application where generation of meaningful hypothesis plays an important role in decision making of certain situation, getting high resolution data with precise features is a necessary and important task. In that context, presence of clouds in satellite imagery sometimes misleads to high reflectance features. This problem becomes more challenging when working with panchromatic (gray band) images as the signature of cloud and high reflectance features like dessert and salt land are somewhere similar in terms of intensity. To address this issue, a cloud mask generation algorithm is proposed in this paper. This algorithm consists of fully connected convolutional neural network (CNN) based on U-net architecture. This algorithm which is named as Megh-Sansuchak requires no specific preprocessing. Experimental results show a significantly accurate cloud mask on a long strip of Cartosat-1 Panchromatic Imagery. Performance matrices for Megh-Sansuchak are estimated with respect to different loss functions.
A New User Oriented Platform to Develop AI for the Estimation of Bio-Geophysical Parameters from EO Data

Machine learning can be considered as a very important area within artificial intelligence and it is characterized by algorithms and techniques that learn by examples. In the last decade, mainly due to the improvements obtained in the field of high performance computing, such as the enhanced exploitation of cloud technology and of graphics processing units (GPU), machine learning models have gained considerable progress as far as remote sensing and Earth Observation (EO) applications are concerned. However, the need of huge quantities of data necessary for the training phase, may be still a limiting factor especially in problems addressing the quantitative estimation of geo-physical parameters. In this paper, we report about the design and the development of a new platform capable of meeting the requirements of scientists and researchers who are attracted by the use of machine learning but meet difficulties in the generation of reliable data sets. The platforms relies on the implementation of radiative transfer models, plus a bunch of appropriate functionalities, in order that simulated data can be added to those available by ground-truth campaigns.
Crack Propagation and Calving Front Monitoring Using SATO Filter

Ice shelves, the floating extensions of the Antarctic Ice Sheet, are important regulators of the ice discharge and need to be monitored. Among others, calving processes and fractures control their ability to restrain the upward ice flow. While models integrate calving laws, their comparison with observations is still sparse and is mainly performed through visual inspection. In the present work, we hypothesized that the combination of high revisit rate SAR remote sensing products with medical imaging techniques can automatically detect cracks and monitor the calving front location. We focused our study on the Pine Island Glacier, in the Amundsen Sea Sector, with Sentinel-1 data over the period 2017 to 2020. Results show that the Sato filtering technique managed to precisely delineate the calving front, the icebergs, and the fractures on the ice shelf.
Wildfire Danger Assessment Over Southwest China Based on Short-Term Features of Weather and Fuel Variables

Wildfire is a typical complex nonlinear process, which has various interactions with weather, fuel, topography, and human variables. Previous studies on wildfire danger modeling mainly focused on the current climate or future scenarios of these variables. However, the outbreaks of wildfire are not only related to various factors at the current moment but also related to the change and trend of factors before the outbreak time point. The short-term features of weather and fuel dynamics $\text{(weather}\&\text{fuel}_{\text{short-term}_{-}\text{features}})$ were therefore assumed to be of great importance in wildfire danger modeling within this context. To verify this hypothesis, the wildfire danger was modeled across the fire-prone area over southwest China from 2001 to 2020. The state-of-the-art eXtreme Gradient Boosting (XGBoost) model was applied to explore the relations between this topography, weather, fuel, and human variables and the wildfire occurrence, as well as to identify the role of five categories of $\text{weather}\&\text{fuel}_{\text{short}-} \text{term}_{-}\text{features}$, i.e., the $t^{C3}, t^{Entropy}$, Ricker's wavelet transformation $(f^{cwt})$, regular statistic description $(t^{RSD})$, and fast Fourier transform $(t^{FFT})$. Results showed that wildfire danger modeling can be improved by consideration of the $\text{weather}\&\text{fuel}_{\text{short-term}_{-}\text{features}}$ which is proved to play an important role in driving the wildfire occurrence by factor importance analysis.
Identification of Commercial Tuna Hotspot in the Southern Waters of Java-Bali Through Satellite Remote Sensing Data

Tuna are mesopredators in oceanic food webs which play an essential role in pelagic communities and high sea ecosystems service. Hence, understanding the environmental preference of commercial tuna species is necessary to project the hot spot and to characterize its habitat. This study used generalized additive models (GAMs) to determine the relationship between the environmental variables derived from satellite data and the abundance of commercial tuna species. We used three environmental data as predictor variables and built seven GAMs models for each tuna species. The result stated that all environmental variables, namely sea surface temperature(SST), sea surface chlorophyll (SSC), and sea surface height (SSH) gave a significant influence on the existence of commercial tuna ($\mathrm{P} < 0.01$) in the study area. Also, we found different spatial patterns among three types of commercial tuna. Bigeye tuna prefers to stay in lower SST, while yellowfin tuna and albacore tuna prefer to stay in a certain SST range. Furthermore, they also have a different pattern of SSH where albacore and bigeye tuna prefer to stay in the higher SSH while yellowfin tuna prefers to stay between 0.49-0.65 m. However, three species have a similar range of SSC.
Explainable Student Agency Analytics

Several studies have shown that complex nonlinear learning analytics (LA) techniques outperform the traditional ones. However, the actual integration of these techniques in automatic LA systems remains rare because they are generally presumed to be opaque. At the same time, the current reviews on LA in higher education point out that LA should be more grounded to the learning science with actual linkage to teachers and pedagogical planning. In this study, we aim to address these two challenges. First, we discuss different techniques that open up the decision-making process of complex techniques and how they can be integrated in LA tools. More precisely, we present various global and local explainable techniques with an example of an automatic LA process that provides information about different resources that can support student agency in higher education institutes. Second, we exemplify these techniques and the LA process through recently collected student agency data in four courses of the same content taught by four different teachers. Altogether, we demonstrate how this process—which we call explainable student agency analytics—can contribute to teachers’ pedagogical planning through the LA cycle.
Integrated Gabor-Based Decision Fusion for Hyperspectral Image Classification

In recent years, Gabor filtering has been successfully applied in spectral-spatial hyperspectral image (HSI) classification tasks due to its strong power to characterize surface materials. A standard Gabor filter involves the real and imaginary parts. A common way to jointly use both the two parts is in the form of Gabor magnitude features. However, this combination form might weaken some unique characteristics of the two parts, therefore leading to limited improvement when compared to the real parts only. To solve this problem, we propose an integrated Gabor-based decision fusion (IG-DF) method for HSI classification. As the term suggests, our approach explores the integration of the two parts of Gabor features by means of a decision-level fusion strategy, where an extensive number of Gabor cubes generated with the real and imaginary parts of Gabor filters as well as different frequencies and orientations are directly fed into each classifier to yield a set of probability outputs. Afterward, the integrated Gabor-based decision fusion strategy is adopted to generate the final classification results from these probability outputs. Our experimental results, conducted on two commonly-used HSI datasets, demonstrate that the proposed IG-DF approach exhibits good improvements when compared with the real-part based and the magnitude feature based methods.
OSVidCap: A Framework for the Simultaneous Recognition and Description of Concurrent Actions in Videos in an Open-Set Scenario

Automatically understanding and describing the visual content of videos in natural language is a challenging task in computer vision. Existing approaches are often designed to describe single events in a closed-set setting. However, in real-world scenarios, concurrent activities and previously unseen actions may appear in a video. This work presents the OSVidCap, a novel open-set video captioning framework that recognizes and describes, in natural language, concurrent known actions and deal with unknown ones. The OSVidCap is based on the encoder-decoder framework and uses a detection-and-tracking-object-based mechanism followed by a background blurring method to focus on specific targets in a video. Additionally, we employ the TI3D Network with the Extreme Value Machine (EVM), which learns representations and recognizes unknown actions. We evaluate the proposed approach on the benchmark ActivityNet Captions dataset. Also, an enhanced version of the LIRIS human activity dataset was proposed by providing descriptions for each action. We also provide spatial, temporal, and caption annotations for existing unlabeled actions in the dataset - considered unknown actions in our experiments. Experimental results showed our method’s effectiveness in recognizing and describing concurrent actions in natural language and the strong ability to deal with detected unknown activities. Based on these results, we believe that the proposed approach can be potentially helpful for many real-world applications, including human behavior analysis, safety monitoring, and surveillance.
Joint Superpixel Segmentation and Graph Convolutional Network Road Extration for High-Resolution Remote Sensing Imagery

Extracting roads from remote sensing images has both civilian and military value, such as GIS data update, road navigation, military command and so on. The existing road extraction methods are mainly based on fully convolutional neural networks, and have achieved the state-of-the-art results. However, the convolutional and deconvolutional forms of these methods destroy the completeness of the extracted road. In this paper, we present a novel road extraction method for extracting complete roads from high-resolution remote sensing imagery based on joint superpixel segmentation and Graph Convolutional Network(GCN). The proposed method retains more spatial detail information as well as effectively improves the integrity of the extracted roads. Experiments were conducted on the Massachusetts Road dataset to compare our proposed method to other commonly used full convolutional techniques for road extraction. The results demonstrated the validity and better performance of the proposed method.
Designing of High Speed Permanent Magnet Electric Machines with Rotor Strength Verification

The paper is devoted to the design of high-speed electric machines with magnetoelectric excitation, taking into account the required electromagnetic and strength parameters provision. The active part sizes of such machines highly depend on electromagnetic loads – magnetic flux density in the air gap, armature line current density. Magnetic flux density, in turn, is a function of the non-magnetic gap magnitude between the rotor and the stator, which is determined by the thickness of the used permanent magnet (PM) sleeve mounted on the rotor surface. The finite element method is used to study the strength characteristics dependence - mechanical stresses and rotor elements deformation on the method of fixing the PM: glue, sleeve, glue and sleeve at the maximum speed of the autonomous mobile object motor-generator. The magnetic flux density in the gap is verified by the finite element method. guidance’s are given on the choice of the method of fixing the SPM on the rotor and the thickness of the carbon fiber sleeve.
Soil Organic Carbon Modelling with Digital Soil Mapping and Remote Sensing for Permanently Vegetated Areas

Soil organic matter is essential for preserving and maintaining a range of soil and ecosystem functions as well as supply and store carbon for climate change mitigation. Digital Soil Mapping techniques will be used to obtain a spatially continuous product, especially over permanently vegetated areas. Recently available satellite remote sensing data, with among other systems the Copernicus Sentinel, will be used as input for environmental covariates. Digital Soil Mapping, coupled together with Remote Sensing products, is a powerful tool to produce soil properties maps and monitoring the changes in soil conditions over time.
Towards Supervised Robot-assisted Physical Therapy after Hand Fractures

This paper reports on robotic support for physical therapists during rehabilitation after hand fractures. The robotic assistance is based on a humanoid robot Pepper, which is configured to demonstrate and eventually supervise 18 standard rehabilitation exercises. Pepper first explains to a patient how an exercise should be performed, then repeats the instructions and counts how many times an exercise has been correctly repeated. At the same time, he converses with the patient to increase motivation to continue exercising. A neural network based approach has been proposed to automatically detect the patient’s performance. It was found that by using the collected datasets (images) to interpret American and Arabic sign languages, it is possible to obtain a reliable feedback on the exercise performance, allowing a decision on the number of successful repetitions for each exercise. Finally, some challenges encountered are highlighted and their possible solutions are discussed.
Inversion Analysis of Sea Ice Physical Parameters Through Radiative Transfer Model and Simulated Annealing Method

In this paper, the work done for a modified sea ice inverse scattering model, the Radiative Transfer Inverse Scattering Model with Simulated Annealing (RTISM-SA) is presented. The study utilizes artificially grown saline ice data measured and reported during the CRRELEX'93 experiments to verify the performance of the inverse model. The results for the retrieval of saline ice thickness using both single and multi-polarization radar backscatter data shall be discussed. The results show that retrieval of sea ice thickness using the inverse model is possible and that the use of multi-polarization data improves the accuracy. Next, the model is extended to incorporate multi-parameter retrieval. The latest results show promise in the use of the RTISM-SA to retrieve multiple physical parameters of the sea ice.
Local Structure Graph Discriminant Embedding for Hyperspectral Image Classification

Graph learning is an effective technique to reduce the dimensionality of hyperspectral image (HSI) and improve classification result. However, the previous graph methods don't consider the local structure of each pixel in HSI. HSI has a complex non-linear structure, and the local structure can be regarded as a linear distribution. Therefore, we propose a local structure graph discriminant embedding (LSGDE) method to better reveal the intrinsic properties of HSI. This method constructs an intraclass and an interclass structure graphs to compact the intraclass samples and separate the interclass samples. Meanwhile, an interclass weighted scatter based on probability distribution is designed to enhance the discriminative ability of different classes. Then, a projection matrix can be obtained to map high-dimensional data into a low-dimensional space. Experiments on the HoustonU data set show that LSGDE can achieve better performance than the related DR methods.
In-Sat: A Novel Land Cover Classification Dataset for Indian Subcontinent

Remote sensing through satellite imagery is applied widely for environmental control, urban planning and land cover classification. To this end, supervised deep learning models can fully exploit the potential of satellite images. However, such models require large annotated datasets. Therefore, through this study we present two novel labelled satellite image datasets, namely In-Sat1, based on Sentinel-2 Satellite Imagery and Google Earth Imagery covering Indian subcontinent regions. We provide benchmarks and detailed analysis for these datasets using state-of-the-art and our improved remote sensing deep learning models which achieve 91% and 81% overall accuracy in region-wise split setting and 98% and 94% overall accuracy in class-wise split setting for Sentinel-2 data and Google Earth data respectively. We also demonstrate the application of our system for change detection.
Edge Guided Structure Extraction for Hyperspectral Image Classification

In this paper, a novel edge guided structure extraction method is proposed for hyperspectral images classification, which consists of the following steps: First, the spectral dimension of the hyperspectral image is reduced with an averaging-based method. Then, the structural features is extracted by an extended relative total variation (ERTV) inspired by a learned edge probability map which serves as one of the major cues in the structure extraction process. Finally, the extracted structural features are fed into SVM for classification. Experimental results on two publicly available hyperspectral data sets demonstrate the competitive performance over several state-of-the-art classification approaches.