pwc-archive/papers-with-abstracts huggingface dataset:
Using Multi-Temporal Sentinel-1 and Sentinel-2 data for water bodies mapping
Authors: Luigi Russo, Francesco Mauro, Babak Memar, Alessandro Sebastianelli, Paolo Gamba, Silvia Liberata Ullo
Date: 2024-01-05 00:00:00
Tasks: Benchmarking

Climate change is intensifying extreme weather events, causing both water scarcity and severe rainfall unpredictability, and posing threats to sustainable development, biodiversity, and access to water and sanitation. This paper aims to provide valuable insights for comprehensive water resource monitoring under diverse meteorological conditions. An extension of the SEN2DWATER dataset is proposed to enhance its capabilities for water basin segmentation. Through the integration of temporally and spatially aligned radar information from Sentinel-1 data with the existing multispectral Sentinel-2 data, a novel multisource and multitemporal dataset is generated. Benchmarking the enhanced dataset involves the application of indices such as the Soil Water Index (SWI) and Normalized Difference Water Index (NDWI), along with an unsupervised Machine Learning (ML) classifier (k-means clustering). Promising results are obtained and potential future developments and applications arising from this research are also explored.
Kronecker Product Feature Fusion for Convolutional Neural Network in Remote Sensing Scene Classification
Authors: Yinzhu Cheng
Date: 2024-01-08 00:00:00
Tasks: Scene Classification

Remote Sensing Scene Classification is a challenging and valuable research topic, in which Convolutional Neural Network (CNN) has played a crucial role. CNN can extract hierarchical convolutional features from remote sensing imagery, and Feature Fusion of different layers can enhance CNN's performance. Two successful Feature Fusion methods, Add and Concat, are employed in certain state-of-the-art CNN algorithms. In this paper, we propose a novel Feature Fusion algorithm, which unifies the aforementioned methods using the Kronecker Product (KPFF), and we discuss the Backpropagation procedure associated with this algorithm. To validate the efficacy of the proposed method, a series of experiments are designed and conducted. The results demonstrate its effectiveness of enhancing CNN's accuracy in Remote sensing scene classification.
Revolutionizing Underwater Exploration of Autonomous Underwater Vehicles (AUVs) and Seabed Image Processing Techniques
Authors: Rajesh Sharma R, Akey Sungheetha, Dr Chinnaiyan R
Date: 2023-11-22 00:00:00

The oceans in the Earth's in one of the last border lines on the World, with only a fraction of their depths having been explored. Advancements in technology have led to the development of Autonomous Underwater Vehicles (AUVs) that can operate independently and perform complex tasks underwater. These vehicles have revolutionized underwater exploration, allowing us to study and understand our oceans like never before. In addition to AUVs, image processing techniques have also been developed that can help us to better understand the seabed and its features. In this comprehensive survey, we will explore the latest advancements in AUV technology and seabed image processing techniques. We'll discuss how these advancements are changing the way we explore and understand our oceans, and their potential impact on the future of marine science. Join us on this journey to discover the exciting world of underwater exploration and the technologies that are driving it forward.
Diffusion MRI with Machine Learning
Authors: Davood Karimi, Simon K. Warfield
Date: 2024-01-01 00:00:00
Tasks: Diffusion  MRI

\hspace{2mm} Diffusion-weighted magnetic resonance imaging (dMRI) of the brain offers unique capabilities including noninvasive probing of tissue microstructure and structural connectivity. It is widely used for clinical assessment of disease and injury, and for neuroscience research. Analyzing the dMRI data to extract useful information for medical and scientific purposes can be challenging. The dMRI measurements may suffer from strong noise and artifacts, and may exhibit high inter-session and inter-scanner variability in the data, as well as inter-subject heterogeneity in brain structure. Moreover, the relationship between measurements and the phenomena of interest can be highly complex. Recent years have witnessed increasing use of machine learning methods for dMRI analysis. This manuscript aims to assess these efforts, with a focus on methods that have addressed data preprocessing and harmonization, microstructure mapping, tractography, and white matter tract analysis. We study the main findings, strengths, and weaknesses of the existing methods and suggest topics for future research. We find that machine learning may be exceptionally suited to tackle some of the difficult tasks in dMRI analysis. However, for this to happen, several shortcomings of existing methods and critical unresolved issues need to be addressed. There is a pressing need to improve evaluation practices, to increase the availability of rich training datasets and validation benchmarks, as well as model generalizability, reliability, and explainability concerns.
Choosing the Right Path for AI Integration in Engineering Companies: A Strategic Guide
Authors: Rimma Dzhusupova, Jan Bosch, Helena Holmstrom Olsson
Date: 2023-12-25 00:00:00

The Engineering, Procurement and Construction (EPC) businesses operating within the energy sector are recognizing the increasing importance of Artificial Intelligence (AI). Many EPC companies and their clients have realized the benefits of applying AI to their businesses in order to reduce manual work, drive productivity, and streamline future operations of engineered installations in a highly competitive industry. The current AI market offers various solutions and services to support this industry, but organizations must understand how to acquire AI technology in the most beneficial way based on their business strategy and available resources. This paper presents a framework for EPC companies in their transformation towards AI. Our work is based on examples of project execution of AI-based products development at one of the biggest EPC contractors worldwide and on insights from EPC vendor companies already integrating AI into their engineering solutions. The paper covers the entire life cycle of building AI solutions, from initial business understanding to deployment and further evolution. The framework identifies how various factors influence the choice of approach toward AI project development within large international engineering corporations. By presenting a practical guide for optimal approach selection, this paper contributes to the research in AI project management and organizational strategies for integrating AI technology into businesses. The framework might also help engineering companies choose the optimum AI approach to create business value.
Training microrobots to swim by a large language model
Authors: Zhuoqun Xu, Lailai Zhu
Date: 2024-01-21 00:00:00
Tasks: Decision Making, Few-Shot Learning, Language Modeling, Language Modelling, Large Language Model

Machine learning and artificial intelligence have recently represented a popular paradigm for designing and optimizing robotic systems across various scales. Recent studies have showcased the innovative application of large language models (LLMs) in industrial control [1] and in directing legged walking robots [2]. In this study, we utilize an LLM, GPT-4, to train two prototypical microrobots for swimming in viscous fluids. Adopting a few-shot learning approach, we develop a minimal, unified prompt composed of only five sentences. The same concise prompt successfully guides two distinct articulated microrobots -- the three-link swimmer and the three-sphere swimmer -- in mastering their signature strokes. These strokes, initially conceptualized by physicists, are now effectively interpreted and applied by the LLM, enabling the microrobots to circumvent the physical constraints inherent to micro-locomotion. Remarkably, our LLM-based decision-making strategy substantially surpasses a traditional reinforcement learning method in terms of training speed. We discuss the nuanced aspects of prompt design, particularly emphasizing the reduction of monetary expenses of using GPT-4.
Tracr: Compiled Transformers as a Laboratory for Interpretability
Authors: David Lindner, János Kramár, Sebastian Farquhar, Matthew Rahtz, Thomas McGrath, Vladimir Mikulik
Date: 2023-01-12 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Decoder

We show how to "compile" human-readable programs into standard decoder-only transformer models. Our compiler, Tracr, generates models with known structure. This structure can be used to design experiments. For example, we use it to study "superposition" in transformers that execute multi-step algorithms. Additionally, the known structure of Tracr-compiled models can serve as ground-truth for evaluating interpretability methods. Commonly, because the "programs" learned by transformers are unknown it is unclear whether an interpretation succeeded. We demonstrate our approach by implementing and examining programs including computing token frequencies, sorting, and parenthesis checking. We provide an open-source implementation of Tracr at https://github.com/google-deepmind/tracr.
Learning Functions of Few Arbitrary Linear Parameters in High Dimensions
Authors: Massimo Fornasier, Karin Schnass, Jan Vybiral
Date: 2010-08-18 00:00:00
Tasks: compressed sensing

Let us assume that $f$ is a continuous function defined on the unit ball of $\mathbb R^d$, of the form $f(x) = g (A x)$, where $A$ is a $k \times d$ matrix and $g$ is a function of $k$ variables for $k \ll d$. We are given a budget $m \in \mathbb N$ of possible point evaluations $f(x_i)$, $i=1,...,m$, of $f$, which we are allowed to query in order to construct a uniform approximating function. Under certain smoothness and variation assumptions on the function $g$, and an {\it arbitrary} choice of the matrix $A$, we present in this paper 1. a sampling choice of the points $\{x_i\}$ drawn at random for each function approximation; 2. algorithms (Algorithm 1 and Algorithm 2) for computing the approximating function, whose complexity is at most polynomial in the dimension $d$ and in the number $m$ of points. Due to the arbitrariness of $A$, the choice of the sampling points will be according to suitable random distributions and our results hold with overwhelming probability. Our approach uses tools taken from the {\it compressed sensing} framework, recent Chernoff bounds for sums of positive-semidefinite matrices, and classical stability bounds for invariant subspaces of singular value decompositions.
SimMMDG: A Simple and Effective Framework for Multi-modal Domain Generalization
Authors: Hao Dong, Ismail Nejjar, Han Sun, Eleni Chatzi, Olga Fink
Date: 2023-10-30 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Contrastive Learning, Diversity, Domain Generalization

In real-world scenarios, achieving domain generalization (DG) presents significant challenges as models are required to generalize to unknown target distributions. Generalizing to unseen multi-modal distributions poses even greater difficulties due to the distinct properties exhibited by different modalities. To overcome the challenges of achieving domain generalization in multi-modal scenarios, we propose SimMMDG, a simple yet effective multi-modal DG framework. We argue that mapping features from different modalities into the same embedding space impedes model generalization. To address this, we propose splitting the features within each modality into modality-specific and modality-shared components. We employ supervised contrastive learning on the modality-shared features to ensure they possess joint properties and impose distance constraints on modality-specific features to promote diversity. In addition, we introduce a cross-modal translation module to regularize the learned features, which can also be used for missing-modality generalization. We demonstrate that our framework is theoretically well-supported and achieves strong performance in multi-modal DG on the EPIC-Kitchens dataset and the novel Human-Animal-Cartoon (HAC) dataset introduced in this paper. Our source code and HAC dataset are available at https://github.com/donghao51/SimMMDG.
Enhancing Adversarial Contrastive Learning via Adversarial Invariant Regularization
Authors: Xilie Xu, Jingfeng Zhang, Feng Liu, Masashi Sugiyama, Mohan Kankanhalli
Date: 2023-04-30 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Contrastive Learning

Adversarial contrastive learning (ACL) is a technique that enhances standard contrastive learning (SCL) by incorporating adversarial data to learn a robust representation that can withstand adversarial attacks and common corruptions without requiring costly annotations. To improve transferability, the existing work introduced the standard invariant regularization (SIR) to impose style-independence property to SCL, which can exempt the impact of nuisance style factors in the standard representation. However, it is unclear how the style-independence property benefits ACL-learned robust representations. In this paper, we leverage the technique of causal reasoning to interpret the ACL and propose adversarial invariant regularization (AIR) to enforce independence from style factors. We regulate the ACL using both SIR and AIR to output the robust representation. Theoretically, we show that AIR implicitly encourages the representational distance between different views of natural data and their adversarial variants to be independent of style factors. Empirically, our experimental results show that invariant regularization significantly improves the performance of state-of-the-art ACL methods in terms of both standard generalization and robustness on downstream tasks. To the best of our knowledge, we are the first to apply causal reasoning to interpret ACL and develop AIR for enhancing ACL-learned robust representations. Our source code is at https://github.com/GodXuxilie/Enhancing_ACL_via_AIR.
AdaGCL: Adaptive Subgraph Contrastive Learning to Generalize Large-scale Graph Training
Authors: Yili Wang, Kaixiong Zhou, Rui Miao, Ninghao Liu, Xin Wang
Date: 2022-10-17 00:00:00
Proceeding: ACM International Conference on Information & Knowledge Management (CIKM) 2022 10
Tasks: Contrastive Learning, Data Augmentation, Node Classification

Training graph neural networks (GNNs) with good generalizability on large-scale graphs is a challenging problem. Existing methods mainly divide the input graph into multiple subgraphs and train them in different batches to improve training scalability. However, the local batches obtained by such a strategy could contain topological bias compared with the complete graph structure. It has been studied that the topological bias results in more significant gaps between training and testing performances, or worse generalization robustness. A straightforward solution is to utilize contrastive learning, and train node embeddings to be robust and invariant among the augmented imperfect graphs. However, most of the existing work are inefficient by contrasting extensive node pairs at the large-scale graph. With random data augmentation, they may deteriorate the embedding process by transforming well-sampled batches into meaningless graph structures.

To bridge the gap between large-scale graph training and contrastive learning, we propose adaptive subgraph contrastive learning (AdaGCL). Given a batch of sampled subgraphs, we propose subgraph-granularity contrastive loss to compare the anchor node with a limited number of subgraphs, which reduces the computation cost. AdaGCL tailors two key components for batch training: (1) Batch-aware view generation to keep the intrinsic individual subgraph structures of batch to learn the informative node embeddings; (2) Batch-aware pair sampling to construct the positive and negative contrasting subgraphs based on anchor node label. Experiments show that AdaGCL can scale up to graphs with millions of nodes, and delivers the consistent improvement than the existing methods on various benchmark datasets. Furthermore, AdaGCL has comparable running time with the state-of-the-art contrastive learning methods that focus on improving efficiency. Finally, ablation studies of the two components of AdaGCL demonstrate their effectiveness to generalize the batch training.
MergedNET: A simple approach for one-shot learning in siamese networks based on similarity layers
Authors: John Atanbori, Samuel Rose
Date: 2022-10-14 00:00:00
Proceeding: Neurocomputing 2022 10
Tasks: Few-Shot Image Classification, Fine-Grained Image Classification, image-classification, Image Classification, Metric Learning, One-Shot Learning

Classifiers trained on disjointed classes with few labelled data points are used in one-shot learning to identify visual concepts from other classes. Recently, Siamese networks and similarity layers have been used to solve the one-shot learning problem, achieving state-of-the-art performance on visual-character recognition datasets. Various techniques have been developed over the years to improve the performance of these networks on fine-grained image classification datasets. They focused primarily on improving the loss and activation functions, augmenting visual features, employing multiscale metric learning, and pre-training and fine-tuning the backbone network. We investigate similarity layers for one-shot learning tasks and propose two frameworks for combining these layers into a MergedNet network. On all four datasets used in our experiment, MergedNet outperformed the baselines based on classification accuracy, and it generalises to other datasets when trained on miniImageNet.
Improved Knowledge Distillation for Crowd Counting on IoT Device
Authors: Zuo Huang, Richard O. Sinnott
Date: 2023-08-02 00:00:00
Proceeding: IEEE International Conference on Edge Computing and Communications 2023 8
Tasks: Crowd Counting, Knowledge Distillation, Quantization

Manual crowd counting for real-world problems is impossible or results in wildly inaccurate estimations. Deep learning is one area that has been applied to address this issue. Crowd counting is a computationally intensive task. Therefore, many crowd counting models employ large-scale deep convolutional neural networks (CNN) to achieve higher accuracy. However, these are typically at the cost of performance and inference speed. This makes such approaches difficult to apply in real-world settings, e.g., on Internet-of-Things (IoT) devices. To tackle this problem, one method is to compress models using pruning and quantization or use of lightweight model backbones. However, such methods often result in a significant loss of accuracy. To address this, some studies have explored knowledge distillation methods to extract useful information from large state-of-the-art (teacher) models to guide/train smaller (student) models. However, knowledge distillation methods suffer from the problem of information loss caused by hint-transformers. Furthermore, teacher models may have a negative impact on student models. In this work, we propose a method based on knowledge distillation that uses self-transformed hints and loss functions that ignore outliers to tackle real-world and challenging crowd counting tasks. Based on our approach, we achieve a MAE of 77.24 and a MSE of 276.17 using the JHU-CROWD++ [1] test set. This is comparable to state-of-the-art deep crowd counting models, but at a fraction of the original model size and complexity, thus making the solution suitable for IoT devices. The source code is available at https://github.com/huangzuo/effcc_distilled.
A Spitzoid Tumor dataset with clinical metadata and Whole Slide Images for Deep Learning models
Authors: Andrés Mosquera-Zamudio, Laëtitia Launet, Rocío del Amor, Anaïs Moscardó, Adrián Colomer, Valery Naranjo & Carlos Monteagudo
Date: 2023-10-16 00:00:00
Proceeding: Nature 2023 10
Tasks: Diagnostic, whole slide images

Spitzoid tumors (ST) are a group of melanocytic tumors of high diagnostic complexity. Since 1948, when Sophie Spitz first described them, the diagnostic uncertainty remains until now, especially in the intermediate category known as Spitz tumor of unknown malignant potential (STUMP) or atypical Spitz tumor. Studies developing deep learning (DL) models to diagnose melanocytic tumors using whole slide imaging (WSI) are scarce, and few used ST for analysis, excluding STUMP. To address this gap, we introduce SOPHIE: the first ST dataset with WSIs, including labels as benign, malignant, and atypical tumors, along with the clinical information of each patient. Additionally, we explain two DL models implemented as validation examples using this database.
MultiPoint: Cross-spectral registration of thermal and optical aerial imagery
Authors: Florian Achermann, Andrey Kolobov, Debadeepta Dey, Timo Hinzmann, Jen Jen Chung, Roland Siegwart, Nicholas Lawrance
Date: 2020-11-18 00:00:00
Proceeding: 2020 Conference on Robot Learning 2020 11
Tasks: Interest Point Detection

While optical cameras are ubiquitous in robotics, some robots can sense
the world in several sections of the electromagnetic spectrum simultaneously,
which can extend their capabilities in fundamental ways. For instance, many
fixed-wing UAVs carry both optical and thermal imaging cameras, potentially allowing them to detect temperature difference-induced atmospheric updrafts, map
their locations, and adjust their flight path accordingly to increase their time aloft.
A key step for unlocking the potential offered by multi-spectral data is generating
consistent, multi-spectral maps of the environment. In this work, we introduce
MultiPoint, a novel data-driven method for generating interest points and associated descriptors for registering optical and thermal image pairs without knowledge
of the relative camera viewpoints. Existing pixel-based alignment methods are accurate but too slow to work in near-real time, while feature-based methods such
as SuperPoint are fast but produce poor-quality cross-spectral matches due to interest point instability in thermal images. MultiPoint capitalizes on the strengths
of both approaches. An offline mutual information-based procedure is used to
align cross-spectral image pairs from a training set, which are then processed by
our generalized multi-spectral homographic adaptation stage to generate highly
repeatable interest points that are invariant across viewpoint changes in both spectra. These are used to train a MultiPoint deep neural network by exposing this
model to both same-spectrum and cross-spectral image pairs. This model is then
deployed for fast and accurate online interest point detection. We show that MultiPoint outperforms existing techniques for feature-based image alignment using
a dataset of real-world thermal-optical imagery captured by a UAV during flights
in different conditions and release this dataset, the first of its kind.