Reviewerly/exHarmony huggingface dataset:
Propaganda Detection Robustness Through Adversarial Attacks Driven by eXplainable AI

Pre-trained language models like BERT have shown remarkable success in many areas, including the detection of propaganda in text messages. However, recent studies have uncovered the vulnerability of these language models to adversarial text attacks. Attacks are perpetrated by perturbing or paraphrasing original text instances to hack the detection model. A crucial task for performing the attack is identifying critical message words on which the detection model leverages. In this sense, this work focuses on the role of the malicious use of eXplainable Artificial Intelligence (xAI) in increasing the effectiveness of adversarial text attacks or, dually, the aid its correct use may provide in measuring the robustness of propaganda detection models. The approach proposed here leverages xAI and Adversarial Text Generation techniques to simulate malicious attacks and measure the robustness of a propaganda detection model based on BERT. The attacks involve generating a new dataset by perturbing critical words in the original one identified with the aid of xAI (SHAP and LIME). The effectiveness of terms determined using xAI methods is compared with a statistical keyword extractor (YAKE!). These methods are adopted to detect the most important words as perturbation targets. The goal is to quantify the impact of disrupted instances on learning model performance. Experiments on the SemEval 2020 task 11 dataset reveal that modifying words detected by xAI methods significantly affects classification performance by reducing accuracy by $$30\%$$ . These results demonstrate the effectiveness of xAI methods in system fooling attempts, highlighting the need to enhance learning system robustness.
Evaluating the Stability of Semantic Concept Representations in CNNs for Robust Explainability

Abstract Analysis of how semantic concepts are represented within Convolutional Neural Networks (CNNs) is a widely used approach in Explainable Artificial Intelligence (XAI) for interpreting CNNs. A motivation is the need for transparency in safety-critical AI-based systems, as mandated in various domains like automated driving. However, to use the concept representations for safety-relevant purposes, like inspection or error retrieval, these must be of high quality and, in particular, stable. This paper focuses on two stability goals when working with concept representations in computer vision CNNs: stability of concept retrieval and of concept attribution. The guiding use-case is a post-hoc explainability framework for object detection (OD) CNNs, towards which existing concept analysis (CA) methods are successfully adapted. To address concept retrieval stability, we propose a novel metric that considers both concept separation and consistency, and is agnostic to layer and concept representation dimensionality. We then investigate impacts of concept abstraction level, number of concept training samples, CNN size, and concept representation dimensionality on stability. For concept attribution stability we explore the effect of gradient instability on gradient-based explainability methods. The results on various CNNs for classification and object detection yield the main findings that (1) the stability of concept retrieval can be enhanced through dimensionality reduction via data aggregation, and (2) in shallow layers where gradient instability is more pronounced, gradient smoothing techniques are advised. Finally, our approach provides valuable insights into selecting the appropriate layer and concept representation dimensionality, paving the way towards CA in safety-critical XAI applications.
Addressing the Scalability Bottleneck of Semantic Technologies at Bosch

At the heart of smart manufacturing is real-time semi-automatic decision-making. Such decisions are vital for optimizing production lines, e.g., reducing resource consumption, improving the quality of discrete manufacturing operations, and optimizing the actual products, e.g., optimizing the sampling rate for measuring product dimensions during production. Such decision-making relies on massive industrial data thus posing a real-time processing bottleneck.
RoXi: A Framework for Reactive Reasoning

The Stream Reasoning research paradigm aims to target application domains that need to solve both data variety and velocity at the same time. Many of these domains benefit from processing the data as close to the source as possible, e.g. in Internet of Things applications we see a paradigm shift towards Edge processing, and in the Web, we see more and more effort towards decentralization of the Web where applications run directly inside the browser. However, current Stream Reasoning engines are not able to reuse the same code-base to run applications in the cloud, edge, or browser. In this paper, we present RoXi, a Reactive Reasoning framework that provides the needed building blocks to realize Stream Reasoning applications that can target cloud, edge, and browser environments.
Deciphering DeFi: A Comprehensive Analysis and Visualization of Risks in Decentralized Finance

Decentralized finance (DeFi) promises a revolution in financial accessibility, transparency, and automation. Yet, its very novelty exposes participants to a number of additional risks and challenges. This study aims to address the risks associated with DeFi, while also conducting a comparative analysis to those of classical/traditional finance (TradFi). After introducing DeFi and its defining characteristics, such as the use of smart contracts, blockchain technology, and decentralized governance, the paper outlines the principal risks associated with DeFi. Drawing insights from an extensive literature review of 200 recent articles, of which 50 were thoroughly analyzed, the study compares risks of DeFi and TradFi, categorizing these into systematic and unsystematic risks. Furthermore, we introduce the ‘risk wheel’, an innovative tool tailored to understand and navigate the subtleties of DeFi risks, finding potential applications in risk assessment, management, and even education. This paper’s primary objective is to provide a detailed and impartial examination of the risks associated with DeFi and their comparison to traditional finance in order to assist stakeholders in making informed decisions and mitigating possible losses.
The 2022 Ludii AI competition

The Ludii AI Competition involves general game playing events focused on developing agents that can play a wide variety of board games. In the 2022 edition, three competition tracks were proposed: Kilothon, General Game Playing, and Learning. All tracks used the Ludii general game system to provide the necessary games and API. This paper reports the motivation, context, and results of the 2022 Ludii AI Competition.
Impact of Imperfect Timekeeping on Quantum Control

In order to unitarily evolve a quantum system, an agent requires knowledge of time, a parameter that no physical clock can ever perfectly characterize. In this Letter, we study how limitations on acquiring knowledge of time impact controlled quantum operations in different paradigms. We show that the quality of timekeeping an agent has access to limits the circuit complexity they are able to achieve within circuit-based quantum computation. We do this by deriving an upper bound on the average gate fidelity achievable under imperfect timekeeping for a general class of random circuits. Another area where quantum control is relevant is quantum thermodynamics. In that context, we show that cooling a qubit can be achieved using a timer of arbitrary quality for control: timekeeping error only impacts the rate of cooling and not the achievable temperature. Our analysis combines techniques from the study of autonomous quantum clocks and the theory of quantum channels to understand the effect of imperfect timekeeping on controlled quantum dynamics.
Lessons Learned in Transcribing 5000 h of Air Traffic Control Communications for Robust Automatic Speech Understanding

Voice communication between air traffic controllers (ATCos) and pilots is critical for ensuring safe and efficient air traffic control (ATC). The handling of these voice communications requires high levels of awareness from ATCos and can be tedious and error-prone. Recent attempts aim at integrating artificial intelligence (AI) into ATC communications in order to lessen ATCos’s workload. However, the development of data-driven AI systems for understanding of spoken ATC communications demands large-scale annotated datasets, which are currently lacking in the field. This paper explores the lessons learned from the ATCO2 project, which aimed to develop an unique platform to collect, preprocess, and transcribe large amounts of ATC audio data from airspace in real time. This paper reviews (i) robust automatic speech recognition (ASR), (ii) natural language processing, (iii) English language identification, and (iv) contextual ASR biasing with surveillance data. The pipeline developed during the ATCO2 project, along with the open-sourcing of its data, encourages research in the ATC field, while the full corpus can be purchased through ELDA. ATCO2 corpora is suitable for developing ASR systems when little or near to no ATC audio transcribed data are available. For instance, the proposed ASR system trained with ATCO2 reaches as low as 17.9% WER on public ATC datasets which is 6.6% absolute WER better than with “out-of-domain” but gold transcriptions. Finally, the release of 5000 h of ASR transcribed speech—covering more than 10 airports worldwide—is a step forward towards more robust automatic speech understanding systems for ATC communications.
Fine-Grained Domain Adaptation for Chinese Syntactic Processing

Syntactic processing is fundamental to natural language processing. It provides rich and comprehensive syntax information in sentences that could be potentially beneficial for downstream tasks. Recently, pretrained language models have shown great success in Chinese syntactic processing, which typically involves word segmentation, POS tagging, and dependency parsing. However, the on-going research never ends since performance would be degraded drastically when tested on a highly-discrepant domain. This problem is widely accepted as domain adaptation, where the test domain differs from the training domain in supervised learning. Self-training is one promising solution for it, and straightforward source-to-target adaptation has already shown remarkable effectiveness in previous work. While this strategy ignores the fact that sentences of the target domain sentences may have very different gaps from the source training domain. More specifically, sentences with large gaps might fail by direct self-training adaptation. To this end, we propose fine-grained domain adaptation for Chinese syntactic processing in this work, aiming to model the gaps between the source and the target domains accurately and progressively. The key idea is to divide the target domain into fine-grained subdomains by using a specified domain distance metric, and then perform gradual self-training on the subdomains. We further offer an intuitive theoretical illustration based on the theory of Kumar et al. (2020) approximately. In addition, a novel representation learning framework is proposed to encode fine-grained subdomains effectively, aiming to utilize the above idea fully. Experimental results on benchmark datasets show that our method can achieve significant improvements over a variety of baselines.
PEAR: Positional-encoded Asynchronous Autoregression for satellite anomaly detection

This paper proposes a Positional-Encoded Asynchronous AutoRegression (PEAR) method for satellite anomaly detection. We empirically observe that a single classification model can hardly detect unknown anomalous situations and neglect the Markov nature of temporal satellite data. To address this, we adopt an autoregressive model to deal with the prediction of unknown anomaly for satellite data. We further propose a non-uniform temporal encoding method for asynchronous data and a median filtering method for more accurate detection. To reduce the effect of outliers, we employ an adaptive threshold selection method to achieve a more robust classification boundary. We test the proposed method on the time series prediction models including LSTM and transformers and we further employ the positional encoding strategy to improve the modeling capabilities of the Transformer model on high-frequency information. Experiments on real satellite data demonstrate that the proposed PEAR method outperforms the baseline method by 55.79%.
Mutual-Taught Deep Clustering

Deep clustering seeks to group data into distinct clusters using deep learning techniques. Existing approaches of deep clustering can be broadly categorized into two groups: offline clustering based on unsupervised representation learning and online clustering based on unsupervised classification. While both groups have demonstrated impressive performance in deep clustering, no study has explored the integration of their respective strengths. To this end, we propose Mutual-Taught Deep Clustering (MTDC), which unifies unsupervised representation learning and unsupervised classification into a framework while realizing mutual promotion using a novel mutual-taught mechanism. Specifically, MTDC alternates between predicting pseudolabels in label space and estimating semantic similarity in feature space during training. Moreover, pseudolabels provide weakly-supervised information to enhance unsupervised representation learning, while semantic similarities function as structural priors that regularize unsupervised classification. Consequently, unsupervised classification and unsupervised representation learning can mutually benefit from one another. MTDC is decoupled from prevailing deep clustering methods. For the sake of clarity, we build upon a straightforward baseline in this paper. Despite its simplicity, we demonstrate that MTDC is exceedingly efficacious and consistently enhances the baseline results by substantial margins. For example, MTDC achieves 2.5%∼7.9% (NMI), 3.0%∼13.9% (ACC), and 3.1%∼16.7% (ARI) gains over the baseline on six widely used image datasets. Source code is available at:https://github.com/yichenwang231/MTDC.
Quantum Rényi entropy functionals for bosonic Gaussian systems

In this study, the quantum Rényi entropy power inequality of order p>1 and power κ is introduced as a quantum analog of the classical Rényi-p entropy power inequality. To derive this inequality, we first exploit the Wehrl-p entropy power inequality on bosonic Gaussian systems via the mixing operation of quantum convolution, which is a generalized beam-splitter operation. This observation directly provides a quantum Rényi-p entropy power inequality over a quasi-probability distribution for D-mode bosonic Gaussian regimes. The proposed inequality is expected to be useful for the nontrivial computing of quantum channel capacities, particularly universal upper bounds on bosonic Gaussian quantum channels, and a Gaussian entanglement witness in the case of Gaussian amplifier via squeezing operations.
Characteristics and identification of risky driving behavior in expressway tunnel based on behavior spectrum

Expressway tunnels are semi-enclosed structures characterized by monotonous alignment transitions and unique lighting environments, which can easily lead to drivers developing constrained and irritable psychology. This may result in risky behaviors such as speeding and fatigued driving. Previous research on tunnel driving behavior mainly focuses on visual factors, neglecting the nonstationary time-series impacts of combined parameters on risky driving. Firstly, 30 drivers were recruited to carry out the real test. Then, based on the evolution of time series, drawing inspiration from the concept of lineage in biology, and considering multiple driving performance indicators, driving behavior chains and the feature spectrum were constructed. The characteristics of the behavioral spectrum were divided into six groups: electroencephalogram, heart rate, eye movement, speed, steering, and car-following behavior. Subsequently, spectral analysis using the spectral radius property of matrix theory revealed distinctive characteristics of risky driving behavior. The study deeply explored the inducing mechanism, hidden patterns, and rules of risky driving behavior under the coupling effect of tunnel environment and drivers' attributes. Finally, the significant features that influence driving behavior were used as input variables for constructing identification models using the Adaptive Boosting (AdaBoost) and Random Forest (RF) algorithms. Synthetic Minority Over-sampling Technique (SMOTE) and Adaptive Synthetic Sampling (ADASYN) were employed for oversampling. The results indicate that the ADASYN-RF algorithm outperformed others, achieving a precise recall rate area under the curve (AUPRC) of 0.978 when using the spectral radius of the speed and steering groups as input variables. These findings offer theoretical guidance for developing tunnel traffic safety strategies.
Power, Equity, and Building Better Robots

As HRI researchers, designers, and developers we need to reflect on the ways that power pervades the social contexts we're designing for and in. What can we do, with the power we have as designers, to produce more equitable HRI?
Efficient Large-Scale Damage Assessment After Natural Disasters With UAVS and Deep Learning

Frequent and increasingly severe natural disasters due to climate change threaten human health and infrastructure. The provision of accurate, timely, and understandable information has the potential to revolutionize disaster management. While traditional analyses provide some insights into the data, the complexity, scale, and multi-disciplinary nature of the data necessitate advanced, intelligent solutions. The main barrier is the lack of near real-time data analysis. Recently there has been a surge of practical applications of real-time semantic segmentation. Real-time semantic segmentation requires fast and high-quality predictions. As a result, lightweight architectures with low latency and computational costs are necessary for efficient semantic segmentation. In this article, we developed several encoder-decoder and two-pathway architectures and compared their performance on a novel RescuNet dataset.
Cross-Domain SAR Ship Detection in Strong Interference Environment Based On Image-to-Image Translation

The model performance of object detection task may dramatically deteriorate when meeting the new dataset with discrepant data distribution compared with trained images. Especially for Synthetic Aperture Radar (SAR) images, the complicated imaging mechanism and diverse environments probably induce intense changes in image appearance and hurt the detection capability and robustness of models based on deep learning. In this paper, a method of learning strong interference characteristics of SAR images is proposed and conducted to generate artificial SAR images as extra training samples in the downstream task -- object detection to improve the detection accuracy and decrease the missing rate of models. Our approach utilized as a data augmentation strategy without annotation cost is confirmed to be efficacious and reliable by multiple experiments.
Towards Unsupervised Multi-Temporal Satellite Image Super-Resolution

Multi-temporal super-resolution (SR) whereby a number of images of the same scene acquired at different times are fused to enhance its spatial resolution has recently enjoyed great success thanks to advances in deep learning methods. However, the literature has so far focused on supervised training approaches that require the availability of high-resolution (HR) images at the target resolution. This is a significant limitation because such imagery may not exist, might be difficult to source or exhibit domain gaps such as different spectral bands or radiometric characteristics. Unsupervised training approaches that do not require imagery beyond the input low resolution are needed to overcome this limitation. This paper presents a first analysis of the problem, taking inspiration from the literature on blind single-image SR, but also focusing on the uniqueness of multi-temporal satellite images. Our preliminary results show that it is indeed possible to develop accurate deep learning models for multi-temporal SR without HR images.
Light: Joint Individual Building Extraction and Height Estimation from Satellite Images Through a Unified Multitask Learning Network

Building extraction and height estimation are two important basic tasks in remote sensing image interpretation, which are widely used in urban planning, real-world 3D construction, and other fields. Most of the existing research regards the two tasks as independent studies. Therefore the height information cannot be fully used to improve the accuracy of building extraction and vice versa. In this work, we combine the individuaL buIlding extraction and heiGHt estimation through a unified multiTask learning network (LIGHT) for the first time, which simultaneously outputs a height map, bounding boxes, and a segmentation mask map of buildings. Specifically, LIGHT consists of an instance segmentation branch and a height estimation branch. In particular, so as to effectively unify multi-scale feature branches and alleviate feature spans between branches, we propose a Gated Cross Task Interaction (GCTI) module that can efficiently perform feature interaction between branches. Experiments on the DFC2023 dataset show that our LIGHT can achieve superior performance, and our GCTI module with ResNet 101 as the backbone can significantly improve the performance of multitask learning by 2.8% AP50 and 6.5% δ <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</inf> , respectively.
Feature Attribution Methods for Multivariate Time-Series Explainability in Remote Sensing

Numerous remote sensing applications rely on temporal satellite data, and Deep learning models are increasingly being used for such tasks. Nevertheless, these models operate as black boxes, lacking transparency and understandability. We address this gap by using explainable AI on an agricultural task. Specifically, we trained a recurrent neural network on individual pixels from multispectral time-series of Sentinel-2 satellite images to predict crop yield. We then applied nine feature attribution methods on a sample of the dataset and computed the spectral and temporal contributions to the final individual predictions. The aggregated results were evaluated qualitatively and quantitatively. Results suggest that LIME and Shapley sampling value methods performed best on the quantitative scores, followed by GradientShap. Most backpropagation-based techniques had highly inconsistent scores across the explained data points. Finally, to guide remote sensing practitioners in using Explainable AI on similar datasets, we further discuss some selection criteria to be considered.
EarthQA: A Question Answering Engine for Earth Observation Data Archives <sup>*</sup>

EarthQA is a question answering engine that accepts questions in natural language (English) that ask for satellite images satisfying certain criteria and returns links to such datasets, that can be then downloaded from the CREODIAS cloud platform. The questions can refer to image metadata (e.g., satellite platform, sensing period, cloud cover etc.) but also to entities from the knowledge graph DB-pedia (e.g., Mount Etna or the city of Munich). In this way, the users can ask questions like "Give me Sentinel-2 satellite images that show Mount Etna, have been taken in February 2021 and have cloud cover less than 10%."