Reviewerly/exHarmony huggingface dataset:
Generative adversarial network augmentation for solving the training data imbalance problem in crop classification

Deep learning models offer great potential for advancing land monitoring using satellite data. However, they face challenges due to imbalanced real-world data distributions of land cover and crop types, hindering scalability and transferability. This letter presents a novel data augmentation method employing Generative Adversarial Neural Networks (GANs) with pixel-to-pixel transformation (pix2pix). This approach generates realistic synthetic satellite images with artificial ground truth masks, even for rare crop class distributions. It enables the creation of additional minority class samples, enhancing control over training data balance and outperforming traditional augmentation methods. Implementing this method improved the overall map accuracy (OA) and intersection over union (IoU) by 1.5% and 2.1%, while average crop type classes' user accuracy (UA) and producer accuracies (PA), as well as IoU, were improved by 11.2%, 6.4% and 10.2%.
Analysing time-use composition as dependent variables in physical activity and sedentary behaviour research: different compositional data analysis approaches

Abstract Recently, there has been a paradigm shift from considering physical activity and sedentary behaviour as “independent” risk factors of health to acknowledging their co-dependency and compositional nature. The focus is now on how these behaviours relate to each other rather than viewing them in isolation. Compositional data analysis (CoDA) is a methodology that has been developed specifically for compositional data and the number of publications using CoDA in physical activity and sedentary behaviour research has increased rapidly in the past years. Yet, only a small proportion of the published studies in physical activity and sedentary behaviour research have investigated the time-use composition as dependent variables. This could be related to challenges regarding the interpretation of the results and the lack of guidelines for deciding which statistical approach to use. Therefore, in this paper, four different approaches for analysing the time-use composition as dependent variables are presented and discussed. This paper advocates that the aim of research should guide how the dependent variable is defined and which data analysis approach is selected, and it encourages researchers to consider analysing time-use components as dependent variables in physical activity and sedentary behaviour research.
Ship Detection under Low-Visibility Weather Interference via an Ensemble Generative Adversarial Network

Maritime ship detection plays a crucial role in smart ships and intelligent transportation systems. However, adverse maritime weather conditions, such as rain streak and fog, can significantly impair the performance of visual systems for maritime traffic. These factors constrain the performance of traffic monitoring systems and ship-detection algorithms for autonomous ship navigation, affecting maritime safety. The paper proposes an approach to resolve the problem by visually removing rain streaks and fog from images, achieving an integrated framework for accurate ship detection. Firstly, the paper employs an attention generation network within an adversarial neural network to focus on the distorted regions of the degraded images. The paper also utilizes a contextual encoder to infer contextual information within the distorted regions, enhancing the credibility of image restoration. Secondly, a weighted bidirectional feature pyramid network (BiFPN) is introduced to achieve rapid multi-scale feature fusion, enhancing the accuracy of maritime ship detection. The proposed GYB framework was validated using the SeaShip dataset. The experimental results show that the proposed framework achieves an average accuracy of 96.3%, a recall of 95.35%, and a harmonic mean of 95.85% in detecting maritime traffic ships under rain-streak and foggy-weather conditions. Moreover, the framework outperforms state-of-the-art ship detection methods in such challenging weather scenarios.
Identifying equipment health status from maintenance records using Lexicon based Unsupervised Sentiment Analysis Adjusted for Negation (LUSAA-N)

During periodic inspection and repair of industrial equipment, technicians create maintenance records in the form of unstructured text to document valuable information regarding the equipment condition, operating environment, failed components, failure mechanism, etc. Inferring the equipment health status from these records can help in planning future maintenance actions to avoid expensive machine failures. However, automatic inference of the equipment health status is non-trivial due to various challenges inherent in analyzing unstructured text such as domain-specific vocabulary and negation words. Further, manual labeling of the records for supervised classification is arduous and impractical. Thus, in this paper, we present a model to automatically extract the equipment health status from maintenance records in a completely unsupervised manner. We show that using general English sentiment lexicons is highly inefficient for analyzing maintenance records, and thus propose a method that leverages sentiment lexicons curated for the maintenance domain and addresses the central issue of negation. We demonstrate the effectiveness of our proposed model over extant research using data from real-life maintenance records from oil rig equipment.
Inference in Probabilistic Answer Set Programming Under the Credal Semantics

Abstract Probabilistic Answer Set Programming under the credal semantics (PASP) describes an uncertain domain through an answer set program extended with probabilistic facts. The PASTA language leverages PASP to express statistical statements. A solver with the same name allows to perform inference in PASTA programs and, in general, in PASP. In this paper, we investigate inference in PASP, propose a new inference algorithm called aspcs based on Second Level Algebraic Model Counting (2AMC), and implement it into the aspmc solver. Then, we compare it with PASTA on a set of benchmarks: the empirical results show that, when the program does not contain aggregates, the new algorithm outperforms PASTA. However, when we consider PASTA statements and aggregates, we need to replace aggregates with a possibly exponential number of rules, and aspcs is slower than PASTA.
Named Entity Recognition and Linking for Entity Extraction from Italian Civil Judgements

The extraction of named entities from court judgments is useful in several downstream applications, such as document anonymization and semantic search engines. In this paper, we discuss the application of named entity recognition and linking (NEEL) to extract entities from Italian civil court judgments. To develop and evaluate our work, we use a corpus of 146 manually annotated court judgments. We use a pipeline that combines a transformer-based Named Entity Recognition (NER) component, a transformer-based Named Entity Linking (NEL) component, and a NIL prediction component. While the NEL and NIL prediction components are not fine-tuned on domain-specific data, the NER component is fine-tuned on the annotated corpus. In addition, we compare different masked language modeling (MLM) adaptation strategies to optimize the result and investigate their impact. Results obtained on a 30-document test set reveal satisfactory performance, especially on the NER task, and emphasize challenges to improve NEEL on similar documents. Our code is available on GitHub.( https://github.com/rpo19/pozzi_aixia_2023 . We are not allowed to publish sensitive data and the NER models trained on sensitive data.)
Towards Automatic Digitalization of Railway Engineering Schematics

Relay-based Railways Interlocking Systems (RRIS) carry out critical functions to control stations. Despite being based on old and hard-to-maintain electro-mechanical technology, RRIS are still pervasive. A powerful CAD modeling and analysis approach based on symbolic logic has been recently proposed to support the re-engineering of relay diagrams into more maintainable computer-based technologies. However, the legacy engineering drawings that need to be digitized consist of large, hand-drawn diagrams dating back several decades. Manually transforming such diagrams into the format of the CAD tool is labor-intensive and error-prone, effectively a bottleneck in the reverse-engineering process. In this paper, we tackle the problem of automatic digitalization of RRIS schematics into the corresponding CAD format with an integrative Artificial Intelligence approach. Deep learning-based methods, segment detection, and clustering techniques for the automated digitalization of engineering schematics are used to detect and classify the single elements of the diagram. These elementary elements can then be aggregated into more complex objects leveraging the domain ontology. First results of the method’s capability of automatically reconstructing the engineering schematics are presented.
Clique-TF-IDF: A New Partitioning Framework Based on Dense Substructures

Natural Language Processing (NLP) techniques are powerful tools for analyzing, understanding, and processing human language with a wide range of applications. In this paper we exploit NLP techniques, combined with Machine Learning clustering algorithms, to find good solutions to a traditional combinatorial problem, namely, the computation of a partition with high modularity of a graph. We introduce a novel framework, dubbed Clique-TF-IDF, for computing a graph partition. Such a framework leverages dense subgraphs of the input graph, modeled as maximal cliques, and characterizes each node in terms of the cliques it belongs to, similarly to a term-document matrix. Our experimental results show that the quality of the partitions produced by algorithm Clique-TF-IDF is comparable with that of the most effective algorithms in the literature. While our focus is on maximal cliques and partitioning algorithms, we believe that this strategy can be generalized to devise AI solutions for a variety of intractable combinatorial problems where some substructures can be efficiently enumerated and exploited.
Deep learning-based biometric image feature extraction for securing medical images through data hiding and joint encryption–compression

Images are promising information carriers when compared to other media documents in the healthcare domain. However, digital data transmission over unprotected wired or wireless networks poses a threat to the security of healthcare systems. As a result, the issue of copyright violation and identity theft can occur due to the unauthorised use of these data. This paper proposes a new secure method under a framework that embeds biometric fingerprint image features in a medical image without any perceptual distortion. This paper uses ResNet152 for biometric image feature extraction in the first stage and features to generate a secret key for embedding in the second stage. The method combines encryption and compression scheme based on a generated key, novel chaotic map and Huffman coding to enhance the security of medical images while reducing the storage consumption or bandwidth requirements if images are transmitted to remote servers. Experimental results show that the proposed method presents superior security with high imperceptibility and compression performance, ensuring its effectiveness as an image protection mechanism for medical applications. Extensive experimental results show that the proposed method achieves an average peak signal-to-noise ratio (PSNR) that is above 54 dB, a structural similarity index measure (SSIM) close to 1, a bit error rate (BER) of 0 and a normalised correlation (NC) of 1. Moreover, this method compresses the images up to 70% when tested on three standard datasets.
The History of Stealth Assessment and a Peek Into Its Future

Interactions with digital environments can yield huge amounts of electronic data—in terms of what was done, when, where, and how. What's known today as stealth assessment arose from a single question four decades ago: How can we mine digital data for meaningful information about a person's cognitive and noncognitive states, then use it to enhance learning? This query undergirded early work in intelligent tutoring systems but was handicapped by the technology of the time. This chapter describes the evolution of the concept and functioning of stealth assessment across the decades—how it emerged and where it may go in the future.
Robust Person Identification and Following in a Mobile Robot Based on Deep Learning and Optical Tracking

There is an exciting synergy between deep learning and robotics, combining the perception skills a deep learning system can achieve with the wide variety of physical responses a robot can perform. This article describes an embedded system integrated into a mobile robot capable of identifying and following a specific person reliably based on a convolutional neural network pipeline. In addition, the design incorporates an optical tracking system for supporting the inferences of the neural networks, allowing the determination of the position of a person using an RGB depth camera. The system runs on an NVIDIA Jetson TX2 board, an embedded System-on-Module capable of performing computationally demanding tasks onboard and handling the complexity needed to run a solid tracking and following algorithm. A robotic mobile base with the Jetson TX2 board attached receives velocity orders to move the system toward the target. The proposed approach has been validated on a mobile robotic platform that successfully follows a determined person, relying on the robustness of the combination of deep learning with optical tracking for working efficiently in a real environment.
Securing <scp>AI</scp>‐based healthcare systems using blockchain technology: A state‐of‐the‐art systematic literature review and future research directions

Abstract Healthcare institutions are progressively integrating artificial intelligence (AI) into their operations. The extraordinary potential of AI is restricted by insufficient medical data for AI model training and adversarial attacks wherein attackers perturb the dataset by adding some noise to it, which leads to the malfunctioning of the AI models, and a lack of trust caused by the opaque operational approach it employs. This Systematic Literature Review (SLR) is a state‐of‐the‐art survey of the research on blockchain technology for securing AI‐integrated healthcare applications. The most relevant articles from the Scopus and Web of Science (WoS) databases were identified using the PRISMA model. Most of the existing literature is about protecting the healthcare data used by AI‐based healthcare systems using blockchain technology, but the modality of data (text, images, audio, and sound) was not specifically mentioned. Information on protecting the training phase and model deployment for AI‐based healthcare systems considering the variations in feature extraction based on the modality of data was also not clearly specified. Hence, the three subfields of AI, namely, natural language processing (NLP), computer vision, and acoustic AI are further studied to identify security loopholes in its implementation pipeline. The three phases, namely the dataset, the training phase, and the trained models need to be protected from adversaries to avoid malfunctioning of the deployed AI models. The nature of the data processed by NLP, computer vision, and acoustic AI, underlying deep neural network (DNN) architectures, the complexity of attacks, and the perceivability of attacks by humans are analyzed to identify the need for security. A blockchain solution for AI‐based healthcare systems is synthesized based on the findings that have demonstrated the distinctive technological features of blockchains. It offers a solution for the privacy and security issues encountered by NLP, computer vision, and acoustic AI to boost the widespread adoption of AI applications in healthcare.
S2AC: Self-Supervised Attention Correlation Alignment Based on Mahalanobis Distance for Image Recognition

Susceptibility to domain changes for image classification hinders the application and development of deep neural networks. Domain adaptation (DA) makes use of domain-invariant characteristics to improve the performance of a model trained on labeled data from one domain (source domain) on an unlabeled domain (target) with a different data distribution. But existing DA methods simply use pretrained models (e.g., AlexNet, ResNet) for feature extraction, which are convolutional models that are trapped in localized features and fail to acquire long-distance dependencies. Furthermore, many approaches depend too much on pseudo-labels, which can impair adaptation efficiency and lead to unstable and inconsistent results. In this research, we present S2AC, a novel approach for unsupervised deep domain adaptation, that makes use of a stacked attention architecture as a feature map extractor. Our method can fuse domain discrepancy with minimizing a linear transformation of the second statistics (covariances) extended by the p-norm, while simultaneously designing pretext tasks on heuristics to improve the generality of the learning representation. In addition, we have developed a new trainable relative position embedding that not only reduces the model parameters but also enhances model accuracy and expedites the training process. To illustrate our method’s efficacy and controllability, we designed extensive experiments based on the Office31, Office_Caltech_10, and OfficeHome datasets. To the best of our knowledge, the proposed method is the first attempt at incorporating attention-based networks and self-supervised learning for image domain adaptation, and has shown promising results.
Deconfounded Visual Question Generation with Causal Inference

Visual Question Generation (VQG) task aims to generate meaningful and logically reasonable questions about the given image targeting an answer. Existing methods mainly focus on the visual concepts present in the image for question generation and have shown remarkable performance in VQG. However, these models frequently learn highly co-occurring object relationships and attributes, which is an inherent bias in question generation. This previously overlooked bias causes models to over-exploit the spurious correlations among visual features, the target answer, and the question. Therefore, they may generate inappropriate questions that contradict the visual content or facts. In this paper, we first introduce a causal perspective on VQG and adopt the causal graph to analyze spurious correlations among variables. Building on the analysis, we propose a Knowledge Enhanced Causal Visual Question Generation (KECVQG) model to mitigate the impact of spurious correlations in question generation. Specifically, an interventional visual feature extractor (IVE) is introduced in KECVQG, which aims to obtain unbiased visual features by disentangling. Then a knowledge-guided representation extractor (KRE) is employed to align unbiased features with external knowledge. Finally, the output features from KRE are sent into a standard transformer decoder to generate questions. Extensive experiments on the VQA v2.0 and OKVQA datasets show that KECVQG significantly outperforms existing models.
Answer-Based Entity Extraction and Alignment for Visual Text Question Answering

As a variant of visual question answering (VQA), visual text question answering (VTQA) provides a text-image pair for each question. Text utilizes named entities to describe corresponding image. Consequently, the ability to perform multi-hop reasoning using named entities between text and image becomes critically important. However, existing models pay relatively less attention to this aspect. Therefore, we propose Answer-Based Entity Extraction and Alignment Model (AEEA) to enable a comprehensive understanding and support multi-hop reasoning. The core of AEEA lies in two main components: AKECMR and answer aware predictor. The former emphasizes the alignment of modalities and effectively distinguishes between intra-modal and inter-modal information, and the latter prioritizes the full utilization of intrinsic semantic information contained in answers during training. Our model outperforms the baseline by 2.24% on test-dev set and 1.06% on test set, securing the third place in VTQA2023(English).
Experimentation-driven frameworks for AI technology adoption in libraries: Practical implementation in the library

This article presents a practical case study centered around the adoption of AI technology within a library setting through the lens of the Curiosity-Based Experimentation (CBE) Framework. It offers insights into the successful implementation of CBE by exploring a librarian's experimentation with a text-to-video generator AI tool. Motivated by the aim to enhance social media interactions and personal branding, the librarian creatively utilized the technology to craft novel videos. These videos, featuring university staff, students, and visitors, yielded engaging content for the library's online platforms. The article underscores the application of AI in libraries, illustrating how a CBE-driven approach can foster innovative solutions and amplify academic community engagement. Finally, some practical implications are provided to libraries which include resource allocation for experimentation, learning from failures, open communication, documentation, and interdisciplinary collaboration to foster innovation and enhancing library value.
A Lightweight Collective-attention Network for Change Detection

Change detection of multi-temporal remote sensing images is mushrooming with the innovations of neural networks, whose daunting challenge lies in locating sporadically distributed spatial-temporal changes given sophisticated scenes and various imaging conditions. Unfortunately, instead of devoting full attention to changes, most existing solutions often expend unnecessary resources yet derive task-irrelevant features. To relieve this issue, we propose a collective-attention network, which enjoys lightweight model architecture yet guarantees high performance. Specifically, an inter-temporal collective-attention module is developed for efficient interaction of bi-temporal features, in which a shared attention distribution is derived via the multiplication of temporal-concatenated queries and spatial-subtracted keys. Additionally, we present a non-change consistency-constraint, enforcing a change-oriented attention distribution and a noise-suppressed treatment. With the learned interaction features, bi-temporal differences are captured simply using the operations of spatial absolute error and temporal concatenation. Finally, decoding multi-scale differences is accomplished by lightweight temporal self-attention and spatial self-attention. Experiments on four datasets demonstrate that our model achieves state-of-the-art performance, yet requires only 1.71M parameters and 1.98G FLOPs.