OpenRaiser/Intern-Atlas huggingface dataset:
Automatic Wood Pith Detector: Local Orientation Estimation and Robust Accumulation
Venue: arXiv cs.CV 2024
Year: 2024

A fully automated technique for wood pith detection (APD), relying on the concentric shape of the structure of wood ring slices, is introduced. The method estimates the ring's local orientations using the 2D structure tensor and finds the pith position, optimizing a cost function designed for this problem. We also present a variant (APD-PCL), using the parallel coordinates space, that enhances the method's effectiveness when there are no clear tree ring patterns. Furthermore, refining previous work by Kurdthongmee, a YoloV8 net is trained for pith detection, producing a deep learning-based approach to the same problem (APD-DL). All methods were tested on seven datasets, including images captured under diverse conditions (controlled laboratory settings, sawmill, and forest) and featuring various tree species (Pinus taeda, Douglas fir, Abies alba, and Gleditsia triacanthos). All proposed approaches outperform existing state-of-the-art methods and can be used in CPU-based real-time applications. Additionally, we provide a novel dataset comprising images of gymnosperm and angiosperm species. Dataset and source code are available at http://github.com/hmarichal93/apd.
Bi-LORA: A Vision-Language Approach for Synthetic Image Detection
Venue: arXiv cs.CV 2024
Year: 2024

Advancements in deep image synthesis techniques, such as generative adversarial networks (GANs) and diffusion models (DMs), have ushered in an era of generating highly realistic images. While this technological progress has captured significant interest, it has also raised concerns about the potential difficulty in distinguishing real images from their synthetic counterparts. This paper takes inspiration from the potent convergence capabilities between vision and language, coupled with the zero-shot nature of vision-language models (VLMs). We introduce an innovative method called Bi-LORA that leverages VLMs, combined with low-rank adaptation (LORA) tuning techniques, to enhance the precision of synthetic image detection for unseen model-generated images. The pivotal conceptual shift in our methodology revolves around reframing binary classification as an image captioning task, leveraging the distinctive capabilities of cutting-edge VLM, notably bootstrapping language image pre-training (BLIP2). Rigorous and comprehensive experiments are conducted to validate the effectiveness of our proposed approach, particularly in detecting unseen diffusion-generated images from unknown diffusion-based generative models during training, showcasing robustness to noise, and demonstrating generalization capabilities to GANs. The obtained results showcase an impressive average accuracy of 93.41% in synthetic image detection on unseen generation models. The code and models associated with this research can be publicly accessed at https://github.com/Mamadou-Keita/VLM-DETECT.
CAM-Based Methods Can See through Walls
Venue: arXiv cs.CV 2024
Year: 2024

CAM-based methods are widely-used post-hoc interpretability method that produce a saliency map to explain the decision of an image classification model. The saliency map highlights the important areas of the image relevant to the prediction. In this paper, we show that most of these methods can incorrectly attribute an important score to parts of the image that the model cannot see. We show that this phenomenon occurs both theoretically and experimentally. On the theory side, we analyze the behavior of GradCAM on a simple masked CNN model at initialization. Experimentally, we train a VGG-like model constrained to not use the lower part of the image and nevertheless observe positive scores in the unseen part of the image. This behavior is evaluated quantitatively on two new datasets. We believe that this is problematic, potentially leading to mis-interpretation of the model's behavior.
Zero-Shot Multi-Lingual Speaker Verification in Clinical Trials
Venue: arXiv cs.LG 2024
Year: 2024

Due to the substantial number of clinicians, patients, and data collection environments involved in clinical trials, gathering data of superior quality poses a significant challenge. In clinical trials, patients are assessed based on their speech data to detect and monitor cognitive and mental health disorders. We propose using these speech recordings to verify the identities of enrolled patients and identify and exclude the individuals who try to enroll multiple times in the same trial. Since clinical studies are often conducted across different countries, creating a system that can perform speaker verification in diverse languages without additional development effort is imperative. We evaluate pre-trained TitaNet, ECAPA-TDNN, and SpeakerNet models by enrolling and testing with speech-impaired patients speaking English, German, Danish, Spanish, and Arabic languages. Our results demonstrate that tested models can effectively generalize to clinical speakers, with less than 2.7% EER for European Languages and 8.26% EER for Arabic. This represents a significant step in developing more versatile and efficient speaker verification systems for cognitive and mental health clinical trials that can be used across a wide range of languages and dialects, substantially reducing the effort required to develop speaker verification systems for multiple languages. We also evaluate how speech tasks and number of speakers involved in the trial influence the performance and show that the type of speech tasks impacts the model performance.
AUTODIFF: Autoregressive Diffusion Modeling for Structure-based Drug Design
Venue: arXiv cs.LG 2024
Year: 2024

Structure-based drug design (SBDD), which aims to generate molecules that can bind tightly to the target protein, is an essential problem in drug discovery, and previous approaches have achieved initial success. However, most existing methods still suffer from invalid local structure or unrealistic conformation issues, which are mainly due to the poor leaning of bond angles or torsional angles. To alleviate these problems, we propose AUTODIFF, a diffusion-based fragment-wise autoregressive generation model. Specifically, we design a novel molecule assembly strategy named conformal motif that preserves the conformation of local structures of molecules first, then we encode the interaction of the protein-ligand complex with an SE(3)-equivariant convolutional network and generate molecules motif-by-motif with diffusion modeling. In addition, we also improve the evaluation framework of SBDD by constraining the molecular weights of the generated molecules in the same range, together with some new metrics, which make the evaluation more fair and practical. Extensive experiments on CrossDocked2020 demonstrate that our approach outperforms the existing models in generating realistic molecules with valid structures and conformations while maintaining high binding affinity.
Improving Retrieval Augmented Open-Domain Question-Answering with Vectorized Contexts
Venue: arXiv cs.CL 2024
Year: 2024

In the era of large language models, applying techniques such as Retrieval Augmented Generation can better address Open-Domain Question-Answering problems. Due to constraints including model sizes and computing resources, the length of context is often limited, and it becomes challenging to empower the model to cover overlong contexts while answering questions from open domains. This paper proposes a general and convenient method to covering longer contexts in Open-Domain Question-Answering tasks. It leverages a small encoder language model that effectively encodes contexts, and the encoding applies cross-attention with origin inputs. With our method, the origin language models can cover several times longer contexts while keeping the computing requirements close to the baseline. Our experiments demonstrate that after fine-tuning, there is improved performance across two held-in datasets, four held-out datasets, and also in two In Context Learning settings.
LastResort at SemEval-2024 Task 3: Exploring Multimodal Emotion Cause Pair Extraction as Sequence Labelling Task
Venue: arXiv cs.CL 2024
Year: 2024

Conversation is the most natural form of human communication, where each utterance can range over a variety of possible emotions. While significant work has been done towards the detection of emotions in text, relatively little work has been done towards finding the cause of the said emotions, especially in multimodal settings. SemEval 2024 introduces the task of Multimodal Emotion Cause Analysis in Conversations, which aims to extract emotions reflected in individual utterances in a conversation involving multiple modalities (textual, audio, and visual modalities) along with the corresponding utterances that were the cause for the emotion. In this paper, we propose models that tackle this task as an utterance labeling and a sequence labeling problem and perform a comparative study of these models, involving baselines using different encoders, using BiLSTM for adding contextual information of the conversation, and finally adding a CRF layer to try to model the inter-dependencies between adjacent utterances more effectively. In the official leaderboard for the task, our architecture was ranked 8th, achieving an F1-score of 0.1759 on the leaderboard.
The Effects of Group Sanctions on Participation and Toxicity: Quasi-experimental Evidence from the Fediverse
Venue: arXiv cs.SI 2024
Year: 2024

Online communities often overlap and coexist, despite incongruent norms and approaches to content moderation. When communities diverge, decentralized and federated communities may pursue group-level sanctions, including defederation (disconnection) to block communication between members of specific communities. We investigate the effects of defederation in the context of the Fediverse, a set of decentralized, interconnected social networks with independent governance. Mastodon and Pleroma, the most popular software powering the Fediverse, allow administrators on one server to defederate from another. We use a difference-in-differences approach and matched controls to estimate the effects of defederation events on participation and message toxicity among affected members of the blocked and blocking servers. We find that defederation causes a drop in activity for accounts on the blocked servers, but not on the blocking servers. Also, we find no evidence of an effect of defederation on message toxicity.
Exploring Automated Distractor Generation for Math Multiple-choice Questions via Large Language Models
Venue: arXiv cs.CL 2024
Year: 2024

Multiple-choice questions (MCQs) are ubiquitous in almost all levels of education since they are easy to administer, grade, and are a reliable format in assessments and practices. One of the most important aspects of MCQs is the distractors, i.e., incorrect options that are designed to target common errors or misconceptions among real students. To date, the task of crafting high-quality distractors largely remains a labor and time-intensive process for teachers and learning content designers, which has limited scalability. In this work, we study the task of automated distractor generation in the domain of math MCQs and explore a wide variety of large language model (LLM)-based approaches, from in-context learning to fine-tuning. We conduct extensive experiments using a real-world math MCQ dataset and find that although LLMs can generate some mathematically valid distractors, they are less adept at anticipating common errors or misconceptions among real students.
Enhancing Ship Classification in Optical Satellite Imagery: Integrating Convolutional Block Attention Module with ResNet for Improved Performance
Venue: arXiv cs.CV 2024
Year: 2024

In this study, we present an advanced convolutional neural network (CNN) architecture for ship classification based on optical satellite imagery, which significantly enhances performance through the integration of a convolutional block attention module (CBAM) and additional architectural innovations. Building upon the foundational ResNet50 model, we first incorporated a standard CBAM to direct the model's focus toward more informative features, achieving an accuracy of 87% compared to 85% of the baseline ResNet50. Further augmentations involved multiscale feature integration, depthwise separable convolutions, and dilated convolutions, culminating in an enhanced ResNet model with improved CBAM. This model demonstrated a remarkable accuracy of 95%, with precision, recall, and F1 scores all witnessing substantial improvements across various ship classes. In particular, the bulk carrier and oil tanker classes exhibited nearly perfect precision and recall rates, underscoring the enhanced capability of the model to accurately identify and classify ships. Attention heatmap analyses further validated the efficacy of the improved model, revealing more focused attention on relevant ship features regardless of background complexities. These findings underscore the potential of integrating attention mechanisms and architectural innovations into CNNs for high-resolution satellite imagery classification. This study navigates through the class imbalance and computational costs and proposes future directions for scalability and adaptability in new or rare ship-type recognition. This study lays the groundwork for applying advanced deep learning techniques in remote sensing, offering insights into scalable and efficient satellite image classification.
Fairness-aware Age-of-Information Minimization in WPT-Assisted Short-Packet Data Collection for mURLLC
Venue: arXiv cs.IT 2024
Year: 2024

The technological landscape is rapidly evolving toward large-scale systems. Networks supporting massive connectivity through numerous Internet of Things (IoT) devices are at the forefront of this advancement. In this paper, we examine Wireless Power Transfer (WPT)-enabled networks, where a server requires to collect data from these IoT devices to compute a task with massive Ultra-Reliable and Low-Latency Communication (mURLLC) services.} We focus on information freshness, using Age-of-Information (AoI) as the key performance metric. Specifically, we aim to minimize the maximum AoI among IoT devices by optimizing the scheduling policy. Our analytical findings demonstrate the convexity of the problem, enabling efficient solutions. We introduce the concept of AoI-oriented cluster capacity and analyze the relationship between the number of supported devices and network AoI performance. Numerical simulations validate our proposed approach's effectiveness in enhancing AoI performance, highlighting its potential for guiding the design of future IoT systems requiring mURLLC services.
Trainable Least Squares to Reduce PAPR in OFDM-based Hybrid Beamforming Systems
Venue: arXiv cs.OH 2024
Year: 2024

In this paper, we propose a trainable least squares (LS) approach for reducing the peak-to-average power ratio (PAPR) of orthogonal frequency division multiplexing (OFDM) signals in a hybrid beamforming (HBF) system. Compared to digital beamforming (DBF), in HBF technology the number of antennas exceeds the number of digital ports. Therefore, PAPR reduction capabilities are restricted by both a limited bandwidth and the reduced size of digital space. The problem is to meet both conditions. Moreover, the major HBF advantage is a reduced system complexity, thus the complexity of the PAPR reduction algorithm is expected to be low. To justify the performance of the proposed trainable LS, we provide a performance bound achieved by convex optimization using the CVX Matlab package. Moreover, the complexity of the proposed algorithm can be comparable to the minimal complexity of the digital ``twin'' calculation in HBF. The abovementioned features prove the feasibility of the trained LS for PAPR reduction in fully-connected HBF.
LoS Sensing-based Channel Estimation in UAV-Assisted OFDM Systems
Venue: arXiv cs.OH 2024
Year: 2024

In unmanned aerial vehicle (UAV)-assisted orthogonal frequency division multiplexing (OFDM) systems, the potential advantage of the line-of-sight (LoS) path, characterized by its high probability of existence, has not been fully harnessed, thereby impeding the improvement of channel estimation (CE) accuracy. Inspired by the ideas of integrated sensing and communication (ISAC), this letter develops a LoS sensing method aimed at detecting the presence of LoS path. Leveraging the prior information obtained from LoS path detection, the detection thresholds for resolvable paths are proposed for LoS and Non-LoS (NLoS) scenarios, respectively. By employing these specifically designed detection thresholds, denoising processing is applied to classical least square (LS) CE, thereby improving the CE accuracy. Simulation results validate the effectiveness of the proposed method in enhancing CE accuracy and demonstrate its robustness against parameter variations.
A remark on conditional entropy
Venue: arXiv cs.IT 2024
Year: 2024

The following note proves that conditional entropy of a sequence is almost time-reversal invariant, specifically they only differ by a small constant factor dependent only upon the forward and backward models that the entropies are being calculated with respect to. This gives rise to a numerical value that quantifies learnability, as well as a methodology to control for distributional shift between datasets. Rough guidelines are given for practitioners.
Remote sensing framework for geological mapping via stacked autoencoders and clustering
Venue: arXiv cs.LG 2024
Year: 2024

Supervised machine learning methods for geological mapping via remote sensing face limitations due to the scarcity of accurately labelled training data that can be addressed by unsupervised learning, such as dimensionality reduction and clustering. Dimensionality reduction methods have the potential to play a crucial role in improving the accuracy of geological maps. Although conventional dimensionality reduction methods may struggle with nonlinear data, unsupervised deep learning models such as autoencoders can model non-linear relationships. Stacked autoencoders feature multiple interconnected layers to capture hierarchical data representations useful for remote sensing data. We present an unsupervised machine learning-based framework for processing remote sensing data using stacked autoencoders for dimensionality reduction and k-means clustering for mapping geological units. We use Landsat 8, ASTER, and Sentinel-2 datasets to evaluate the framework for geological mapping of the Mutawintji region in Western New South Wales, Australia. We also compare stacked autoencoders with principal component analysis (PCA) and canonical autoencoders. Our results reveal that the framework produces accurate and interpretable geological maps, efficiently discriminating rock units. The results reveal that the combination of stacked autoencoders with Sentinel-2 data yields the best performance accuracy when compared to other combinations. We find that stacked autoencoders enable better extraction of complex and hierarchical representations of the input data when compared to canonical autoencoders and PCA. We also find that the generated maps align with prior geological knowledge of the study area while providing novel insights into geological structures.
A Holistic Indicator of Polarization to Measure Online Sexism
Venue: arXiv cs.SI 2024
Year: 2024

The online trend of the manosphere and feminist discourse on social networks requires a holistic measure of the level of sexism in an online community. This indicator is important for policymakers and moderators of online communities (e.g., subreddits) and computational social scientists, either to revise moderation strategies based on the degree of sexism or to match and compare the temporal sexism across different platforms and communities with real-time events and infer social scientific insights. In this paper, we build a model that can provide a comparable holistic indicator of toxicity targeted toward male and female identity and male and female individuals. Despite previous supervised NLP methods that require annotation of toxic comments at the target level (e.g. annotating comments that are specifically toxic toward women) to detect targeted toxic comments, our indicator uses supervised NLP to detect the presence of toxicity and unsupervised word embedding association test to detect the target automatically. We apply our model to gender discourse communities (e.g., r/TheRedPill, r/MGTOW, r/FemaleDatingStrategy) to detect the level of toxicity toward genders (i.e., sexism). Our results show that our framework accurately and consistently (93% correlation) measures the level of sexism in a community. We finally discuss how our framework can be generalized in the future to measure qualities other than toxicity (e.g. sentiment, humor) toward general-purpose targets and turn into an indicator of different sorts of polarizations.
Is Exploration All You Need? Effective Exploration Characteristics for Transfer in Reinforcement Learning
Venue: arXiv cs.LG 2024
Year: 2024

In deep reinforcement learning (RL) research, there has been a concerted effort to design more efficient and productive exploration methods while solving sparse-reward problems. These exploration methods often share common principles (e.g., improving diversity) and implementation details (e.g., intrinsic reward). Prior work found that non-stationary Markov decision processes (MDPs) require exploration to efficiently adapt to changes in the environment with online transfer learning. However, the relationship between specific exploration characteristics and effective transfer learning in deep RL has not been characterized. In this work, we seek to understand the relationships between salient exploration characteristics and improved performance and efficiency in transfer learning. We test eleven popular exploration algorithms on a variety of transfer types -- or ``novelties'' -- to identify the characteristics that positively affect online transfer learning. Our analysis shows that some characteristics correlate with improved performance and efficiency across a wide range of transfer tasks, while others only improve transfer performance with respect to specific environment changes. From our analysis, make recommendations about which exploration algorithm characteristics are best suited to specific transfer situations.