OpenRaiser/Intern-Atlas huggingface dataset:
Joint Age and Coverage-Optimal Satellite Constellation Relaying in Cislunar Communications with Hybrid Orbits
Venue: arXiv cs.NI 2024
Year: 2024

With the ever-increasing lunar missions, a growing interest develops in designing data relay satellite constellations for cislunar communications, which is challenged by the constrained visibility and huge distance between the earth and moon in pursuit of establishing real-time communication links. In this work, therefore, we propose an age and coverage optimal relay satellite constellation for cislunar communication by considering the self-rotation of the earth as well as the orbital motion of the moon, which consists of hybrid Earth-Moon Libration 1/2 (EML1/L2) points Halo orbits, ordinary lunar orbits, and Geostationary Earth Orbit (GEO) satellites. In particular, by minimizing both the number of satellites and the average per-device Age of Information (AoI) while maximizing the coverage ratio of specific lunar surface regions, a multi-objective optimization problem is formulated and solved by using a well-designed Nondominated Sorting Genetic Algorithm-II (NSGA-II). The simulation results demonstrate that our proposed hybrid constellation significantly outperforms traditional Walker Star and Delta constellations in terms of both AoI and the coverage of communication.
Online-LoRA: Task-free Online Continual Learning via Low Rank Adaptation
Venue: arXiv cs.CV 2024
Year: 2024

Catastrophic forgetting is a significant challenge in online continual learning (OCL), especially for non-stationary data streams that do not have well-defined task boundaries. This challenge is exacerbated by the memory constraints and privacy concerns inherent in rehearsal buffers. To tackle catastrophic forgetting, in this paper, we introduce Online-LoRA, a novel framework for task-free OCL. Online-LoRA allows to finetune pre-trained Vision Transformer (ViT) models in real-time to address the limitations of rehearsal buffers and leverage pre-trained models' performance benefits. As the main contribution, our approach features a novel online weight regularization strategy to identify and consolidate important model parameters. Moreover, Online-LoRA leverages the training dynamics of loss values to enable the automatic recognition of the data distribution shifts. Extensive experiments across many task-free OCL scenarios and benchmark datasets (including CIFAR-100, ImageNet-R, ImageNet-S, CUB-200 and CORe50) demonstrate that Online-LoRA can be robustly adapted to various ViT architectures, while achieving better performance compared to SOTA methods. Our code will be publicly available at: https://github.com/Christina200/Online-LoRA-official.git.
Data-Driven Distributed Common Operational Picture from Heterogeneous Platforms using Multi-Agent Reinforcement Learning
Venue: arXiv cs.MA 2024
Year: 2024

The integration of unmanned platforms equipped with advanced sensors promises to enhance situational awareness and mitigate the "fog of war" in military operations. However, managing the vast influx of data from these platforms poses a significant challenge for Command and Control (C2) systems. This study presents a novel multi-agent learning framework to address this challenge. Our method enables autonomous and secure communication between agents and humans, which in turn enables real-time formation of an interpretable Common Operational Picture (COP). Each agent encodes its perceptions and actions into compact vectors, which are then transmitted, received and decoded to form a COP encompassing the current state of all agents (friendly and enemy) on the battlefield. Using Deep Reinforcement Learning (DRL), we jointly train COP models and agent's action selection policies. We demonstrate resilience to degraded conditions such as denied GPS and disrupted communications. Experimental validation is performed in the Starcraft-2 simulation environment to evaluate the precision of the COPs and robustness of policies. We report less than 5% error in COPs and policies resilient to various adversarial conditions. In summary, our contributions include a method for autonomous COP formation, increased resilience through distributed prediction, and joint training of COP models and multi-agent RL policies. This research advances adaptive and resilient C2, facilitating effective control of heterogeneous unmanned platforms.
Asterisk*: Keep it Simple
Venue: arXiv cs.CL 2024
Year: 2024

This paper describes Asterisk, a compact GPT-based model for generating text embeddings. The model uses a minimalist architecture with two layers, two attention heads, and 256 embedding dimensions. By applying knowledge distillation from larger pretrained models, we explore the trade-offs between model size and performance while minimizing computational and memory requirements. The model is primarily evaluated and optimized for classification tasks, with experimental results showing its moderate performance in zero-shot classification across various downstream applications. With additional configuration, the model performance can approach or even surpass that of larger architectures on specific classification tasks.
Image2Text2Image: A Novel Framework for Label-Free Evaluation of Image-to-Text Generation with Text-to-Image Diffusion Models
Venue: arXiv cs.CV 2024
Year: 2024

Evaluating the quality of automatically generated image descriptions is a complex task that requires metrics capturing various dimensions, such as grammaticality, coverage, accuracy, and truthfulness. Although human evaluation provides valuable insights, its cost and time-consuming nature pose limitations. Existing automated metrics like BLEU, ROUGE, METEOR, and CIDEr attempt to fill this gap, but they often exhibit weak correlations with human judgment. To address this challenge, we propose a novel evaluation framework called Image2Text2Image, which leverages diffusion models, such as Stable Diffusion or DALL-E, for text-to-image generation. In the Image2Text2Image framework, an input image is first processed by a selected image captioning model, chosen for evaluation, to generate a textual description. Using this generated description, a diffusion model then creates a new image. By comparing features extracted from the original and generated images, we measure their similarity using a designated similarity metric. A high similarity score suggests that the model has produced a faithful textual description, while a low score highlights discrepancies, revealing potential weaknesses in the model's performance. Notably, our framework does not rely on human-annotated reference captions, making it a valuable tool for assessing image captioning models. Extensive experiments and human evaluations validate the efficacy of our proposed Image2Text2Image evaluation framework. The code and dataset will be published to support further research in the community.
WavShadow: Wavelet Based Shadow Segmentation and Removal
Venue: arXiv cs.CV 2024
Year: 2024

Shadow removal and segmentation remain challenging tasks in computer vision, particularly in complex real world scenarios. This study presents a novel approach that enhances the ShadowFormer model by incorporating Masked Autoencoder (MAE) priors and Fast Fourier Convolution (FFC) blocks, leading to significantly faster convergence and improved performance. We introduce key innovations: (1) integration of MAE priors trained on Places2 dataset for better context understanding, (2) adoption of Haar wavelet features for enhanced edge detection and multiscale analysis, and (3) implementation of a modified SAM Adapter for robust shadow segmentation. Extensive experiments on the challenging DESOBA dataset demonstrate that our approach achieves state of the art results, with notable improvements in both convergence speed and shadow removal quality.
On Differentially Private String Distances
Venue: arXiv cs.DS 2024
Year: 2024

Given a database of bit strings $A_1,\ldots,A_m\in \{0,1\}^n$, a fundamental data structure task is to estimate the distances between a given query $B\in \{0,1\}^n$ with all the strings in the database. In addition, one might further want to ensure the integrity of the database by releasing these distance statistics in a secure manner. In this work, we propose differentially private (DP) data structures for this type of tasks, with a focus on Hamming and edit distance. On top of the strong privacy guarantees, our data structures are also time- and space-efficient. In particular, our data structure is $ε$-DP against any sequence of queries of arbitrary length, and for any query $B$ such that the maximum distance to any string in the database is at most $k$, we output $m$ distance estimates. Moreover, - For Hamming distance, our data structure answers any query in $\widetilde O(mk+n)$ time and each estimate deviates from the true distance by at most $\widetilde O(k/e^{ε/\log k})$; - For edit distance, our data structure answers any query in $\widetilde O(mk^2+n)$ time and each estimate deviates from the true distance by at most $\widetilde O(k/e^{ε/(\log k \log n)})$. For moderate $k$, both data structures support sublinear query operations. We obtain these results via a novel adaptation of the randomized response technique as a bit flipping procedure, applied to the sketched strings.
Fact or Fiction? Can LLMs be Reliable Annotators for Political Truths?
Venue: arXiv cs.CL 2024
Year: 2024

Political misinformation poses significant challenges to democratic processes, shaping public opinion and trust in media. Manual fact-checking methods face issues of scalability and annotator bias, while machine learning models require large, costly labelled datasets. This study investigates the use of state-of-the-art large language models (LLMs) as reliable annotators for detecting political factuality in news articles. Using open-source LLMs, we create a politically diverse dataset, labelled for bias through LLM-generated annotations. These annotations are validated by human experts and further evaluated by LLM-based judges to assess the accuracy and reliability of the annotations. Our approach offers a scalable and robust alternative to traditional fact-checking, enhancing transparency and public trust in media.
Quantitative Assessment of Intersectional Empathetic Bias and Understanding
Venue: arXiv cs.CL 2024
Year: 2024

A growing amount of literature critiques the current operationalizations of empathy based on loose definitions of the construct. Such definitions negatively affect dataset quality, model robustness, and evaluation reliability. We propose an empathy evaluation framework that operationalizes empathy close to its psychological origins. The framework measures the variance in responses of LLMs to prompts using existing metrics for empathy and emotional valence. The variance is introduced through the controlled generation of the prompts by varying social biases affecting context understanding, thus impacting empathetic understanding. The control over generation ensures high theoretical validity of the constructs in the prompt dataset. Also, it makes high-quality translation, especially into languages that currently have little-to-no way of evaluating empathy or bias, such as the Slavonic family, more manageable. Using chosen LLMs and various prompt types, we demonstrate the empathy evaluation with the framework, including multiple-choice answers and free generation. The variance in our initial evaluation sample is small and we were unable to measure convincing differences between the empathetic understanding in contexts given by different social groups. However, the results are promising because the models showed significant alterations their reasoning chains needed to capture the relatively subtle changes in the prompts. This provides the basis for future research into the construction of the evaluation sample and statistical methods for measuring the results.
Beyond Correlation: Evaluating Multimedia Quality Models with the Constrained Concordance Index
Venue: arXiv cs.MM 2024
Year: 2024

This study investigates the evaluation of multimedia quality models, focusing on the inherent uncertainties in subjective Mean Opinion Score (MOS) ratings due to factors like rater inconsistency and bias. Traditional statistical measures such as Pearson's Correlation Coefficient (PCC), Spearman's Rank Correlation Coefficient (SRCC), and Kendall's Tau (KTAU) often fail to account for these uncertainties, leading to inaccuracies in model performance assessment. We introduce the Constrained Concordance Index (CCI), a novel metric designed to overcome the limitations of existing metrics by considering the statistical significance of MOS differences and excluding comparisons where MOS confidence intervals overlap. Through comprehensive experiments across various domains including speech and image quality assessment, we demonstrate that CCI provides a more robust and accurate evaluation of instrumental quality models, especially in scenarios of low sample sizes, rater group variability, and restriction of range. Our findings suggest that incorporating rater subjectivity and focusing on statistically significant pairs can significantly enhance the evaluation framework for multimedia quality prediction models. This work not only sheds light on the overlooked aspects of subjective rating uncertainties but also proposes a methodological advancement for more reliable and accurate quality model evaluation.
Exploring the Reductions Between SSP-NP-complete Problems and Developing a Compendium Website Displaying the Results
Venue: arXiv cs.CC 2024
Year: 2024

SSP reductions are a type of polynomial reductions that also preserve the solutions of the instances. This means there is a mapping from each solution in the original instance to one in the reduced instance, allowing direct deduction of an original solution from a solution in the reduced instance. SSP reductions can be used to show SSP-NP completeness of a problem, which is interesting because it has been proven that two min-max variants of SSP-NP complete problems are $Σ_2^p$-complete. Min-max optimization problems are optimization problems with the objective of minimizing the maximum outcome. An example is a power network prone to attack by terrorists. Lets say the government wants to minimize the damage that can be achieved by the terrorists, e.g. by protecting the electricity poles which were found to be most important for distributing power. Most theoretical computer scientists assume min-max optimization problems to generally be $Σ_2^p$-complete, and with the help of the SSP-framework, this was shown for network interdiction and min-max regret robust optimization. This paper is devoted to the exploration of SSP reductions and the collection of SSP-NP-complete problems as well as SSP reductions. Additionally, a compendium website is developed, in which the SSP-NP-complete problems and SSP reductions are displayed in a graph, so that it becomes easy to understand the relations between the problems. The website is also extendable to include problems from other complexity classes and reductions that are not SSP reductions, such as gap-preserving reductions, fixed parameter tractable reductions or fine-grained reductions. Overall, 19 new SSP reductions are found, resulting in eight new SSP-NP-completeness proofs, and the compendium website is developed to enable theoretical computer scientists to easily access the results and provide a knowledge base for future research.
Reliability-Based Design Optimization Incorporating Extended Optimal Uncertainty Quantification
Venue: arXiv cs.CE 2024
Year: 2024

Reliability-based design optimization (RBDO) approaches aim to identify the best design of an engineering problem, whilst the probability of failure (PoF) remains below an acceptable value. Thus, the incorporation of the sharpest bounds on the PoF under given constraints on the uncertain input quantities strongly strenghtens the significance of RBDO results, since unjustified assumptions on the input quantities are avoided. In this contribution, the extended Optimal Uncertainty Quantification framework is embedded within an RBDO context in terms of a double loop approach. By that, the mathematically sharpest bounds on the PoF as well as on the cost function can be computed for all design candidates and compared with acceptable values. The extended OUQ allows the incorporation of aleatory as well as epistemic uncertainties, where the definition of probability density functions is not necessarily required and just given data on the input can be included. Specifically, not only bounds on the values themselves, but also bounds on moment constraints can be taken into account. Thus, inadmissible assumptions on the data can be avoided, while the optimal design of a problem can be identified. The capability of the resulting framework is firstly shown by means of a benchmark problem under the influence of polymorphic uncertainties. Afterwards, a realistic engineering problem is analyzed, where the positioning of laser-hardened lines within a steel sheet for a car crash structure are optimized.
Financial Fraud Detection using Jump-Attentive Graph Neural Networks
Venue: arXiv cs.LG 2024
Year: 2024

As the availability of financial services online continues to grow, the incidence of fraud has surged correspondingly. Fraudsters continually seek new and innovative ways to circumvent the detection algorithms in place. Traditionally, fraud detection relied on rule-based methods, where rules were manually created based on transaction data features. However, these techniques soon became ineffective due to their reliance on manual rule creation and their inability to detect complex data patterns. Today, a significant portion of the financial services sector employs various machine learning algorithms, such as XGBoost, Random Forest, and neural networks, to model transaction data. While these techniques have proven more efficient than rule-based methods, they still fail to capture interactions between different transactions and their interrelationships. Recently, graph-based techniques have been adopted for financial fraud detection, leveraging graph topology to aggregate neighborhood information of transaction data using Graph Neural Networks (GNNs). Despite showing improvements over previous methods, these techniques still struggle to keep pace with the evolving camouflaging tactics of fraudsters and suffer from information loss due to over-smoothing. In this paper, we propose a novel algorithm that employs an efficient neighborhood sampling method, effective for camouflage detection and preserving crucial feature information from non-similar nodes. Additionally, we introduce a novel GNN architecture that utilizes attention mechanisms and preserves holistic neighborhood information to prevent information loss. We test our algorithm on financial data to show that our method outperforms other state-of-the-art graph algorithms.
Autonomous Industrial Control using an Agentic Framework with Large Language Models
Venue: arXiv cs.MA 2024
Year: 2024

As chemical plants evolve towards full autonomy, the need for effective fault handling and control in dynamic, unpredictable environments becomes increasingly critical. This paper proposes an innovative approach to industrial automation, introducing validation and reprompting architectures utilizing large language model (LLM)-based autonomous control agents. The proposed agentic system, comprising of operator, validator, and reprompter agents, enables autonomous management of control tasks, adapting to unforeseen disturbances without human intervention. By utilizing validation and reprompting architectures, the framework allows agents to recover from errors and continuously improve decision-making in real-time industrial scenarios. We hypothesize that this mechanism will enhance performance and reliability across a variety of LLMs, offering a path toward fully autonomous systems capable of handling unexpected challenges, paving the way for robust, adaptive control in complex industrial environments. To demonstrate the concept's effectiveness, we created a simple case study involving a temperature control experiment embedded on a microcontroller device, validating the proposed approach.
Diamond open access and open infrastructures have shaped the Canadian scholarly journal landscape since the start of the digital era
Venue: arXiv cs.DL 2024
Year: 2024

Scholarly publishing involves multiple stakeholders having various types of interest. In Canada, the implication of universities, the presence of societies and the availability of governmental support for periodicals seem to have contributed to a rather diverse ecosystem of journals. This study presents in detail the current state of these journals, in addition to past trends and transformations during the 20th century and, in particular, the digital era. To this effect, we created a new dataset, including a total of 1256 journals, 944 of which appeared to be active today, specifically focusing on the supporting organizations behind the journals, the types of (open) access, disciplines, geographic origins, languages of publication and hosting platforms and tools. The main overarching traits across Canadian scholarly journals are an important presence of Diamond open access, which has been adopted by 62% of the journals, a predominance of the Social Sciences and Humanities disciplines and a scarce presence of the major commercial publishers. The digital era allowed for the development of open infrastructures, which contributed to the creation of a new generation of journals that massively adopted Diamond open access, often supported by university libraries. However, journal cessation also increased, especially among the recently founded journals. These results provide valuable insights for the design of tailored practices and policies that cater to the needs of different types of periodicals and that take into account the evolving practices across the Canadian scholarly journal landscape.
The Empirical Impact of Data Sanitization on Language Models
Venue: arXiv cs.CL 2024
Year: 2024

Data sanitization in the context of language modeling involves identifying sensitive content, such as personally identifiable information (PII), and redacting them from a dataset corpus. It is a common practice used in natural language processing (NLP) to maintain privacy. Nevertheless, the impact of data sanitization on the language understanding capability of a language model remains less studied. This paper empirically analyzes the effects of data sanitization across several benchmark language-modeling tasks including comprehension question answering (Q&A), entailment, sentiment analysis, and text classification. Our experiments cover a wide spectrum comprising finetuning small-scale language models, to prompting large language models (LLMs), on both original and sanitized datasets, and comparing their performance across the tasks. Interestingly, our results suggest that for some tasks such as sentiment analysis or entailment, the impact of redaction is quite low, typically around 1-5%, while for tasks such as comprehension Q&A there is a big drop of >25% in performance observed in redacted queries as compared to the original. For tasks that have a higher impact, we perform a deeper dive to inspect the presence of task-critical entities. Finally, we investigate correlation between performance and number of redacted entities, and also suggest a strategy to repair an already redacted dataset by means of content-based subsampling. Additional details are available at https://sites.google.com/view/datasan.