OpenRaiser/Intern-Atlas huggingface dataset:
A Causal Inspired Early-Branching Structure for Domain Generalization
Venue: arXiv cs.CV 2024
Year: 2024

Learning domain-invariant semantic representations is crucial for achieving domain generalization (DG), where a model is required to perform well on unseen target domains. One critical challenge is that standard training often results in entangled semantic and domain-specific features. Previous works suggest formulating the problem from a causal perspective and solving the entanglement problem by enforcing marginal independence between the causal (\ie semantic) and non-causal (\ie domain-specific) features. Despite its simplicity, the basic marginal independent-based idea alone may be insufficient to identify the causal feature. By d-separation, we observe that the causal feature can be further characterized by being independent of the domain conditioned on the object, and we propose the following two strategies as complements for the basic framework. First, the observation implicitly implies that for the same object, the causal feature should not be associated with the non-causal feature, revealing that the common practice of obtaining the two features with a shared base feature extractor and two lightweight prediction heads might be inappropriate. To meet the constraint, we propose a simple early-branching structure, where the causal and non-causal feature obtaining branches share the first few blocks while diverging thereafter, for better structure design; Second, the observation implies that the causal feature remains invariant across different domains for the same object. To this end, we suggest that augmentation should be incorporated into the framework to better characterize the causal feature, and we further suggest an effective random domain sampling scheme to fulfill the task. Theoretical and experimental results show that the two strategies are beneficial for the basic marginal independent-based framework. Code is available at \url{https://github.com/liangchen527/CausEB}.
Physical Memory Attacks and a Memory Safe Management System for Memory Defense
Venue: arXiv cs.CR 2024
Year: 2024

Programming errors, defective hardware components (such as hard disk spindle defects), and environmental hazards can lead to invalid memory operations. In addition, less predictable forms of environmental stress, such as radiation, thermal influence, and energy fluctuations, can induce hardware faults. Sometimes, a soft error can occur instead of a complete failure, such as a bit-flip. The 'natural' factors that can cause bit-flips are replicable through targeted attacks that result in significant compromises, including full privileged system access. Existing physical defense solutions have consistently been circumvented shortly after deployment. We will explore the concept of a novel software-based low-level layer that can protect vulnerable memory targeted by physical attack vectors related to bit-flip vulnerabilities.
Digital Twin-assisted Reinforcement Learning for Resource-aware Microservice Offloading in Edge Computing
Venue: arXiv cs.NI 2024
Year: 2024

Collaborative edge computing (CEC) has emerged as a promising paradigm, enabling edge nodes to collaborate and execute microservices from end devices. Microservice offloading, a fundamentally important problem, decides when and where microservices are executed upon the arrival of services. However, the dynamic nature of the real-world CEC environment often leads to inefficient microservice offloading strategies, resulting in underutilized resources and network congestion. To address this challenge, we formulate an online joint microservice offloading and bandwidth allocation problem, JMOBA, to minimize the average completion time of services. In this paper, we introduce a novel microservice offloading algorithm, DTDRLMO, which leverages deep reinforcement learning (DRL) and digital twin technology. Specifically, we employ digital twin techniques to predict and adapt to changing edge node loads and network conditions of CEC in real-time. Furthermore, this approach enables the generation of an efficient offloading plan, selecting the most suitable edge node for each microservice. Simulation results on real-world and synthetic datasets demonstrate that DTDRLMO outperforms heuristic and learning-based methods in average service completion time.
Token Alignment via Character Matching for Subword Completion
Venue: arXiv cs.CL 2024
Year: 2024

Generative models, widely utilized in various applications, can often struggle with prompts corresponding to partial tokens. This struggle stems from tokenization, where partial tokens fall out of distribution during inference, leading to incorrect or nonsensical outputs. This paper examines a technique to alleviate the tokenization artifact on text completion in generative models, maintaining performance even in regular non-subword cases. The method, termed token alignment, involves backtracking to the last complete tokens and ensuring the model's generation aligns with the prompt. This approach showcases marked improvement across many partial token scenarios, including nuanced cases like space-prefix and partial indentation, with only a minor time increase. The technique and analysis detailed in this paper contribute to the continuous advancement of generative models in handling partial inputs, bearing relevance for applications like code completion and text autocompletion.
Improved Dynamics for the Maximum Common Subgraph Problem
Venue: arXiv cs.DM 2024
Year: 2024

The Maximum Common Subgraph (MCS) problem plays a crucial role across various domains, bridging theoretical exploration and practical applications in fields like bioinformatics and social network analysis. Despite its wide applicability, MCS is notoriously challenging and is classified as an NP-Complete (NPC) problem. This study introduces new heuristics aimed at mitigating these challenges through the reformulation of the MCS problem as the Maximum Clique and its complement, the Maximum Independent Set. Our first heuristic leverages the Motzkin-Straus theorem to reformulate the Maximum Clique Problem as a constrained optimization problem, continuing the work of Pelillo in Replicator Equations, Maximal Cliques, and Graph Isomorphism (1999) with replicator dynamics and introducing annealed imitation heuristics as in Dominant Sets and Hierarchical Clustering (Pavan and Pelillo, 2003) to improve chances of convergence to better local optima. The second technique applies heuristics drawn upon strategies for the Maximum Independent Set problem to efficiently reduce graph sizes as used by Akiwa and Iwata in 2014. This enables faster computation and, in many instances, yields near-optimal solutions. Furthermore we look at the implementation of both techniques in a single algorithm and find that it is a promising approach. Our techniques were tested on randomly generated Erdős-Rényi graph pairs. Results indicate the potential for application and substantial impact on future research directions.
Improved Trade-offs Between Amortization and Download Bandwidth for Linear HSS
Venue: arXiv cs.IT 2024
Year: 2024

A Homomorphic Secret Sharing (HSS) scheme is a secret-sharing scheme that shares a secret $x$ among $s$ servers, and additionally allows an output client to reconstruct some function $f(x)$ using information that can be locally computed by each server. A key parameter in HSS schemes is download rate, which quantifies how much information the output client needs to download from the servers. Often, download rate is improved by amortizing over $\ell$ instances of the problem, making $\ell$ also a key parameter of interest. Recent work (Fosli, Ishai, Kolobov, and Wootters 2022) established a limit on the download rate of linear HSS schemes for computing low-degree polynomials and constructed schemes that achieve this optimal download rate; their schemes required amortization over $\ell = Ω(s \log(s))$ instances of the problem. Subsequent work (Blackwell and Wootters, 2023) completely characterized linear HSS schemes that achieve optimal download rate in terms of a coding-theoretic notion termed optimal labelweight codes. A consequence of this characterization was that $\ell = Ω(s \log(s))$ is in fact necessary to achieve optimal download rate. In this paper, we characterize all linear HSS schemes, showing that schemes of any download rate are equivalent to a generalization of optimal labelweight codes. This equivalence is constructive and provides a way to obtain an explicit linear HSS scheme from any linear code. Using this characterization, we present explicit linear HSS schemes with slightly sub-optimal rate but with much improved amortization $\ell = O(s)$. Our constructions are based on algebraic geometry codes (specifically Hermitian codes and Goppa codes).
Improving Acoustic Word Embeddings through Correspondence Training of Self-supervised Speech Representations
Venue: arXiv cs.CL 2024
Year: 2024

Acoustic word embeddings (AWEs) are vector representations of spoken words. An effective method for obtaining AWEs is the Correspondence Auto-Encoder (CAE). In the past, the CAE method has been associated with traditional MFCC features. Representations obtained from self-supervised learning (SSL)-based speech models such as HuBERT, Wav2vec2, etc., are outperforming MFCC in many downstream tasks. However, they have not been well studied in the context of learning AWEs. This work explores the effectiveness of CAE with SSL-based speech representations to obtain improved AWEs. Additionally, the capabilities of SSL-based speech models are explored in cross-lingual scenarios for obtaining AWEs. Experiments are conducted on five languages: Polish, Portuguese, Spanish, French, and English. HuBERT-based CAE model achieves the best results for word discrimination in all languages, despite Hu-BERT being pre-trained on English only. Also, the HuBERT-based CAE model works well in cross-lingual settings. It outperforms MFCC-based CAE models trained on the target languages when trained on one source language and tested on target languages.
Faster Projected GAN: Towards Faster Few-Shot Image Generation
Venue: arXiv cs.CV 2024
Year: 2024

In order to solve the problems of long training time, large consumption of computing resources and huge parameter amount of GAN network in image generation, this paper proposes an improved GAN network model, which is named Faster Projected GAN, based on Projected GAN. The proposed network is mainly focuses on the improvement of generator of Projected GAN. By introducing depth separable convolution (DSC), the number of parameters of the Projected GAN is reduced, the training speed is accelerated, and memory is saved. Experimental results show that on ffhq-1k, art-painting, Landscape and other few-shot image datasets, a 20% speed increase and a 15% memory saving are achieved. At the same time, FID loss is less or no loss, and the amount of model parameters is better controlled. At the same time, significant training speed improvement has been achieved in the small sample image generation task of special scenes such as earthquake scenes with few public datasets.
5 Year Update to the Next Steps in Quantum Computing
Venue: arXiv cs.ET 2024
Year: 2024

It has been 5 years since the Computing Community Consortium (CCC) Workshop on Next Steps in Quantum Computing, and significant progress has been made in closing the gap between useful quantum algorithms and quantum hardware. Yet much remains to be done, in particular in terms of mitigating errors and moving towards error-corrected machines. As we begin to transition from the Noisy-Intermediate Scale Quantum (NISQ) era to a future of fault-tolerant machines, now is an opportune time to reflect on how to apply what we have learned thus far and what research needs to be done to realize computational advantage with quantum machines.
Procedural terrain generation with style transfer
Venue: arXiv cs.CV 2024
Year: 2024

In this study we introduce a new technique for the generation of terrain maps, exploiting a combination of procedural generation and Neural Style Transfer. We consider our approach to be a viable alternative to competing generative models, with our technique achieving greater versatility, lower hardware requirements and greater integration in the creative process of designers and developers. Our method involves generating procedural noise maps using either multi-layered smoothed Gaussian noise or the Perlin algorithm. We then employ an enhanced Neural Style transfer technique, drawing style from real-world height maps. This fusion of algorithmic generation and neural processing holds the potential to produce terrains that are not only diverse but also closely aligned with the morphological characteristics of real-world landscapes, with our process yielding consistent terrain structures with low computational cost and offering the capability to create customized maps. Numerical evaluations further validate our model's enhanced ability to accurately replicate terrain morphology, surpassing traditional procedural methods.
Automating SBOM Generation with Zero-Shot Semantic Similarity
Venue: arXiv cs.SE 2024
Year: 2024

It is becoming increasingly important in the software industry, especially with the growing complexity of software ecosystems and the emphasis on security and compliance for manufacturers to inventory software used on their systems. A Software-Bill-of-Materials (SBOM) is a comprehensive inventory detailing a software application's components and dependencies. Current approaches rely on case-based reasoning to inconsistently identify the software components embedded in binary files. We propose a different route, an automated method for generating SBOMs to prevent disastrous supply-chain attacks. Remaining on the topic of static code analysis, we interpret this problem as a semantic similarity task wherein a transformer model can be trained to relate a product name to corresponding version strings. Our test results are compelling, demonstrating the model's strong performance in the zero-shot classification task, further demonstrating the potential for use in a real-world cybersecurity context.
A Dataset for the Validation of Truth Inference Algorithms Suitable for Online Deployment
Venue: arXiv cs.HC 2024
Year: 2024

For the purpose of efficient and cost-effective large-scale data labeling, crowdsourcing is increasingly being utilized. To guarantee the quality of data labeling, multiple annotations need to be collected for each data sample, and truth inference algorithms have been developed to accurately infer the true labels. Despite previous studies having released public datasets to evaluate the efficacy of truth inference algorithms, these have typically focused on a single type of crowdsourcing task and neglected the temporal information associated with workers' annotation activities. These limitations significantly restrict the practical applicability of these algorithms, particularly in the context of long-term and online truth inference. In this paper, we introduce a substantial crowdsourcing annotation dataset collected from a real-world crowdsourcing platform. This dataset comprises approximately two thousand workers, one million tasks, and six million annotations. The data was gathered over a period of approximately six months from various types of tasks, and the timestamps of each annotation were preserved. We analyze the characteristics of the dataset from multiple perspectives and evaluate the effectiveness of several representative truth inference algorithms on this dataset. We anticipate that this dataset will stimulate future research on tracking workers' abilities over time in relation to different types of tasks, as well as enhancing online truth inference.
From "um" to "yeah": Producing, predicting, and regulating information flow in human conversation
Venue: arXiv cs.CL 2024
Year: 2024

Conversation demands attention. Speakers must call words to mind, listeners must make sense of them, and both together must negotiate this flow of information, all in fractions of a second. We used large language models to study how this works in a large-scale dataset of English-language conversation, the CANDOR corpus. We provide a new estimate of the information density of unstructured conversation, of approximately 13 bits/second, and find significant effects associated with the cognitive load of both retrieving, and presenting, that information. We also reveal a role for backchannels -- the brief yeahs, uh-huhs, and mhmms that listeners provide -- in regulating the production of novelty: the lead-up to a backchannel is associated with declining information rate, while speech downstream rebounds to previous rates. Our results provide new insights into long-standing theories of how we respond to fluctuating demands on cognitive resources, and how we negotiate those demands in partnership with others.
Envision3D: One Image to 3D with Anchor Views Interpolation
Venue: arXiv cs.CV 2024
Year: 2024

We present Envision3D, a novel method for efficiently generating high-quality 3D content from a single image. Recent methods that extract 3D content from multi-view images generated by diffusion models show great potential. However, it is still challenging for diffusion models to generate dense multi-view consistent images, which is crucial for the quality of 3D content extraction. To address this issue, we propose a novel cascade diffusion framework, which decomposes the challenging dense views generation task into two tractable stages, namely anchor views generation and anchor views interpolation. In the first stage, we train the image diffusion model to generate global consistent anchor views conditioning on image-normal pairs. Subsequently, leveraging our video diffusion model fine-tuned on consecutive multi-view images, we conduct interpolation on the previous anchor views to generate extra dense views. This framework yields dense, multi-view consistent images, providing comprehensive 3D information. To further enhance the overall generation quality, we introduce a coarse-to-fine sampling strategy for the reconstruction algorithm to robustly extract textured meshes from the generated dense images. Extensive experiments demonstrate that our method is capable of generating high-quality 3D content in terms of texture and geometry, surpassing previous image-to-3D baseline methods.
Unveiling the Truth: Exploring Human Gaze Patterns in Fake Images
Venue: arXiv cs.CV 2024
Year: 2024

Creating high-quality and realistic images is now possible thanks to the impressive advancements in image generation. A description in natural language of your desired output is all you need to obtain breathtaking results. However, as the use of generative models grows, so do concerns about the propagation of malicious content and misinformation. Consequently, the research community is actively working on the development of novel fake detection techniques, primarily focusing on low-level features and possible fingerprints left by generative models during the image generation process. In a different vein, in our work, we leverage human semantic knowledge to investigate the possibility of being included in frameworks of fake image detection. To achieve this, we collect a novel dataset of partially manipulated images using diffusion models and conduct an eye-tracking experiment to record the eye movements of different observers while viewing real and fake stimuli. A preliminary statistical analysis is conducted to explore the distinctive patterns in how humans perceive genuine and altered images. Statistical findings reveal that, when perceiving counterfeit samples, humans tend to focus on more confined regions of the image, in contrast to the more dispersed observational pattern observed when viewing genuine images. Our dataset is publicly available at: https://github.com/aimagelab/unveiling-the-truth.
Formalizing Date Arithmetic and Statically Detecting Ambiguities for the Law
Venue: arXiv cs.PL 2024
Year: 2024

Legal expert systems routinely rely on date computations to determine the eligibility of a citizen to social benefits or whether an application has been filed on time. Unfortunately, date arithmetic exhibits many corner cases, which are handled differently from one library to the other, making faithfully transcribing the law into code error-prone, and possibly leading to heavy financial and legal consequences for users. In this work, we aim to provide a solid foundation for date arithmetic working on days, months and years. We first present a novel, formal semantics for date computations, and formally establish several semantic properties through a mechanization in the F* proof assistant. Building upon this semantics, we then propose a static analysis by abstract interpretation to automatically detect ambiguities in date computations. We finally integrate our approach in the Catala language, a recent domain-specific language for formalizing computational law, and use it to analyze the Catala implementation of the French housing benefits, leading to the discovery of several date-related ambiguities.
LMStyle Benchmark: Evaluating Text Style Transfer for Chatbots
Venue: arXiv cs.CL 2024
Year: 2024

Since the breakthrough of ChatGPT, large language models (LLMs) have garnered significant attention in the research community. With the development of LLMs, the question of text style transfer for conversational models has emerged as a natural extension, where chatbots may possess their own styles or even characters. However, standard evaluation metrics have not yet been established for this new settings. This paper aims to address this issue by proposing the LMStyle Benchmark, a novel evaluation framework applicable to chat-style text style transfer (C-TST), that can measure the quality of style transfer for LLMs in an automated and scalable manner. In addition to conventional style strength metrics, LMStyle Benchmark further considers a novel aspect of metrics called appropriateness, a high-level metrics take account of coherence, fluency and other implicit factors without the aid of reference samples. Our experiments demonstrate that the new evaluation methods introduced by LMStyle Benchmark have a higher correlation with human judgments in terms of appropriateness. Based on LMStyle Benchmark, we present a comprehensive list of evaluation results for popular LLMs, including LLaMA, Alpaca, and Vicuna, reflecting their stylistic properties, such as formality and sentiment strength, along with their appropriateness.
AraTrust: An Evaluation of Trustworthiness for LLMs in Arabic
Venue: arXiv cs.CL 2024
Year: 2024

The swift progress and widespread acceptance of artificial intelligence (AI) systems highlight a pressing requirement to comprehend both the capabilities and potential risks associated with AI. Given the linguistic complexity, cultural richness, and underrepresented status of Arabic in AI research, there is a pressing need to focus on Large Language Models (LLMs) performance and safety for Arabic-related tasks. Despite some progress in their development, there is a lack of comprehensive trustworthiness evaluation benchmarks, which presents a major challenge in accurately assessing and improving the safety of LLMs when prompted in Arabic. In this paper, we introduce AraTrust, the first comprehensive trustworthiness benchmark for LLMs in Arabic. AraTrust comprises 522 human-written multiple-choice questions addressing diverse dimensions related to truthfulness, ethics, safety, physical health, mental health, unfairness, illegal activities, privacy, and offensive language. We evaluated a set of LLMs against our benchmark to assess their trustworthiness. GPT-4 was the most trustworthy LLM, while open-source models, particularly AceGPT 7B and Jais 13B, struggled to achieve a score of 60% in our benchmark.