OpenRaiser/Intern-Atlas huggingface dataset:
A Joint Learning Model with Variational Interaction for Multilingual Program Translation
Venue: arXiv cs.SE 2024
Year: 2024

Programs implemented in various programming languages form the foundation of software applications. To alleviate the burden of program migration and facilitate the development of software systems, automated program translation across languages has garnered significant attention. Previous approaches primarily focus on pairwise translation paradigms, learning translation between pairs of languages using bilingual parallel data. However, parallel data is difficult to collect for some language pairs, and the distribution of program semantics across languages can shift, posing challenges for pairwise program translation. In this paper, we argue that jointly learning a unified model to translate code across multiple programming languages is superior to separately learning from bilingual parallel data. We propose Variational Interaction for Multilingual Program Translation~(VIM-PT), a disentanglement-based generative approach that jointly trains a unified model for multilingual program translation across multiple languages. VIM-PT disentangles code into language-shared and language-specific features, using variational inference and interaction information with a novel lower bound, then achieves program translation through conditional generation. VIM-PT demonstrates four advantages: 1) captures language-shared information more accurately from various implementations and improves the quality of multilingual program translation, 2) mines and leverages the capability of non-parallel data, 3) addresses the distribution shift of program semantics across languages, 4) and serves as a unified model, reducing deployment complexity.
Multi-Agent Path Finding with Real Robot Dynamics and Interdependent Tasks for Automated Warehouses
Venue: arXiv cs.RO 2024
Year: 2024

Multi-Agent Path Finding (MAPF) is an important optimization problem underlying the deployment of robots in automated warehouses and factories. Despite the large body of work on this topic, most approaches make heavy simplifications, both on the environment and the agents, which make the resulting algorithms impractical for real-life scenarios. In this paper, we consider a realistic problem of online order delivery in a warehouse, where a fleet of robots bring the products belonging to each order from shelves to workstations. This creates a stream of inter-dependent pickup and delivery tasks and the associated MAPF problem consists of computing realistic collision-free robot trajectories fulfilling these tasks. To solve this MAPF problem, we propose an extension of the standard Prioritized Planning algorithm to deal with the inter-dependent tasks (Interleaved Prioritized Planning) and a novel Via-Point Star (VP*) algorithm to compute an optimal dynamics-compliant robot trajectory to visit a sequence of goal locations while avoiding moving obstacles. We prove the completeness of our approach and evaluate it in simulation as well as in a real warehouse.
Aiding Humans in Financial Fraud Decision Making: Toward an XAI-Visualization Framework
Venue: arXiv cs.LG 2024
Year: 2024

AI prevails in financial fraud detection and decision making. Yet, due to concerns about biased automated decision making or profiling, regulations mandate that final decisions are made by humans. Financial fraud investigators face the challenge of manually synthesizing vast amounts of unstructured information, including AI alerts, transaction histories, social media insights, and governmental laws. Current Visual Analytics (VA) systems primarily support isolated aspects of this process, such as explaining binary AI alerts and visualizing transaction patterns, thus adding yet another layer of information to the overall complexity. In this work, we propose a framework where the VA system supports decision makers throughout all stages of financial fraud investigation, including data collection, information synthesis, and human criteria iteration. We illustrate how VA can claim a central role in AI-aided decision making, ensuring that human judgment remains in control while minimizing potential biases and labor-intensive tasks.
Exploring the Potential of Synthetic Data to Replace Real Data
Venue: arXiv cs.CV 2024
Year: 2024

The potential of synthetic data to replace real data creates a huge demand for synthetic data in data-hungry AI. This potential is even greater when synthetic data is used for training along with a small number of real images from domains other than the test domain. We find that this potential varies depending on (i) the number of cross-domain real images and (ii) the test set on which the trained model is evaluated. We introduce two new metrics, the train2test distance and $\text{AP}_\text{t2t}$, to evaluate the ability of a cross-domain training set using synthetic data to represent the characteristics of test instances in relation to training performance. Using these metrics, we delve deeper into the factors that influence the potential of synthetic data and uncover some interesting dynamics about how synthetic data impacts training performance. We hope these discoveries will encourage more widespread use of synthetic data.
Next Generation Multiple Access with Cell-Free Massive MIMO
Venue: arXiv cs.IT 2024
Year: 2024

To meet the unprecedented mobile traffic demands of future wireless networks, a paradigm shift from conventional cellular networks to distributed communication systems is imperative. Cell-free massive multiple-input multiple-output (CF-mMIMO) represents a practical and scalable embodiment of distributed/network MIMO systems. It inherits not only the key benefits of co-located massive MIMO systems but also the macro-diversity gains from distributed systems. This innovative architecture has demonstrated significant potential in enhancing network performance from various perspectives, outperforming co-located mMIMO and conventional small-cell systems. Moreover, CF-mMIMO offers flexibility in integration with emerging wireless technologies such as full-duplex (FD), non-orthogonal transmission schemes, millimeter-wave (mmWave) communications, ultra-reliable low-latency communication (URLLC), unmanned aerial vehicle (UAV)-aided communication, and reconfigurable intelligent surfaces (RISs). In this paper, we provide an overview of current research efforts on CF-mMIMO systems and their promising future application scenarios. We then elaborate on new requirements for CF-mMIMO networks in the context of these technological breakthroughs. We also present several current open challenges and outline future research directions aimed at fully realizing the potential of CF mMIMO systems in meeting the evolving demands of future wireless networks.
Securing Biometric Data: Fully Homomorphic Encryption in Multimodal Iris and Face Recognition
Venue: arXiv cs.CR 2024
Year: 2024

Multimodal biometric systems have gained popularity for their enhanced recognition accuracy and resistance to attacks like spoofing. This research explores methods for fusing iris and face feature vectors and implements robust security measures to protect fused databases and conduct matching operations on encrypted templates using fully homomorphic encryption (FHE). Evaluations on the QFIRE-I database demonstrate that our method effectively balances user privacy and accuracy while maintaining a high level of precision. Through experimentation, we demonstrate the effectiveness of employing FHE for template protection and matching within the encrypted domain, achieving notable results: a 96.41% True Acceptance Rate (TAR) for iris recognition, 81.19% TAR for face recognition, 98.81% TAR for iris fusion (left and right), and achieving a 100% TAR at 0.1% false acceptance rate (FAR) for face and iris fusion. The application of FHE presents a promising solution for ensuring accurate template matching while safeguarding user privacy and mitigating information leakage.
New weighted additive spanners
Venue: arXiv cs.DS 2024
Year: 2024

Ahmed, Bodwin, Sahneh, Kobourov, and Spence (WG 2020) introduced additive spanners for weighted graphs and constructed (i) a $+2W_{\max}$ spanner with $O(n^{3/2})$ edges and (ii) a $+4W_{\max}$ spanner with $\tilde{O}(n^{7/5})$ edges, and (iii) a $+8W_{\max}$ spanner with $O(n^{4/3})$ edges, for any weighted graph with $n$ vertices. Here $W_{\max} = \max_{e\in E}w(e)$ is the maximum edge weight in the graph. Their results for $+2W_{\max}$, $+4W_{\max}$, and $+8W_{\max}$ match the state-of-the-art bounds for the unweighted counterparts where $W_{\max} = 1$. They left open the question of constructing a $+6W_{\max}$ spanner with $O(n^{4/3})$ edges. Elkin, Gitlitz, and Neiman (DISC 2021) made significant progress on this problem by showing that there exists a $+(6+ε)W_{\max}$ spanner with $O(n^{4/3}/ε)$ edges for any fixed constant $ε> 0$. Indeed, their result is stronger as the additive stretch is local: the stretch for any pair $u,v$ is $+(6+ε)W_{uv}$ where $W_{uv}$ is the maximum weight edge on the shortest path from $u$ to $v$. In this work, we resolve the problem posted by Ahmed et al. (WG 2020) up to a poly-logarithmic factor in the number of edges: We construct a $+6W_{\max}$ spanner with $\tilde{O}(n^{4/3})$ edges. We extend the construction for $+6$-spanners of Woodruff (ICALP 2010), and our main contribution is an analysis tailoring to the weighted setting. The stretch of our spanner could also be made local, in the sense of Elkin, Gitlitz, and Neiman (DISC 2021). We also study the fast constructions of additive spanners with $+6W_{\max}$ and $+4W_{\max}$ stretches. We obtain, among other things, an algorithm for constructing a $+(6+ε)W_{\max}$ spanner of $\tilde{O}(\frac{n^{4/3}}ε)$ edges in $\tilde{O}(n^2)$ time.
Visions of Destruction: Exploring a Potential of Generative AI in Interactive Art
Venue: arXiv cs.HC 2024
Year: 2024

This paper explores the potential of generative AI within interactive art, employing a practice-based research approach. It presents the interactive artwork "Visions of Destruction" as a detailed case study, highlighting its innovative use of generative AI to create a dynamic, audience-responsive experience. This artwork applies gaze-based interaction to dynamically alter digital landscapes, symbolizing the impact of human activities on the environment by generating contemporary collages created with AI, trained on data about human damage to nature, and guided by audience interaction. The transformation of pristine natural scenes into human-made and industrialized landscapes through viewer interaction serves as a stark reminder of environmental degradation. The paper thoroughly explores the technical challenges and artistic innovations involved in creating such an interactive art installation, emphasizing the potential of generative AI to revolutionize artistic expression, audience engagement, and especially the opportunities for the interactive art field. It offers insights into the conceptual framework behind the artwork, aiming to evoke a deeper understanding and reflection on the Anthropocene era and human-induced climate change. This study contributes significantly to the field of creative AI and interactive art, blending technology and environmental consciousness in a compelling, thought-provoking manner.
Comparative Analysis: Violence Recognition from Videos using Transfer Learning
Venue: arXiv cs.CV 2024
Year: 2024

Action recognition has become a hot topic in computer vision. However, the main applications of computer vision in video processing have focused on detection of relatively simple actions while complex events such as violence detection have been comparatively less investigated. This study focuses on the benchmarking of various deep learning techniques on a complex dataset. Next, a larger dataset is utilized to test the uplift from increasing volume of data. The dataset size increase from 500 to 1,600 videos resulted in a notable average accuracy improvement of 6% across four models.
Enhancing Neural Network Interpretability Through Conductance-Based Information Plane Analysis
Venue: arXiv cs.LG 2024
Year: 2024

The Information Plane is a conceptual framework used to analyze the flow of information in neural networks, but traditional methods based on activations may not fully capture the dynamics of information processing. This paper introduces a new approach that uses layer conductance, a measure of sensitivity to input features, to enhance the Information Plane analysis. By incorporating gradient-based contributions, we provide a more precise characterization of information dynamics within the network. The proposed conductance-based Information Plane and a new Information Transformation Efficiency (ITE) metric are evaluated on pretrained ResNet50 and VGG16 models using the ImageNet dataset. Our results demonstrate the ability to identify critical hidden layers that contribute significantly to model performance and interpretability, giving insights into information compression, preservation, and utilization across layers. The conductance-based approach offers a granular perspective on feature attribution, enhancing our understanding of the decision-making processes within neural networks. Furthermore, our empirical findings challenge certain theoretical predictions of the Information Bottleneck theory, highlighting the complexities of information dynamics in real-world data scenarios. The proposed method not only advances our understanding of information dynamics in neural networks but also has the potential to significantly impact the broader field of Artificial Intelligence by enabling the development of more interpretable, efficient, and robust models.
StyleSpeech: Parameter-efficient Fine Tuning for Pre-trained Controllable Text-to-Speech
Venue: arXiv cs.SD 2024
Year: 2024

This paper introduces StyleSpeech, a novel Text-to-Speech~(TTS) system that enhances the naturalness and accuracy of synthesized speech. Building upon existing TTS technologies, StyleSpeech incorporates a unique Style Decorator structure that enables deep learning models to simultaneously learn style and phoneme features, improving adaptability and efficiency through the principles of Lower Rank Adaptation~(LoRA). LoRA allows efficient adaptation of style features in pre-trained models. Additionally, we introduce a novel automatic evaluation metric, the LLM-Guided Mean Opinion Score (LLM-MOS), which employs large language models to offer an objective and robust protocol for automatically assessing TTS system performance. Extensive testing on benchmark datasets shows that our approach markedly outperforms existing state-of-the-art baseline methods in producing natural, accurate, and high-quality speech. These advancements not only pushes the boundaries of current TTS system capabilities, but also facilitate the application of TTS system in more dynamic and specialized, such as interactive virtual assistants, adaptive audiobooks, and customized voice for gaming. Speech samples can be found in https://style-speech.vercel.app
TART: Boosting Clean Accuracy Through Tangent Direction Guided Adversarial Training
Venue: arXiv cs.LG 2024
Year: 2024

Adversarial training has been shown to be successful in enhancing the robustness of deep neural networks against adversarial attacks. However, this robustness is accompanied by a significant decline in accuracy on clean data. In this paper, we propose a novel method, called Tangent Direction Guided Adversarial Training (TART), that leverages the tangent space of the data manifold to ameliorate the existing adversarial defense algorithms. We argue that training with adversarial examples having large normal components significantly alters the decision boundary and hurts accuracy. TART mitigates this issue by estimating the tangent direction of adversarial examples and allocating an adaptive perturbation limit according to the norm of their tangential component. To the best of our knowledge, our paper is the first work to consider the concept of tangent space and direction in the context of adversarial defense. We validate the effectiveness of TART through extensive experiments on both simulated and benchmark datasets. The results demonstrate that TART consistently boosts clean accuracy while retaining a high level of robustness against adversarial attacks. Our findings suggest that incorporating the geometric properties of data can lead to more effective and efficient adversarial training methods.
Benchmarking Reinforcement Learning Methods for Dexterous Robotic Manipulation with a Three-Fingered Gripper
Venue: arXiv cs.RO 2024
Year: 2024

Reinforcement Learning (RL) training is predominantly conducted in cost-effective and controlled simulation environments. However, the transfer of these trained models to real-world tasks often presents unavoidable challenges. This research explores the direct training of RL algorithms in controlled yet realistic real-world settings for the execution of dexterous manipulation. The benchmarking results of three RL algorithms trained on intricate in-hand manipulation tasks within practical real-world contexts are presented. Our study not only demonstrates the practicality of RL training in authentic real-world scenarios, facilitating direct real-world applications, but also provides insights into the associated challenges and considerations. Additionally, our experiences with the employed experimental methods are shared, with the aim of empowering and engaging fellow researchers and practitioners in this dynamic field of robotics.
RAW-Adapter: Adapting Pre-trained Visual Model to Camera RAW Images
Venue: arXiv cs.CV 2024
Year: 2024

sRGB images are now the predominant choice for pre-training visual models in computer vision research, owing to their ease of acquisition and efficient storage. Meanwhile, the advantage of RAW images lies in their rich physical information under variable real-world challenging lighting conditions. For computer vision tasks directly based on camera RAW data, most existing studies adopt methods of integrating image signal processor (ISP) with backend networks, yet often overlook the interaction capabilities between the ISP stages and subsequent networks. Drawing inspiration from ongoing adapter research in NLP and CV areas, we introduce RAW-Adapter, a novel approach aimed at adapting sRGB pre-trained models to camera RAW data. RAW-Adapter comprises input-level adapters that employ learnable ISP stages to adjust RAW inputs, as well as model-level adapters to build connections between ISP stages and subsequent high-level networks. Additionally, RAW-Adapter is a general framework that could be used in various computer vision frameworks. Abundant experiments under different lighting conditions have shown our algorithm's state-of-the-art (SOTA) performance, demonstrating its effectiveness and efficiency across a range of real-world and synthetic datasets.
Generative-AI for AI/ML Model Adaptive Retraining in Beyond 5G Networks
Venue: arXiv cs.NI 2024
Year: 2024

Beyond fifth-generation (B5G) networks aim to support high data rates, low-latency applications, and massive machine communications. Artificial Intelligence/Machine Learning (AI/ML) can help to improve B5G network performance and efficiency. However, dynamic service demands of B5G use cases cause AI/ML model performance degradation, resulting in Service Level Agreements (SLA) violations, over- or under-provisioning of resources, etc. Retraining is essential to address the performance degradation of the AI/ML models. Existing threshold and periodic retraining approaches have potential disadvantages, such as SLA violations and inefficient resource utilization for setting a threshold parameter in a dynamic environment. This paper proposes a novel approach that predicts when to retrain AI/ML models using Generative Artificial Intelligence. The proposed predictive approach is evaluated for a Quality of Service Prediction use case on the Open Radio Access Network (O-RAN) Software Community platform and compared to the predictive approach based on the classifier and a threshold approach. Also, a realtime dataset from the Colosseum testbed is considered to evaluate Network Slicing (NS) use case with the proposed predictive approach. The results show that the proposed predictive approach outperforms both the classifier-based predictive and threshold approaches.
Dynamic operator management in meta-heuristics using reinforcement learning: an application to permutation flowshop scheduling problems
Venue: arXiv cs.LG 2024
Year: 2024

This study develops a framework based on reinforcement learning to dynamically manage a large portfolio of search operators within meta-heuristics. Using the idea of tabu search, the framework allows for continuous adaptation by temporarily excluding less efficient operators and updating the portfolio composition during the search. A Q-learning-based adaptive operator selection mechanism is used to select the most suitable operator from the dynamically updated portfolio at each stage. Unlike traditional approaches, the proposed framework requires no input from the experts regarding the search operators, allowing domain-specific non-experts to effectively use the framework. The performance of the proposed framework is analyzed through an application to the permutation flowshop scheduling problem. The results demonstrate the superior performance of the proposed framework against state-of-the-art algorithms in terms of optimality gap and convergence speed.
SpikingSSMs: Learning Long Sequences with Sparse and Parallel Spiking State Space Models
Venue: arXiv cs.CL 2024
Year: 2024

Known as low energy consumption networks, spiking neural networks (SNNs) have gained a lot of attention within the past decades. While SNNs are increasing competitive with artificial neural networks (ANNs) for vision tasks, they are rarely used for long sequence tasks, despite their intrinsic temporal dynamics. In this work, we develop spiking state space models (SpikingSSMs) for long sequence learning by leveraging on the sequence learning abilities of state space models (SSMs). Inspired by dendritic neuron structure, we hierarchically integrate neuronal dynamics with the original SSM block, meanwhile realizing sparse synaptic computation. Furthermore, to solve the conflict of event-driven neuronal dynamics with parallel computing, we propose a light-weight surrogate dynamic network which accurately predicts the after-reset membrane potential and compatible to learnable thresholds, enabling orders of acceleration in training speed compared with conventional iterative methods. On the long range arena benchmark task, SpikingSSM achieves competitive performance to state-of-the-art SSMs meanwhile realizing on average 90\% of network sparsity. On language modeling, our network significantly surpasses existing spiking large language models (spikingLLMs) on the WikiText-103 dataset with only a third of the model size, demonstrating its potential as backbone architecture for low computation cost LLMs.