OpenRaiser/Intern-Atlas huggingface dataset:
Cooperative Decision-Making for CAVs at Unsignalized Intersections: A MARL Approach with Attention and Hierarchical Game Priors
Venue: arXiv cs.RO 2024
Year: 2024

The development of autonomous vehicles has shown great potential to enhance the efficiency and safety of transportation systems. However, the decision-making issue in complex human-machine mixed traffic scenarios, such as unsignalized intersections, remains a challenge for autonomous vehicles. While reinforcement learning (RL) has been used to solve complex decision-making problems, existing RL methods still have limitations in dealing with cooperative decision-making of multiple connected autonomous vehicles (CAVs), ensuring safety during exploration, and simulating realistic human driver behaviors. In this paper, a novel and efficient algorithm, Multi-Agent Game-prior Attention Deep Deterministic Policy Gradient (MA-GA-DDPG), is proposed to address these limitations. Our proposed algorithm formulates the decision-making problem of CAVs at unsignalized intersections as a decentralized multi-agent reinforcement learning problem and incorporates an attention mechanism to capture interaction dependencies between ego CAV and other agents. The attention weights between the ego vehicle and other agents are then used to screen interaction objects and obtain prior hierarchical game relations, based on which a safety inspector module is designed to improve the traffic safety. Furthermore, both simulation and hardware-in-the-loop experiments were conducted, demonstrating that our method outperforms other baseline approaches in terms of driving safety, efficiency, and comfort.
Towards Democratizing Multilingual Large Language Models For Medicine Through A Two-Stage Instruction Fine-tuning Approach
Venue: arXiv cs.CL 2024
Year: 2024

Open-source, multilingual medical large language models (LLMs) have the potential to serve linguistically diverse populations across different regions. Adapting generic LLMs for healthcare often requires continual pretraining, but this approach is computationally expensive and sometimes impractical. Instruction fine-tuning on a specific task may not always guarantee optimal performance due to the lack of broader domain knowledge that the model needs to understand and reason effectively in diverse scenarios. To address these challenges, we introduce two multilingual instruction fine-tuning datasets, MMed-IFT and MMed-IFT-MC, containing over 200k high-quality medical samples in six languages. We propose a two-stage training paradigm: the first stage injects general medical knowledge using MMed-IFT, while the second stage fine-tunes task-specific multiple-choice questions with MMed-IFT-MC. Our method achieves competitive results on both English and multilingual benchmarks, striking a balance between computational efficiency and performance. We plan to make our dataset and model weights public at \url{https://github.com/SpassMed/Med-Llama3} in the future.
A CLIP-based siamese approach for meme classification
Venue: arXiv cs.MM 2024
Year: 2024

Memes are an increasingly prevalent element of online discourse in social networks, especially among young audiences. They carry ideas and messages that range from humorous to hateful, and are widely consumed. Their potentially high impact requires adequate means of control to moderate their use in large scale. In this work, we propose SimCLIP a deep learning-based architecture for cross-modal understanding of memes, leveraging a pre-trained CLIP encoder to produce context-aware embeddings and a Siamese fusion technique to capture the interactions between text and image. We perform an extensive experimentation on seven meme classification tasks across six datasets. We establish a new state of the art in Memotion7k with a 7.25% relative F1-score improvement, and achieve super-human performance on Harm-P with 13.73% F1-Score improvement. Our approach demonstrates the potential for compact meme classification models, enabling accurate and efficient meme monitoring. We share our code at https://github.com/jahuerta92/meme-classification-simclip
Parf: Adaptive Parameter Refining for Abstract Interpretation
Venue: arXiv cs.LO 2024
Year: 2024

The core challenge in applying abstract interpretation lies in the configuration of abstraction and analysis strategies encoded by a large number of external parameters of static analysis tools. To attain low false-positive rates (i.e., accuracy) while preserving analysis efficiency, tuning the parameters heavily relies on expert knowledge and is thus difficult to automate. In this paper, we present a fully automated framework called Parf to adaptively tune the external parameters of abstract interpretation-based static analyzers. Parf models various types of parameters as random variables subject to probability distributions over latticed parameter spaces. It incrementally refines the probability distributions based on accumulated intermediate results generated by repeatedly sampling and analyzing, thereby ultimately yielding a set of highly accurate parameter settings within a given time budget. We have implemented Parf on top of Frama-C/Eva - an off-the-shelf open-source static analyzer for C programs - and compared it against the expert refinement strategy and Frama-C/Eva's official configurations over the Frama-C OSCS benchmark. Experimental results indicate that Parf achieves the lowest number of false positives on 34/37 (91.9%) program repositories with exclusively best results on 12/37 (32.4%) cases. In particular, Parf exhibits promising performance for analyzing complex, large-scale real-world programs.
Celcomen: spatial causal disentanglement for single-cell and tissue perturbation modeling
Venue: arXiv cs.LG 2024
Year: 2024

Celcomen leverages a mathematical causality framework to disentangle intra- and inter- cellular gene regulation programs in spatial transcriptomics and single-cell data through a generative graph neural network. It can learn gene-gene interactions, as well as generate post-perturbation counterfactual spatial transcriptomics, thereby offering access to experimentally inaccessible samples. We validated its disentanglement, identifiability, and counterfactual prediction capabilities through simulations and in clinically relevant human glioblastoma, human fetal spleen, and mouse lung cancer samples. Celcomen provides the means to model disease and therapy induced changes allowing for new insights into single-cell spatially resolved tissue responses relevant to human health.
Learning control of underactuated double pendulum with Model-Based Reinforcement Learning
Venue: arXiv cs.RO 2024
Year: 2024

This report describes our proposed solution for the second AI Olympics competition held at IROS 2024. Our solution is based on a recent Model-Based Reinforcement Learning algorithm named MC-PILCO. Besides briefly reviewing the algorithm, we discuss the most critical aspects of the MC-PILCO implementation in the tasks at hand.
Applying Attribution Explanations in Truth-Discovery Quantitative Bipolar Argumentation Frameworks
Venue: arXiv cs.AI 2024
Year: 2024

Explaining the strength of arguments under gradual semantics is receiving increasing attention. For example, various studies in the literature offer explanations by computing the attribution scores of arguments or edges in Quantitative Bipolar Argumentation Frameworks (QBAFs). These explanations, known as Argument Attribution Explanations (AAEs) and Relation Attribution Explanations (RAEs), commonly employ removal-based and Shapley-based techniques for computing the attribution scores. While AAEs and RAEs have proven useful in several applications with acyclic QBAFs, they remain largely unexplored for cyclic QBAFs. Furthermore, existing applications tend to focus solely on either AAEs or RAEs, but do not compare them directly. In this paper, we apply both AAEs and RAEs, to Truth Discovery QBAFs (TD-QBAFs), which assess the trustworthiness of sources (e.g., websites) and their claims (e.g., the severity of a virus), and feature complex cycles. We find that both AAEs and RAEs can provide interesting explanations and can give non-trivial and surprising insights.
Fast Generation of Custom Floating-Point Spatial Filters on FPGAs
Venue: arXiv cs.AR 2024
Year: 2024

Convolutional Neural Networks (CNNs) have been utilised in many image and video processing applications. The convolution operator, also known as a spatial filter, is usually a linear operation, but this linearity compromises essential features and details inherent in the non-linearity present in many applications. However, due to its slow processing, the use of a nonlinear spatial filter is a significant bottleneck in many software applications. Further, due to their complexity, they are difficult to accelerate in FPGA or VLSI architectures. This paper presents novel FPGA implementations of linear and nonlinear spatial filters. More specifically, the arithmetic computations are carried out in custom floating-point, enabling a tradeoff of precision and hardware compactness, reducing algorithm development time. Further, we show that it is possible to process video at a resolution of 1080p with a frame rate of 60 frames per second, using a low-cost FPGA board. Finally, we show that using a domain-specific language will allow the rapid prototyping of image processing algorithms in custom floating-point arithmetic, allowing non-experts to quickly develop real-time video processing applications.
MMEvol: Empowering Multimodal Large Language Models with Evol-Instruct
Venue: arXiv cs.CL 2024
Year: 2024

The development of Multimodal Large Language Models (MLLMs) has seen significant advancements with increasing demands in various fields (e.g., multimodal agents, embodied intelligence). While model-driven approaches attempt to enhance MLLMs capabilities through diverse architectures, the gains have become increasingly marginal. Conversely, data-driven methods, which scale up image-text instruction data, are more effective but face limited data diversity and complexity challenges. The absence of high-quality data constitutes a significant development barrier for MLLMs. To address the data quality bottleneck, we propose MMEvol, a novel multimodal instruction data evolution framework. This framework iteratively improve data quality through a refined combination of fine-grained perception, cognitive reasoning, and interaction evolution, generating a more complex and diverse image-text instruction dataset that empowers MLLMs with enhanced capabilities. Beginning with an initial set of instructions, SEED-163K, we utilize MMEvol to systematically broaden the diversity of instruction types, extend visual reasoning steps to improve cognitive reasoning abilities, and thoroughly explore fine-grained information within images to enhance visual understanding and robustness. To comprehensively evaluate the effectiveness of our approach, we conduct extensive qualitative analysis and quantitative experiments across 13 vision-language tasks. Compared to baseline models trained with the initial seed data, the results demonstrate that our method achieves an average accuracy improvement of 3.1 percentage points. Furthermore, our approach reaches state-of-the-art (SOTA) performance in nine tasks using significantly less data compared to state-of-the-art models.
CSRec: Rethinking Sequential Recommendation from A Causal Perspective
Venue: arXiv cs.IR 2024
Year: 2024

The essence of sequential recommender systems (RecSys) lies in understanding how users make decisions. Most existing approaches frame the task as sequential prediction based on users' historical purchase records. While effective in capturing users' natural preferences, this formulation falls short in accurately modeling actual recommendation scenarios, particularly in accounting for how unsuccessful recommendations influence future purchases. Furthermore, the impact of the RecSys itself on users' decisions has not been appropriately isolated and quantitatively analyzed. To address these challenges, we propose a novel formulation of sequential recommendation, termed Causal Sequential Recommendation (CSRec). Instead of predicting the next item in the sequence, CSRec aims to predict the probability of a recommended item's acceptance within a sequential context and backtrack how current decisions are made. Critically, CSRec facilitates the isolation of various factors that affect users' final decisions, especially the influence of the recommender system itself, thereby opening new avenues for the design of recommender systems. CSRec can be seamlessly integrated into existing methodologies. Experimental evaluations on both synthetic and real-world datasets demonstrate that the proposed implementation significantly improves upon state-of-the-art baselines.
The Impact of Virtual Achievements on Online Learning Applications
Venue: arXiv cs.HC 2024
Year: 2024

In recent times a number of platforms are using badge-based achievements or leaderboards to increase user involvement and participation. Due to recent advancements, there is a question of up to what extent virtual achievement systems have on users using particular platforms. Here in this paper, we discuss measuring the impact of the leaderboard-based achievement system by integrating it into an online learning android application UPSC Pre that has thousands of questions and answers categorized topic wise related to UPSC exams, one of the toughest exams to crack in the world. We are conducting the experiment on 10 randomly chosen students who are using the app in a controlled setting and the data measurement is done using the Firebase Analytics tool by Google. We observed that the students using the leaderboard have increased participation without any reductions in their quality and the time of using the platform has increased compared to previous engagements. Students who participated in the experiment felt the leaderboard was competitive and enjoyed gaining positions on the leaderboard and wanted it in the User interface for other platforms as well. The research has an impact in designing the learning applications to cater and improve the user experience in the future. The results are not limited to educational purposes but can be expanded to other fields such as self development applications, other research projects, Gaming industry and many more.
Big-Thick Data generation via reference and personal context unification
Venue: arXiv cs.HC 2024
Year: 2024

Smart devices generate vast amounts of big data, mainly in the form of sensor data. While allowing for the prediction of many aspects of human behaviour (e.g., physical activities, transportation modes), this data has a major limitation in that it is not thick, that is, it does not carry information about the context within which it was generated. Context - what was accomplished by a user, how and why, and in which overall situation - all these factors must be explicitly represented for the data to be self-explanatory and meaningful. In this paper, we introduce Big-Thick Data as highly contextualized data encoding, for each and every user, both her subjective personal view of the world and the objective view of an all-observing third party taken as reference. We model big-thick data by enforcing the distinction between personal context and reference context. We show that these two types of context can be unified in many different ways, thus allowing for different types of questions about the users' behaviour and the world around them and, also, for multiple different answers to the same question. We validate the model with a case study that integrates the personal big-thick data of one hundred and fifty-eight University students over a period of four weeks with the reference context built using the data provided by OpenStreetMap.
Automating the Practice of Science -- Opportunities, Challenges, and Implications
Venue: arXiv cs.CY 2024
Year: 2024

Automation transformed various aspects of our human civilization, revolutionizing industries and streamlining processes. In the domain of scientific inquiry, automated approaches emerged as powerful tools, holding promise for accelerating discovery, enhancing reproducibility, and overcoming the traditional impediments to scientific progress. This article evaluates the scope of automation within scientific practice and assesses recent approaches. Furthermore, it discusses different perspectives to the following questions: Where do the greatest opportunities lie for automation in scientific practice?; What are the current bottlenecks of automating scientific practice?; and What are significant ethical and practical consequences of automating scientific practice? By discussing the motivations behind automated science, analyzing the hurdles encountered, and examining its implications, this article invites researchers, policymakers, and stakeholders to navigate the rapidly evolving frontier of automated scientific practice.
OPAL: Outlier-Preserved Microscaling Quantization Accelerator for Generative Large Language Models
Venue: arXiv cs.LG 2024
Year: 2024

To overcome the burden on the memory size and bandwidth due to ever-increasing size of large language models (LLMs), aggressive weight quantization has been recently studied, while lacking research on quantizing activations. In this paper, we present a hardware-software co-design method that results in an energy-efficient LLM accelerator, named OPAL, for generation tasks. First of all, a novel activation quantization method that leverages the microscaling data format while preserving several outliers per sub-tensor block (e.g., four out of 128 elements) is proposed. Second, on top of preserving outliers, mixed precision is utilized that sets 5-bit for inputs to sensitive layers in the decoder block of an LLM, while keeping inputs to less sensitive layers to 3-bit. Finally, we present the OPAL hardware architecture that consists of FP units for handling outliers and vectorized INT multipliers for dominant non-outlier related operations. In addition, OPAL uses log2-based approximation on softmax operations that only requires shift and subtraction to maximize power efficiency. As a result, we are able to improve the energy efficiency by 1.6~2.2x, and reduce the area by 2.4~3.1x with negligible accuracy loss, i.e., <1 perplexity increase.
Towards Narrowing the Generalization Gap in Deep Boolean Networks
Venue: arXiv cs.LG 2024
Year: 2024

The rapid growth of the size and complexity in deep neural networks has sharply increased computational demands, challenging their efficient deployment in real-world scenarios. Boolean networks, constructed with logic gates, offer a hardware-friendly alternative that could enable more efficient implementation. However, their ability to match the performance of traditional networks has remained uncertain. This paper explores strategies to enhance deep Boolean networks with the aim of surpassing their traditional counterparts. We propose novel methods, including logical skip connections and spatiality preserving sampling, and validate them on vision tasks using widely adopted datasets, demonstrating significant improvement over existing approaches. Our analysis shows how deep Boolean networks can maintain high performance while minimizing computational costs through 1-bit logic operations. These findings suggest that Boolean networks are a promising direction for efficient, high-performance deep learning models, with significant potential for advancing hardware-accelerated AI applications.
KModels: Unlocking AI for Business Applications
Venue: arXiv cs.SE 2024
Year: 2024

As artificial intelligence (AI) continues to rapidly advance, there is a growing demand to integrate AI capabilities into existing business applications. However, a significant gap exists between the rapid progress in AI and how slowly AI is being embedded into business environments. Deploying well-performing lab models into production settings, especially in on-premise environments, often entails specialized expertise and imposes a heavy burden of model management, creating significant barriers to implementing AI models in real-world applications. KModels leverages proven libraries and platforms (Kubeflow Pipelines, KServe) to streamline AI adoption by supporting both AI developers and consumers. It allows model developers to focus solely on model development and share models as transportable units (Templates), abstracting away complex production deployment concerns. KModels enables AI consumers to eliminate the need for a dedicated data scientist, as the templates encapsulate most data science considerations while providing business-oriented control. This paper presents the architecture of KModels and the key decisions that shape it. We outline KModels' main components as well as its interfaces. Furthermore, we explain how KModels is highly suited for on-premise deployment but can also be used in cloud environments. The efficacy of KModels is demonstrated through the successful deployment of three AI models within an existing Work Order Management system. These models operate in a client's data center and are trained on local data, without data scientist intervention. One model improved the accuracy of Failure Code specification for work orders from 46% to 83%, showcasing the substantial benefit of accessible and localized AI solutions.
SVFit: Parameter-Efficient Fine-Tuning of Large Pre-Trained Models Using Singular Values
Venue: arXiv cs.LG 2024
Year: 2024

Large pre-trained models (LPMs) have demonstrated exceptional performance in diverse natural language processing and computer vision tasks. However, fully fine-tuning these models poses substantial memory challenges, particularly in resource-constrained environments. Parameter-efficient fine-tuning (PEFT) methods, such as LoRA, mitigate this issue by adjusting only a small subset of parameters. Nevertheless, these methods typically employ random initialization for low-rank matrices, which can lead to inefficiencies in gradient descent and diminished generalizability due to suboptimal starting points. To address these limitations, we propose SVFit, a novel PEFT approach that leverages singular value decomposition (SVD) to initialize low-rank matrices using critical singular values as trainable parameters. Specifically, SVFit performs SVD on the pre-trained weight matrix to obtain the best rank-r approximation matrix, emphasizing the most critical singular values that capture over 99% of the matrix's information. These top-r singular values are then used as trainable parameters to scale the fundamental subspaces of the matrix, facilitating rapid domain adaptation. Extensive experiments across various pre-trained models in natural language understanding, text-to-image generation, and image classification tasks reveal that SVFit outperforms LoRA while requiring 16 times fewer trainable parameters.