OpenRaiser/Intern-Atlas huggingface dataset:
A Problem-Based Learning Approach to Teaching Design in CS1
Venue: arXiv cs.CY 2024
Year: 2024

Design skills are increasingly recognized as a core competency for software professionals. Unfortunately, these skills are difficult to teach because design requires freedom and open-ended thinking, but new designers require a structured process to keep them from being overwhelmed by possibilities. We scaffolded this by creating worksheets for every Design Thinking step, and embedding them in a PowerPoint deck on which students can collaborate. We present our experience teaching a team design project course to 200 first-year-university students, taking them from user interviews to functional prototypes. To challenge and support every student in a class where high school programming experience ranged from zero hours to three computer science courses, we gave teams the option of developing single-user or multi-user (distributed) web applications, using two Event-Driven Programming frameworks. We identified common failure modes from previous years, and developed the scaffolded approach and problem definition to avoid them. The techniques developed include using a "game matrix" for structured brainstorming and developing projects that require students to empathize with users very different from themselves. We present quantitative and qualitative evidence from surveys and focus groups that show how these strategies impacted learning, and the extent to which students' awareness of the strategies led to the development of metacognitive abilities.
Iter-AHMCL: Alleviate Hallucination for Large Language Model via Iterative Model-level Contrastive Learning
Venue: arXiv cs.CL 2024
Year: 2024

The development of Large Language Models (LLMs) has significantly advanced various AI applications in commercial and scientific research fields, such as scientific literature summarization, writing assistance, and knowledge graph construction. However, a significant challenge is the high risk of hallucination during LLM inference, which can lead to security concerns like factual inaccuracies, inconsistent information, and fabricated content. To tackle this issue, it is essential to develop effective methods for reducing hallucination while maintaining the original capabilities of the LLM. This paper introduces a novel approach called Iterative Model-level Contrastive Learning (Iter-AHMCL) to address hallucination. This method modifies the representation layers of pre-trained LLMs by using contrastive `positive' and `negative' models, trained on data with and without hallucinations. By leveraging the differences between these two models, we create a more straightforward pathway to eliminate hallucinations, and the iterative nature of contrastive learning further enhances performance. Experimental validation on four pre-trained foundation LLMs (LLaMA2, Alpaca, LLaMA3, and Qwen) finetuning with a specially designed dataset shows that our approach achieves an average improvement of 10.1 points on the TruthfulQA benchmark. Comprehensive experiments demonstrate the effectiveness of Iter-AHMCL in reducing hallucination while maintaining the general capabilities of LLMs.
COMET: Towards Partical W4A4KV4 LLMs Serving
Venue: arXiv cs.AR 2024
Year: 2024

Quantization is a widely-used compression technology to reduce the overhead of serving large language models (LLMs) on terminal devices and in cloud data centers. However, prevalent quantization methods, such as 8-bit weight-activation or 4-bit weight-only quantization, achieve limited performance improvements due to poor support for low-precision (e.g., 4-bit) activation. This work, for the first time, realizes practical W4A4KV4 serving for LLMs, fully utilizing the INT4 tensor cores on modern GPUs and reducing the memory bottleneck caused by the KV cache. Specifically, we propose a novel fine-grained mixed-precision quantization algorithm (FMPQ) that compresses most activations into 4-bit with negligible accuracy loss. To support mixed-precision matrix multiplication for W4A4 and W4A8, we develop a highly optimized W4Ax kernel. Our approach introduces a novel mixed-precision data layout to facilitate access and fast dequantization for activation and weight tensors, utilizing the GPU's software pipeline to hide the overhead of data loading and conversion. Additionally, we propose fine-grained streaming multiprocessor (SM) scheduling to achieve load balance across different SMs. We integrate the optimized W4Ax kernel into our inference framework, COMET, and provide efficient management to support popular LLMs such as LLaMA-3-70B. Extensive evaluations demonstrate that, when running LLaMA family models on a single A100-80G-SMX4, COMET achieves a kernel-level speedup of \textbf{$2.88\times$} over cuBLAS and a \textbf{$2.02 \times$} throughput improvement compared to TensorRT-LLM from an end-to-end framework perspective.
The State of Robot Motion Generation
Venue: arXiv cs.RO 2024
Year: 2024

This paper reviews the large spectrum of methods for generating robot motion proposed over the 50 years of robotics research culminating in recent developments. It crosses the boundaries of methodologies, typically not surveyed together, from those that operate over explicit models to those that learn implicit ones. The paper discusses the current state-of-the-art as well as properties of varying methodologies, highlighting opportunities for integration.
Learning Differentiable Tensegrity Dynamics using Graph Neural Networks
Venue: arXiv cs.RO 2024
Year: 2024

Tensegrity robots are composed of rigid struts and flexible cables. They constitute an emerging class of hybrid rigid-soft robotic systems and are promising systems for a wide array of applications, ranging from locomotion to assembly. They are difficult to control and model accurately, however, due to their compliance and high number of degrees of freedom. To address this issue, prior work has introduced a differentiable physics engine designed for tensegrity robots based on first principles. In contrast, this work proposes the use of graph neural networks to model contact dynamics over a graph representation of tensegrity robots, which leverages their natural graph-like cable connectivity between end caps of rigid rods. This learned simulator can accurately model 3-bar and 6-bar tensegrity robot dynamics in simulation-to-simulation experiments where MuJoCo is used as the ground truth. It can also achieve higher accuracy than the previous differentiable engine for a real 3-bar tensegrity robot, for which the robot state is only partially observable. When compared against direct applications of recent mesh-based graph neural network simulators, the proposed approach is computationally more efficient, both for training and inference, while achieving higher accuracy. Code and data are available at https://github.com/nchen9191/tensegrity_gnn_simulator_public
OmnixR: Evaluating Omni-modality Language Models on Reasoning across Modalities
Venue: arXiv cs.AI 2024
Year: 2024

We introduce OmnixR, an evaluation suite designed to benchmark SoTA Omni-modality Language Models, such as GPT-4o and Gemini. Evaluating OLMs, which integrate multiple modalities such as text, vision, and audio, presents unique challenges. Particularly, the user message might often consist of multiple modalities, such that OLMs have to establish holistic understanding and reasoning across modalities to accomplish the task. Existing benchmarks are limited to single modality or dual-modality tasks, overlooking comprehensive multi-modal assessments of model reasoning. To address this, OmnixR offers two evaluation variants: (1)synthetic subset: a synthetic dataset generated automatically by translating text into multiple modalities--audio, images, video, and hybrids (Omnify). (2)realistic subset: a real-world dataset, manually curated and annotated by experts, for evaluating cross-modal reasoning in natural settings. OmnixR presents a unique evaluation towards assessing OLMs over a diverse mix of modalities, such as a question that involves video, audio, and text, providing a rigorous cross-modal reasoning testbed unlike any existing benchmarks. Our experiments find that all state-of-the-art OLMs struggle with OmnixR questions that require integrating information from multiple modalities to answer. Further analysis highlights differences in reasoning behavior, underscoring the challenges of omni-modal AI alignment.
Game Theory Meets Statistical Mechanics in Deep Learning Design
Venue: arXiv cs.LG 2024
Year: 2024

We present a novel deep graphical representation that seamlessly merges principles of game theory with laws of statistical mechanics. It performs feature extraction, dimensionality reduction, and pattern classification within a single learning framework. Our approach draws an analogy between neurons in a network and players in a game theory model. Furthermore, each neuron viewed as a classical particle (subject to statistical physics' laws) is mapped to a set of actions representing specific activation value, and neural network layers are conceptualized as games in a sequential cooperative game theory setting. The feed-forward process in deep learning is interpreted as a sequential game, where each game comprises a set of players. During training, neurons are iteratively evaluated and filtered based on their contributions to a payoff function, which is quantified using the Shapley value driven by an energy function. Each set of neurons that significantly contributes to the payoff function forms a strong coalition. These neurons are the only ones permitted to propagate the information forward to the next layers. We applied this methodology to the task of facial age estimation and gender classification. Experimental results demonstrate that our approach outperforms both multi-layer perceptron and convolutional neural network models in terms of efficiency and accuracy.
Fusion from Decomposition: A Self-Supervised Approach for Image Fusion and Beyond
Venue: arXiv cs.CV 2024
Year: 2024

Image fusion is famous as an alternative solution to generate one high-quality image from multiple images in addition to image restoration from a single degraded image. The essence of image fusion is to integrate complementary information from source images. Existing fusion methods struggle with generalization across various tasks and often require labor-intensive designs, in which it is difficult to identify and extract useful information from source images due to the diverse requirements of each fusion task. Additionally, these methods develop highly specialized features for different downstream applications, hindering the adaptation to new and diverse downstream tasks. To address these limitations, we introduce DeFusion++, a novel framework that leverages self-supervised learning (SSL) to enhance the versatility of feature representation for different image fusion tasks. DeFusion++ captures the image fusion task-friendly representations from large-scale data in a self-supervised way, overcoming the constraints of limited fusion datasets. Specifically, we introduce two innovative pretext tasks: common and unique decomposition (CUD) and masked feature modeling (MFM). CUD decomposes source images into abstract common and unique components, while MFM refines these components into robust fused features. Jointly training of these tasks enables DeFusion++ to produce adaptable representations that can effectively extract useful information from various source images, regardless of the fusion task. The resulting fused representations are also highly adaptable for a wide range of downstream tasks, including image segmentation and object detection. DeFusion++ stands out by producing versatile fused representations that can enhance both the quality of image fusion and the effectiveness of downstream high-level vision tasks, simplifying the process with the elegant fusion framework.
Covert Multicast in UAV-Enabled Wireless Communication Systems With One-hop and Two-hop Strategies
Venue: arXiv cs.CR 2024
Year: 2024

This paper delves into the time-efficient covert multicast in a wireless communication system facilitated by Unmanned Aerial Vehicle (UAV), in which the UAV aims to disseminate a common covert information to multiple ground users (GUs) while suffering from the risk of detection by a ground warden (Willie). We propose one hop (OH) and two hop (TH) transmission schemes, first develop a theoretical framework for performance modeling of both the detection error probability at Willie and the transmission time at UAV. The optimization problems subject to the covertness constraint for the two transmission schemes are then formulated to gain insights into the system settings of the UAV's prior transmit probability, transmit power and horizontal location that affect the minimum transmission time. The optimization problems are non-convex and challenging to give numerical results. We thus explore the optimal setting of the transmit power and the prior transmit probability for the UAV separately under specific parameters with two schemes. We further propose a particle swarm optimization (PSO) based algorithm and an exhaustive algorithm to provide the joint solutions for the optimization problem with the OH transmission scheme and TH scheme, respectively. Finally, the efficiency of the proposed PSO-based algorithm is substantiated through extensive numerical results.
Conjunction Subspaces Test for Conformal and Selective Classification
Venue: arXiv cs.LG 2024
Year: 2024

In this paper, we present a new classifier, which integrates significance testing results over different random subspaces to yield consensus p-values for quantifying the uncertainty of classification decision. The null hypothesis is that the test sample has no association with the target class on a randomly chosen subspace, and hence the classification problem can be formulated as a problem of testing for the conjunction of hypotheses. The proposed classifier can be easily deployed for the purpose of conformal prediction and selective classification with reject and refine options by simply thresholding the consensus p-values. The theoretical analysis on the generalization error bound of the proposed classifier is provided and empirical studies on real data sets are conducted as well to demonstrate its effectiveness.
Neuron-based Personality Trait Induction in Large Language Models
Venue: arXiv cs.CL 2024
Year: 2024

Large language models (LLMs) have become increasingly proficient at simulating various personality traits, an important capability for supporting related applications (e.g., role-playing). To further improve this capacity, in this paper, we present a neuron-based approach for personality trait induction in LLMs, with three major technical contributions. First, we construct PersonalityBench, a large-scale dataset for identifying and evaluating personality traits in LLMs. This dataset is grounded in the Big Five personality traits from psychology and is designed to assess the generative capabilities of LLMs towards specific personality traits. Second, by leveraging PersonalityBench, we propose an efficient method for identifying personality-related neurons within LLMs by examining the opposite aspects of a given trait. Third, we develop a simple yet effective induction method that manipulates the values of these identified personality-related neurons. This method enables fine-grained control over the traits exhibited by LLMs without training and modifying model parameters. Extensive experiments validate the efficacy of our neuron identification and trait induction methods. Notably, our approach achieves comparable performance as fine-tuned models, offering a more efficient and flexible solution for personality trait induction in LLMs. We provide access to all the mentioned resources at https://github.com/RUCAIBox/NPTI.
Privacy by Design: Bringing User Awareness to Privacy Risks in Internet of Things
Venue: arXiv cs.HC 2024
Year: 2024

This paper aims to cover and summarize the field of IoT and related privacy concerns through the lens of privacy by design. With the ever-increasing incorporation of technology within our daily lives and an ever-growing active research into smart devices and technologies, privacy concerns are inevitable. We intend to briefly cover the broad topic of privacy in the IoT space, the inherent challenges and risks in such systems, and a few recent techniques that intend to resolve these issues on the subdomain level and a system scale level. We then proceed to approach this situation through design thinking and privacy-by-design, given that most of the prior efforts are based on resolving privacy concerns on technical grounds with system-level design. We participated in a co-design workshop for the privacy of a content creation platform and used those findings to deploy a survey-based mechanism to tackle some key concern areas for user groups and formulate design principles for privacy that promote transparent, user-centered, and awareness-provoking privacy design.
Guaranteeing MMS for All but One Agent When Allocating Indivisible Chores
Venue: arXiv cs.GT 2024
Year: 2024

We study the problem of allocating $m$ indivisible chores to $n$ agents with additive cost functions under the fairness notion of maximin share (MMS). In this work, we propose a notion called $α$-approximate all-but-one maximin share ($α$-AMMS) which is a stronger version of $α$-approximate MMS. An allocation is called $α$-AMMS if $n-1$ agents are guaranteed their MMS values and the remaining agent is guaranteed $α$-approximation of her MMS value. We show that there exist $α$-AMMS allocations, with $α= 9/8$ for three agents; $α= 4/3$ for four agents; and $α= (n+1)^2/4n$ for $n\geq 5$ agents.
SELF-BART : A Transformer-based Molecular Representation Model using SELFIES
Venue: arXiv cs.CE 2024
Year: 2024

Large-scale molecular representation methods have revolutionized applications in material science, such as drug discovery, chemical modeling, and material design. With the rise of transformers, models now learn representations directly from molecular structures. In this study, we develop an encoder-decoder model based on BART that is capable of leaning molecular representations and generate new molecules. Trained on SELFIES, a robust molecular string representation, our model outperforms existing baselines in downstream tasks, demonstrating its potential in efficient and effective molecular data analysis and manipulation.
HerO at AVeriTeC: The Herd of Open Large Language Models for Verifying Real-World Claims
Venue: arXiv cs.CL 2024
Year: 2024

To tackle the AVeriTeC shared task hosted by the FEVER-24, we introduce a system that only employs publicly available large language models (LLMs) for each step of automated fact-checking, dubbed the Herd of Open LLMs for verifying real-world claims (HerO). For evidence retrieval, a language model is used to enhance a query by generating hypothetical fact-checking documents. We prompt pretrained and fine-tuned LLMs for question generation and veracity prediction by crafting prompts with retrieved in-context samples. HerO achieved 2nd place on the leaderboard with the AVeriTeC score of 0.57, suggesting the potential of open LLMs for verifying real-world claims. For future research, we make our code publicly available at https://github.com/ssu-humane/HerO.
Prompt Compression for Large Language Models: A Survey
Venue: arXiv cs.CL 2024
Year: 2024

Leveraging large language models (LLMs) for complex natural language tasks typically requires long-form prompts to convey detailed requirements and information, which results in increased memory usage and inference costs. To mitigate these challenges, multiple efficient methods have been proposed, with prompt compression gaining significant research interest. This survey provides an overview of prompt compression techniques, categorized into hard prompt methods and soft prompt methods. First, the technical approaches of these methods are compared, followed by an exploration of various ways to understand their mechanisms, including the perspectives of attention optimization, Parameter-Efficient Fine-Tuning (PEFT), modality integration, and new synthetic language. We also examine the downstream adaptations of various prompt compression techniques. Finally, the limitations of current prompt compression methods are analyzed, and several future directions are outlined, such as optimizing the compression encoder, combining hard and soft prompts methods, and leveraging insights from multimodality.
Orienteering (with Time Windows) on Restricted Graph Classes
Venue: arXiv cs.DS 2024
Year: 2024

Given a graph with edge costs and vertex profits and given a budget B, the Orienteering Problem asks for a walk of cost at most B of maximum profit. Additionally, each profit may be given with a time window within it can be collected by the walk. While the Orienteering Problem and thus the version with time windows are NP-hard in general, it remains open on numerous special graph classes. Since in several applications, especially for planning a route from A to B with waypoints, the input graph can be restricted to tree-like or path-like structures, in this paper we consider orienteering on these graph classes. While the Orienteering Problem with time windows is NP-hard even on undirected paths and cycles, and remains so even if all profits must be collected, we show that for directed paths it can be solved efficiently, even if each profit can be collected in one of several time windows. The same case is shown to be NP-hard for directed cycles. Particularly interesting is the Orienteering Problem on a directed cycle with one time window per profit. We give an efficient algorithm for the case where all time windows are shorter than the length of the cycle, resulting in a 2-approximation for the general setting. We further develop a polynomial-time approximation scheme for this problem and give a polynomial algorithm for the case where all profits must be collected. For the Orienteering Problem with time windows for the edges, we give a quadratic time algorithm for undirected paths and observe that the problem is NP-hard for trees. In the variant without time windows, we show that on trees and thus on graphs with bounded tree-width the Orienteering Problem remains NP-hard. We present, however, an FPT algorithm to solve orienteering with unit profits that we then use to obtain a ($1+\varepsilon$)-approximation algorithm on graphs with arbitrary profits and bounded tree-width.
Approaching Metaheuristic Deep Learning Combos for Automated Data Mining
Venue: arXiv cs.LG 2024
Year: 2024

Lack of data on which to perform experimentation is a recurring issue in many areas of research, particularly in machine learning. The inability of most automated data mining techniques to be generalized to all types of data is inherently related with their dependency on those types which deems them ineffective against anything slightly different. Meta-heuristics are algorithms which attempt to optimize some solution independently of the type of data used, whilst classifiers or neural networks focus on feature extrapolation and dimensionality reduction to fit some model onto data arranged in a particular way. These two algorithmic fields encompass a group of characteristics which when combined are seemingly capable of achieving data mining regardless of how it is arranged. To this end, this work proposes a means of combining meta-heuristic methods with conventional classifiers and neural networks in order to perform automated data mining. Experiments on the MNIST dataset for handwritten digit recognition were performed and it was empirically observed that using a ground truth labeled dataset's validation accuracy is inadequate for correcting labels of other previously unseen data instances.