OpenRaiser/Intern-Atlas huggingface dataset:
Can LLMs Capture Human Preferences?
Venue: arXiv cs.CL 2024
Year: 2024

We explore the viability of Large Language Models (LLMs), specifically OpenAI's GPT-3.5 and GPT-4, in emulating human survey respondents and eliciting preferences, with a focus on intertemporal choices. Leveraging the extensive literature on intertemporal discounting for benchmarking, we examine responses from LLMs across various languages and compare them to human responses, exploring preferences between smaller, sooner, and larger, later rewards. Our findings reveal that both GPT models demonstrate less patience than humans, with GPT-3.5 exhibiting a lexicographic preference for earlier rewards, unlike human decision-makers. Though GPT-4 does not display lexicographic preferences, its measured discount rates are still considerably larger than those found in humans. Interestingly, GPT models show greater patience in languages with weak future tense references, such as German and Mandarin, aligning with existing literature that suggests a correlation between language structure and intertemporal preferences. We demonstrate how prompting GPT to explain its decisions, a procedure we term "chain-of-thought conjoint," can mitigate, but does not eliminate, discrepancies between LLM and human responses. While directly eliciting preferences using LLMs may yield misleading results, combining chain-of-thought conjoint with topic modeling aids in hypothesis generation, enabling researchers to explore the underpinnings of preferences. Chain-of-thought conjoint provides a structured framework for marketers to use LLMs to identify potential attributes or factors that can explain preference heterogeneity across different customers and contexts.
PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits
Venue: arXiv cs.CL 2024
Year: 2024

Despite the many use cases for large language models (LLMs) in creating personalized chatbots, there has been limited research on evaluating the extent to which the behaviors of personalized LLMs accurately and consistently reflect specific personality traits. We consider studying the behavior of LLM-based agents which we refer to as LLM personas and present a case study with GPT-3.5 and GPT-4 to investigate whether LLMs can generate content that aligns with their assigned personality profiles. To this end, we simulate distinct LLM personas based on the Big Five personality model, have them complete the 44-item Big Five Inventory (BFI) personality test and a story writing task, and then assess their essays with automatic and human evaluations. Results show that LLM personas' self-reported BFI scores are consistent with their designated personality types, with large effect sizes observed across five traits. Additionally, LLM personas' writings have emerging representative linguistic patterns for personality traits when compared with a human writing corpus. Furthermore, human evaluation shows that humans can perceive some personality traits with an accuracy of up to 80%. Interestingly, the accuracy drops significantly when the annotators were informed of AI authorship.
Hypernode Automata
Venue: arXiv cs.FL 2024
Year: 2024

We introduce hypernode automata as a new specification formalism for hyperproperties of concurrent systems. They are finite automata with nodes labeled with hypernode logic formulas and transitions labeled with actions. A hypernode logic formula specifies relations between sequences of variable values in different system executions. Unlike HyperLTL, hypernode logic takes an asynchronous view on execution traces by constraining the values and the order of value changes of each variable without correlating the timing of the changes. Different execution traces are synchronized solely through the transitions of hypernode automata. Hypernode automata naturally combine asynchronicity at the node level with synchronicity at the transition level. We show that the model-checking problem for hypernode automata is decidable over action-labeled Kripke structures, whose actions induce transitions of the specification automaton. For this reason, hypernode automaton is a suitable formalism for specifying and verifying asynchronous hyperproperties, such as declassifying observational determinism in multi-threaded programs.
Maximum values of the Sombor-index-like graph invariants of trees and connected graphs
Venue: arXiv cs.DM 2024
Year: 2024

A set of novel vertex-degree-based invariants was introduced by Gutman, denoted by \newline $SO_1, SO_2, \ldots,SO_6$. These invariants were constructed through geometric reasoning based on a new graph invariant framework. Motivated by proposed open problems in [Z. Tang, Q. Li, H. Deng, \textit{Trees with Extremal Values of the Sombor-Index-Like Graph Invariants}, MATCH Commun. Math. Comput. Chem. \textbf{90} (2023) 203-222], we have found the maximum values of $SO_5$ and $SO_6$ in the set of molecular trees with a given number of vertices, respectively, and we have found the maximum value of $SO_5$ in a class of connected graphs.
General Neural Gauge Fields
Venue: arXiv cs.CV 2024
Year: 2024

The recent advance of neural fields, such as neural radiance fields, has significantly pushed the boundary of scene representation learning. Aiming to boost the computation efficiency and rendering quality of 3D scenes, a popular line of research maps the 3D coordinate system to another measuring system, e.g., 2D manifolds and hash tables, for modeling neural fields. The conversion of coordinate systems can be typically dubbed as \emph{gauge transformation}, which is usually a pre-defined mapping function, e.g., orthogonal projection or spatial hash function. This begs a question: can we directly learn a desired gauge transformation along with the neural field in an end-to-end manner? In this work, we extend this problem to a general paradigm with a taxonomy of discrete \& continuous cases, and develop a learning framework to jointly optimize gauge transformations and neural fields. To counter the problem that the learning of gauge transformations can collapse easily, we derive a general regularization mechanism from the principle of information conservation during the gauge transformation. To circumvent the high computation cost in gauge learning with regularization, we directly derive an information-invariant gauge transformation which allows to preserve scene information inherently and yield superior performance. Project: https://fnzhan.com/Neural-Gauge-Fields
Can Large Language Models Transform Computational Social Science?
Venue: arXiv cs.CL 2024
Year: 2024

Large Language Models (LLMs) are capable of successfully performing many language processing tasks zero-shot (without training data). If zero-shot LLMs can also reliably classify and explain social phenomena like persuasiveness and political ideology, then LLMs could augment the Computational Social Science (CSS) pipeline in important ways. This work provides a road map for using LLMs as CSS tools. Towards this end, we contribute a set of prompting best practices and an extensive evaluation pipeline to measure the zero-shot performance of 13 language models on 25 representative English CSS benchmarks. On taxonomic labeling tasks (classification), LLMs fail to outperform the best fine-tuned models but still achieve fair levels of agreement with humans. On free-form coding tasks (generation), LLMs produce explanations that often exceed the quality of crowdworkers' gold references. We conclude that the performance of today's LLMs can augment the CSS research pipeline in two ways: (1) serving as zero-shot data annotators on human annotation teams, and (2) bootstrapping challenging creative generation tasks (e.g., explaining the underlying attributes of a text). In summary, LLMs are posed to meaningfully participate in social science analysis in partnership with humans.
Black-box Prompt Tuning with Subspace Learning
Venue: arXiv cs.CL 2024
Year: 2024

Black-box prompt tuning employs derivative-free optimization algorithms to learn prompts within low-dimensional subspaces rather than back-propagating through the network of Large Language Models (LLMs). Recent studies reveal that black-box prompt tuning lacks versatility across tasks and LLMs, which we believe is related to the suboptimal choice of subspaces. In this paper, we introduce Black-box prompt tuning with Subspace Learning (BSL) to enhance the versatility of black-box prompt tuning. Based on the assumption that nearly optimal prompts for similar tasks reside in a common subspace, we propose identifying such subspaces through meta-learning on a collection of similar source tasks. Consequently, for a target task that shares similarities with the source tasks, we expect that optimizing within the identified subspace can yield a prompt that performs well on the target task. Experimental results confirm that our BSL framework consistently achieves competitive performance across various downstream tasks and LLMs.
REINFOREST: Reinforcing Semantic Code Similarity for Cross-Lingual Code Search Models
Venue: arXiv cs.SE 2024
Year: 2024

This paper introduces a novel code-to-code search technique that enhances the performance of Large Language Models (LLMs) by including both static and dynamic features as well as utilizing both similar and dissimilar examples during training. We present the first-ever code search method that encodes dynamic runtime information during training without the need to execute either the corpus under search or the search query at inference time and the first code search technique that trains on both positive and negative reference samples. To validate the efficacy of our approach, we perform a set of studies demonstrating the capability of enhanced LLMs to perform cross-language code-to-code search. Our evaluation demonstrates that the effectiveness of our approach is consistent across various model architectures and programming languages. We outperform the state-of-the-art cross-language search tool by up to 44.7\%. Moreover, our ablation studies reveal that even a single positive and negative reference sample in the training process results in substantial performance improvements demonstrating both similar and dissimilar references are important parts of code search. Importantly, we show that enhanced well-crafted, fine-tuned models consistently outperform enhanced larger modern LLMs without fine tuning, even when enhancing the largest available LLMs highlighting the importance for open-sourced models. To ensure the reproducibility and extensibility of our research, we present an open-sourced implementation of our tool and training procedures called REINFOREST.
DBAT: Dynamic Backward Attention Transformer for Material Segmentation with Cross-Resolution Patches
Venue: arXiv cs.CV 2024
Year: 2024

The objective of dense material segmentation is to identify the material categories for every image pixel. Recent studies adopt image patches to extract material features. Although the trained networks can improve the segmentation performance, their methods choose a fixed patch resolution which fails to take into account the variation in pixel area covered by each material. In this paper, we propose the Dynamic Backward Attention Transformer (DBAT) to aggregate cross-resolution features. The DBAT takes cropped image patches as input and gradually increases the patch resolution by merging adjacent patches at each transformer stage, instead of fixing the patch resolution during training. We explicitly gather the intermediate features extracted from cross-resolution patches and merge them dynamically with predicted attention masks. Experiments show that our DBAT achieves an accuracy of 86.85%, which is the best performance among state-of-the-art real-time models. Like other successful deep learning solutions with complex architectures, the DBAT also suffers from lack of interpretability. To address this problem, this paper examines the properties that the DBAT makes use of. By analysing the cross-resolution features and the attention weights, this paper interprets how the DBAT learns from image patches. We further align features to semantic labels, performing network dissection, to infer that the proposed model can extract material-related features better than other methods. We show that the DBAT model is more robust to network initialisation, and yields fewer variable predictions compared to other models. The project code is available at https://github.com/heng-yuwen/Dynamic-Backward-Attention-Transformer.
Active Continual Learning: On Balancing Knowledge Retention and Learnability
Venue: arXiv cs.LG 2024
Year: 2024

Acquiring new knowledge without forgetting what has been learned in a sequence of tasks is the central focus of continual learning (CL). While tasks arrive sequentially, the training data are often prepared and annotated independently, leading to the CL of incoming supervised learning tasks. This paper considers the under-explored problem of active continual learning (ACL) for a sequence of active learning (AL) tasks, where each incoming task includes a pool of unlabelled data and an annotation budget. We investigate the effectiveness and interplay between several AL and CL algorithms in the domain, class and task-incremental scenarios. Our experiments reveal the trade-off between two contrasting goals of not forgetting the old knowledge and the ability to quickly learn new knowledge in CL and AL, respectively. While conditioning the AL query strategy on the annotations collected for the previous tasks leads to improved task performance on the domain and task incremental learning, our proposed forgetting-learning profile suggests a gap in balancing the effect of AL and CL for the class-incremental scenario.
Egglog Python: A Pythonic Library for E-graphs
Venue: arXiv cs.PL 2024
Year: 2024

E-graphs have emerged as a versatile data structure with applications in synthesis, optimization, and verification through techniques such as equality saturation. This paper introduces Python bindings for the experimental egglog library (previously called egg-smol), which aims to bring the benefits of e-graphs to the Python ecosystem. The bindings offer a high-level, Pythonic API providing an accessible and familiar interface for Python users. By integrating e-graph techniques with Python, we hope to enable collaboration and innovation across various domains in the scientific computing and machine learning communities. We discuss the advantages of using Python bindings for both Python and existing egg-smol users, as well as possible future directions for development.
From Muller to Parity and Rabin Automata: Optimal Transformations Preserving (History) Determinism
Venue: arXiv cs.FL 2024
Year: 2024

We study transformations of automata and games using Muller conditions into equivalent ones using parity or Rabin conditions. We present two transformations, one that turns a deterministic Muller automaton into an equivalent deterministic parity automaton, and another that provides an equivalent history-deterministic Rabin automaton. We show a strong optimality result: the obtained automata are minimal amongst those that can be derived from the original automaton by duplication of states. We introduce the notions of locally bijective morphisms and history-deterministic mappings to formalise the correctness and optimality of these transformations. The proposed transformations are based on a novel structure, called the alternating cycle decomposition, inspired by and extending Zielonka trees. In addition to providing optimal transformations of automata, the alternating cycle decomposition offers fundamental information on their structure. We use this information to give crisp characterisations on the possibility of relabelling automata with different acceptance conditions and to perform a systematic study of a normal form for parity automata.
Which Games are Unaffected by Absolute Commitments?
Venue: arXiv cs.GT 2024
Year: 2024

We identify a subtle security issue that impacts mechanism design in scenarios in which agents can absolutely commit to strategies. Absolute commitments allow the strategy of an agent to depend on the commitments made by the other agents. This changes fundamental game-theoretic assumptions by inducing a meta-game in which agents choose which strategies they commit to. We say that a game that is unaffected by such commitments is Stackelberg resilient and show that computing it is intractible in general, although it can be computed efficiently for two-player games of perfect information. We show the intuitive, but technically non-trivial result, that, if a game is resilient when some number of players have the capacity to make commitments, it is also resilient when these commitments are available to fewer players. We demonstrate the non-triviality of Stackelberg resilience by analyzing two escrow mechanisms from the literature. These mechanisms have the same intended functionality, but we show that only one is Stackelberg resilient. Our model is particularly relevant in Web3 scenarios, where these absolute commitments can be realized by the automated and irrevocable nature of smart contracts. Our work highlights an important issue in ensuring the secure design of Web3. In particular, our work suggests that smart contracts already deployed on major blockchains may be susceptible to these attacks.
Signal vs Noise in Eye-tracking Data: Biometric Implications and Identity Information Across Frequencies
Venue: arXiv cs.HC 2024
Year: 2024

Prior research states that frequencies below 75 Hz in eye-tracking data represent the primary eye movement termed ``signal'' while those above 75 Hz are deemed ``noise''. This study examines the biometric significance of this signal-noise distinction and its privacy implications. There are important individual differences in a person's eye movement, which lead to reliable biometric performance in the ``signal'' part. Despite minimal eye-movement information in the ``noise'' recordings, there might be significant individual differences. Our results confirm the ``signal'' predominantly contains identity-specific information, yet the ``noise'' also possesses unexpected identity-specific data. This consistency holds for both short-(approx. 20 min) and long-term (approx. 1 year) biometric evaluations. Understanding the location of identity data within the eye movement spectrum is essential for privacy preservation.
Set-Type Belief Propagation with Applications to Poisson Multi-Bernoulli SLAM
Venue: arXiv cs.AI 2024
Year: 2024

Belief propagation (BP) is a useful probabilistic inference algorithm for efficiently computing approximate marginal probability densities of random variables. However, in its standard form, BP is only applicable to the vector-type random variables with a fixed and known number of vector elements, while certain applications rely on RFSs with an unknown number of vector elements. In this paper, we develop BP rules for factor graphs defined on sequences of RFSs where each RFS has an unknown number of elements, with the intention of deriving novel inference methods for RFSs. Furthermore, we show that vector-type BP is a special case of set-type BP, where each RFS follows the Bernoulli process. To demonstrate the validity of developed set-type BP, we apply it to the PMB filter for SLAM, which naturally leads to new set-type BP-mapping, SLAM, multi-target tracking, and simultaneous localization and tracking filters. Finally, we explore the relationships between the vector-type BP and the proposed set-type BP PMB-SLAM implementations and show a performance gain of the proposed set-type BP PMB-SLAM filter in comparison with the vector-type BP-SLAM filter.
From Zero to Hero: Harnessing Transformers for Biomedical Named Entity Recognition in Zero- and Few-shot Contexts
Venue: arXiv cs.CL 2024
Year: 2024

Supervised named entity recognition (NER) in the biomedical domain depends on large sets of annotated texts with the given named entities. The creation of such datasets can be time-consuming and expensive, while extraction of new entities requires additional annotation tasks and retraining the model. To address these challenges, this paper proposes a method for zero- and few-shot NER in the biomedical domain. The method is based on transforming the task of multi-class token classification into binary token classification and pre-training on a large amount of datasets and biomedical entities, which allow the model to learn semantic relations between the given and potentially novel named entity labels. We have achieved average F1 scores of 35.44% for zero-shot NER, 50.10% for one-shot NER, 69.94% for 10-shot NER, and 79.51% for 100-shot NER on 9 diverse evaluated biomedical entities with fine-tuned PubMedBERT-based model. The results demonstrate the effectiveness of the proposed method for recognizing new biomedical entities with no or limited number of examples, outperforming previous transformer-based methods, and being comparable to GPT3-based models using models with over 1000 times fewer parameters. We make models and developed code publicly available.
Flame: Simplifying Topology Extension in Federated Learning
Venue: arXiv cs.LG 2024
Year: 2024

Distributed machine learning approaches, including a broad class of federated learning (FL) techniques, present a number of benefits when deploying machine learning applications over widely distributed infrastructures. The benefits are highly dependent on the details of the underlying machine learning topology, which specifies the functionality executed by the participating nodes, their dependencies and interconnections. Current systems lack the flexibility and extensibility necessary to customize the topology of a machine learning deployment. We present Flame, a new system that provides flexibility of the topology configuration of distributed FL applications around the specifics of a particular deployment context, and is easily extensible to support new FL architectures. Flame achieves this via a new high-level abstraction Topology Abstraction Graphs (TAGs). TAGs decouple the ML application logic from the underlying deployment details, making it possible to specialize the application deployment with reduced development effort. Flame is released as an open source project, and its flexibility and extensibility support a variety of topologies and mechanisms, and can facilitate the development of new FL methodologies.
The Multi-cluster Two-Wave Fading Model
Venue: arXiv cs.IT 2024
Year: 2024

We introduce and characterize the Multi-cluster Two-Wave (MTW) fading model, which generalizes \textit{both} the Durgin's Two-Wave with Diffuse Power (TWDP) and the $κ$-$μ$ models under a common umbrella. The MTW model consists of an arbitrary number of clusters of waves each of which may include one or two dominant (specular) components. The chief probability functions of the MTW fading model are obtained, including the probability density function, the cumulative distribution function and the generalized moment-generating function. \textcolor{black}{The proposed model is empirically validated using channel measurements in the sub-THz band and} a number of applications are exemplified, including the outage probability in noise-limited and interference-limited scenarios and the energy detection probability. %Exact expressions for the outage capacity are also obtained. A composite Inverse Gamma (IG)/MTW model is also investigated, thus extending the proposed propagation model to include shadowing.