OpenRaiser/Intern-Atlas huggingface dataset:
Deep Reinforcement Learning with Dynamic Graphs for Adaptive Informative Path Planning
Venue: arXiv cs.RO 2024
Year: 2024

Autonomous robots are often employed for data collection due to their efficiency and low labour costs. A key task in robotic data acquisition is planning paths through an initially unknown environment to collect observations given platform-specific resource constraints, such as limited battery life. Adaptive online path planning in 3D environments is challenging due to the large set of valid actions and the presence of unknown occlusions. To address these issues, we propose a novel deep reinforcement learning approach for adaptively replanning robot paths to map targets of interest in unknown 3D environments. A key aspect of our approach is a dynamically constructed graph that restricts planning actions local to the robot, allowing us to react to newly discovered static obstacles and targets of interest. For replanning, we propose a new reward function that balances between exploring the unknown environment and exploiting online-discovered targets of interest. Our experiments show that our method enables more efficient target discovery compared to state-of-the-art learning and non-learning baselines. We also showcase our approach for orchard monitoring using an unmanned aerial vehicle in a photorealistic simulator. We open-source our code and model at: https://github.com/dmar-bonn/ipp-rl-3d.
What Values Do ImageNet-trained Classifiers Enact?
Venue: arXiv cs.CY 2024
Year: 2024

We identify "values" as actions that classifiers take that speak to open questions of significant social concern. Investigating a classifier's values builds on studies of social bias that uncover how classifiers participate in social processes beyond their creators' forethought. In our case, this participation involves what counts as nutritious, what it means to be modest, and more. Unlike AI social bias, however, a classifier's values are not necessarily morally loathsome. Attending to image classifiers' values can facilitate public debate and introspection about the future of society. To substantiate these claims, we report on an extensive examination of both ImageNet training/validation data and ImageNet-trained classifiers with custom testing data. We identify perceptual decision boundaries in 118 categories that address open questions in society, and through quantitative testing of rival datasets we find that ImageNet-trained classifiers enact at least 7 values through their perceptual decisions. To contextualize these results, we develop a conceptual framework that integrates values, social bias, and accuracy, and we describe a rhetorical method for identifying how context affects the values that a classifier enacts. We also discover that classifier performance does not straightforwardly reflect the proportions of subgroups in a training set. Our findings bring a rich sense of the social world to ML researchers that can be applied to other domains beyond computer vision.
Simple inexpensive vertex and edge invariants distinguishing dataset strongly regular graphs
Venue: arXiv cs.CC 2024
Year: 2024

While standard Weisfeiler-Leman vertex labels are not able to distinguish even vertices of regular graphs, there is proposed and tested family of inexpensive polynomial time vertex and edge invariants, distinguishing much more difficult SRGs (strongly regular graphs), also often their vertices. Among 43717 SRGs from dataset by Edward Spence, proposed vertex invariants alone were able to distinguish all but 4 pairs of graphs, which were easily distinguished by further application of proposed edge invariants. Specifically, proposed vertex invariants are traces or sorted diagonals of $(A|_{N_a})^p$ adjacency matrix $A$ restricted to $N_a$ neighborhood of vertex $a$, already for $p=3$ distinguishing all SRGs from 6 out of 13 sets in this dataset, 8 if adding $p=4$. Proposed edge invariants are analogously traces or diagonals of powers of $\bar{A}_{ab,cd}=A_{ab} A_{ac} A_{bd}$, nonzero for $(a,b)$ being edges. As SRGs are considered the most difficult cases for graph isomorphism problem, such algebraic-combinatorial invariants bring hope that this problem is polynomial time.
Cayley hashing with cookies
Venue: arXiv cs.CR 2024
Year: 2024

Cayley hash functions are based on a simple idea of using a pair of semigroup elements, A and B, to hash the 0 and 1 bit, respectively, and then to hash an arbitrary bit string in the natural way, by using multiplication of elements in the semigroup. The main advantage of Cayley hash functions compared to, say, hash functions in the SHA family is that when an already hashed document is amended, one does not have to hash the whole amended document all over again, but rather hash just the amended part and then multiply the result by the hash of the original document. Some authors argued that this may be a security hazard, specifically that this property may facilitate finding a second preimage by splitting a long bit string into shorter pieces. In this paper, we offer a way to get rid of this alleged disadvantage and keep the advantages at the same time. We call this method ``Cayley hashing with cookies" using terminology borrowed from the theory of random walks in a random environment. For the platform semigroup, we use 2x2 matrices over F_p.
Distributed Fair Assignment and Rebalancing for Mobility-on-Demand Systems via an Auction-based Method
Venue: arXiv cs.FL 2024
Year: 2024

In this paper, we consider fair assignment of complex requests for Mobility-On-Demand systems. We model the transportation requests as temporal logic formulas that must be satisfied by a fleet of vehicles. We require that the assignment of requests to vehicles is performed in a distributed manner based only on communication between vehicles while ensuring fair allocation. Our approach to the vehicle-request assignment problem is based on a distributed auction scheme with no centralized bidding that leverages utility history correction of bids to improve fairness. Complementarily, we propose a rebalancing scheme that employs rerouting vehicles to more rewarding areas to increase the potential future utility and ensure a fairer utility distribution. We adopt the max-min and deviation of utility as the two criteria for fairness. We demonstrate the methods in the mid-Manhattan map with a large number of requests generated in different probability settings. We show that we increase the fairness between vehicles based on the fairness criteria without degenerating the servicing quality.
When the Body Became Data: Historical Data Cultures and Anatomical Illustration
Venue: arXiv cs.HC 2024
Year: 2024

With changing attitudes around knowledge, medicine, art, and technology, the human body has become a source of information and, ultimately, shareable and analyzable data. Centuries of illustrations and visualizations of the body occur within particular historical, social, and political contexts. These contexts are enmeshed in different so-called data cultures: ways that data, knowledge, and information are conceptualized and collected, structured and shared. In this work, we explore how information about the body was collected as well as the circulation, impact, and persuasive force of the resulting images. We show how mindfulness of data cultural influences remain crucial for today's designers, researchers, and consumers of visualizations. We conclude with a call for the field to reflect on how visualizations are not timeless and contextless mirrors on objective data, but as much a product of our time and place as the visualizations of the past.
Smooth real-time motion planning based on a cascade dual-quaternion screw-geometry MPC
Venue: arXiv cs.RO 2024
Year: 2024

This paper investigates the tracking problem of a smooth coordinate-invariant trajectory using dual quaternion algebra. The proposed architecture consists of a cascade structure in which the outer-loop MPC performs real-time smoothing of the manipulator's end-effector twist while an inner-loop kinematic controller ensures tracking of the instantaneous desired end-effector pose. Experiments on a $7$-DoF Franka Emika Panda robotic manipulator validate the proposed method demonstrating its application to constraint the robot twists, accelerations and jerks within prescribed bounds.
PAC Learnability under Explanation-Preserving Graph Perturbations
Venue: arXiv cs.LG 2024
Year: 2024

Graphical models capture relations between entities in a wide range of applications including social networks, biology, and natural language processing, among others. Graph neural networks (GNN) are neural models that operate over graphs, enabling the model to leverage the complex relationships and dependencies in graph-structured data. A graph explanation is a subgraph which is an `almost sufficient' statistic of the input graph with respect to its classification label. Consequently, the classification label is invariant, with high probability, to perturbations of graph edges not belonging to its explanation subgraph. This work considers two methods for leveraging such perturbation invariances in the design and training of GNNs. First, explanation-assisted learning rules are considered. It is shown that the sample complexity of explanation-assisted learning can be arbitrarily smaller than explanation-agnostic learning. Next, explanation-assisted data augmentation is considered, where the training set is enlarged by artificially producing new training samples via perturbation of the non-explanation edges in the original training set. It is shown that such data augmentation methods may improve performance if the augmented data is in-distribution, however, it may also lead to worse sample complexity compared to explanation-agnostic learning rules if the augmented data is out-of-distribution. Extensive empirical evaluations are provided to verify the theoretical analysis.
A Roadmap to Pluralistic Alignment
Venue: arXiv cs.AI 2024
Year: 2024

With increased power and prevalence of AI systems, it is ever more critical that AI systems are designed to serve all, i.e., people with diverse values and perspectives. However, aligning models to serve pluralistic human values remains an open research question. In this piece, we propose a roadmap to pluralistic alignment, specifically using language models as a test bed. We identify and formalize three possible ways to define and operationalize pluralism in AI systems: 1) Overton pluralistic models that present a spectrum of reasonable responses; 2) Steerably pluralistic models that can steer to reflect certain perspectives; and 3) Distributionally pluralistic models that are well-calibrated to a given population in distribution. We also formalize and discuss three possible classes of pluralistic benchmarks: 1) Multi-objective benchmarks, 2) Trade-off steerable benchmarks, which incentivize models to steer to arbitrary trade-offs, and 3) Jury-pluralistic benchmarks which explicitly model diverse human ratings. We use this framework to argue that current alignment techniques may be fundamentally limited for pluralistic AI; indeed, we highlight empirical evidence, both from our own experiments and from other work, that standard alignment procedures might reduce distributional pluralism in models, motivating the need for further research on pluralistic alignment.
Language-Based Augmentation to Address Shortcut Learning in Object Goal Navigation
Venue: arXiv cs.RO 2024
Year: 2024

Deep Reinforcement Learning (DRL) has shown great potential in enabling robots to find certain objects (e.g., `find a fridge') in environments like homes or schools. This task is known as Object-Goal Navigation (ObjectNav). DRL methods are predominantly trained and evaluated using environment simulators. Although DRL has shown impressive results, the simulators may be biased or limited. This creates a risk of shortcut learning, i.e., learning a policy tailored to specific visual details of training environments. We aim to deepen our understanding of shortcut learning in ObjectNav, its implications and propose a solution. We design an experiment for inserting a shortcut bias in the appearance of training environments. As a proof-of-concept, we associate room types to specific wall colors (e.g., bedrooms with green walls), and observe poor generalization of a state-of-the-art (SOTA) ObjectNav method to environments where this is not the case (e.g., bedrooms with blue walls). We find that shortcut learning is the root cause: the agent learns to navigate to target objects, by simply searching for the associated wall color of the target object's room. To solve this, we propose Language-Based (L-B) augmentation. Our key insight is that we can leverage the multimodal feature space of a Vision-Language Model (VLM) to augment visual representations directly at the feature-level, requiring no changes to the simulator, and only an addition of one layer to the model. Where the SOTA ObjectNav method's success rate drops 69%, our proposal has only a drop of 23%.
Opening the AI black box: program synthesis via mechanistic interpretability
Venue: arXiv cs.LG 2024
Year: 2024

We present MIPS, a novel method for program synthesis based on automated mechanistic interpretability of neural networks trained to perform the desired task, auto-distilling the learned algorithm into Python code. We test MIPS on a benchmark of 62 algorithmic tasks that can be learned by an RNN and find it highly complementary to GPT-4: MIPS solves 32 of them, including 13 that are not solved by GPT-4 (which also solves 30). MIPS uses an integer autoencoder to convert the RNN into a finite state machine, then applies Boolean or integer symbolic regression to capture the learned algorithm. As opposed to large language models, this program synthesis technique makes no use of (and is therefore not limited by) human training data such as algorithms and code from GitHub. We discuss opportunities and challenges for scaling up this approach to make machine-learned models more interpretable and trustworthy.
A Light-weight and Unsupervised Method for Near Real-time Behavioral Analysis using Operational Data Measurement
Venue: arXiv cs.DC 2024
Year: 2024

Monitoring the status of large computing systems is essential to identify unexpected behavior and improve their performance and uptime. However, due to the large-scale and distributed design of such computing systems as well as a large number of monitoring parameters, automated monitoring methods should be applied. Such automatic monitoring methods should also have the ability to adapt themselves to the continuous changes in the computing system. In addition, they should be able to identify behavioral anomalies in useful time, to perform appropriate reactions. This work proposes a general lightweight and unsupervised method for near real-time anomaly detection using operational data measurement on large computing systems. The proposed model requires as little as 4 hours of data and 50 epochs for each training process to accurately resemble the behavioral pattern of computing systems.
Enhancing Accessibility of Rural Populations through Vehicle-based Services
Venue: arXiv cs.CY 2024
Year: 2024

Improving access to essential public services like healthcare and education is crucial for human development, particularly in rural Sub-Saharan Africa. However, limited reliable transportation and sparse public facilities present significant challenges. Mobile facilities like mobile clinics offer a cost-effective solution to enhance spatial accessibility for the rural population.Public authorities require detailed demand distribution data to allocate resources efficiently and maximize the impact of mobile facilities. This includes determining optimal vehicle service stop locations and estimating operational costs. Our integrated approach utilizes GIS data and an accessibility scaling factor to assess spatial accessibility for rural populations. We tailor demand structures to account for remote and underserved populations. To reduce average travel distances to 5 km, we apply a clustering algorithm and optimize vehicle service stop locations. In a case study in rural Ethiopia, focusing on four key public services, our analysis demonstrates that mobile facilities can address 39-62\% of unmet demand, even in areas with widely dispersed populations. This approach aids decision-makers, including fleet operators, policymakers, and public authorities in Sub-Saharan Africa, during project evaluation and planning for mobile facilities. By enhancing spatial accessibility and optimizing resource allocation, our methodology contributes to the effective delivery of essential public services to underserved populations.
Large Language Model for Table Processing: A Survey
Venue: arXiv cs.AI 2024
Year: 2024

Tables, typically two-dimensional and structured to store large amounts of data, are essential in daily activities like database queries, spreadsheet manipulations, web table question answering, and image table information extraction. Automating these table-centric tasks with Large Language Models (LLMs) or Visual Language Models (VLMs) offers significant public benefits, garnering interest from academia and industry. This survey provides a comprehensive overview of table-related tasks, examining both user scenarios and technical aspects. It covers traditional tasks like table question answering as well as emerging fields such as spreadsheet manipulation and table data analysis. We summarize the training techniques for LLMs and VLMs tailored for table processing. Additionally, we discuss prompt engineering, particularly the use of LLM-powered agents, for various table-related tasks. Finally, we highlight several challenges, including diverse user input when serving and slow thinking using chain-of-thought.
Best Practices for Text Annotation with Large Language Models
Venue: arXiv cs.CL 2024
Year: 2024

Large Language Models (LLMs) have ushered in a new era of text annotation, as their ease-of-use, high accuracy, and relatively low costs have meant that their use has exploded in recent months. However, the rapid growth of the field has meant that LLM-based annotation has become something of an academic Wild West: the lack of established practices and standards has led to concerns about the quality and validity of research. Researchers have warned that the ostensible simplicity of LLMs can be misleading, as they are prone to bias, misunderstandings, and unreliable results. Recognizing the transformative potential of LLMs, this paper proposes a comprehensive set of standards and best practices for their reliable, reproducible, and ethical use. These guidelines span critical areas such as model selection, prompt engineering, structured prompting, prompt stability analysis, rigorous model validation, and the consideration of ethical and legal implications. The paper emphasizes the need for a structured, directed, and formalized approach to using LLMs, aiming to ensure the integrity and robustness of text annotation practices, and advocates for a nuanced and critical engagement with LLMs in social scientific research.
Threats and Limitations of Terrestrial Broadcast Attacks
Venue: arXiv cs.CR 2024
Year: 2024

The DVB standard does not mandate the use of authentication and integrity protection for transport streams. This allows malicious third parties to replace legitimate broadcasts by overpowering terrestrial transmissions. The rogue signal can then deliver a malicious broadcast stream to exploit security vulnerabilities on Smart TVs (STVs) in range. We implemented a proof-of-concept attack based on a malicious Hybrid Broadcast Broadband TV app, able to acquire permanent system-level access to an STV over the air, in less than 10 s. These attacks, however, are severely limited in range due to required co-channel protection ratios (CCPRs), which is in direct contradiction to previous publications. We present evidence for these limitations in form of laboratory experiments, extensive simulations, and field measurements. To this end, we developed an automated, low-cost method for CCPR determination, as well as a method for non-disruptive attack range measurements based on a gap filler and the resulting channel impulse response.
UEyes: An Eye-Tracking Dataset across User Interface Types
Venue: arXiv cs.HC 2024
Year: 2024

Different types of user interfaces differ significantly in the number of elements and how they are displayed. To examine how such differences affect the way users look at UIs, we collected and analyzed a large eye-tracking-based dataset, UEyes (62 participants, 1,980 UI screenshots, near 20K eye movement sequences), covering four major UI types: webpage, desktop UI, mobile UI, and poster. Furthermore, we analyze and discuss the differences in important factors, such as color, location, and gaze direction across UI types, individual viewing strategies and potential future directions. This position paper is a derivative of our recent paper with a particular focus on the UEyes dataset.
Universal Neural Functionals
Venue: arXiv cs.LG 2024
Year: 2024

A challenging problem in many modern machine learning tasks is to process weight-space features, i.e., to transform or extract information from the weights and gradients of a neural network. Recent works have developed promising weight-space models that are equivariant to the permutation symmetries of simple feedforward networks. However, they are not applicable to general architectures, since the permutation symmetries of a weight space can be complicated by recurrence or residual connections. This work proposes an algorithm that automatically constructs permutation equivariant models, which we refer to as universal neural functionals (UNFs), for any weight space. Among other applications, we demonstrate how UNFs can be substituted into existing learned optimizer designs, and find promising improvements over prior methods when optimizing small image classifiers and language models. Our results suggest that learned optimizers can benefit from considering the (symmetry) structure of the weight space they optimize. We open-source our library for constructing UNFs at https://github.com/AllanYangZhou/universal_neural_functional.
Automated Data-Driven Discovery of Material Models Based on Symbolic Regression: A Case Study on Human Brain Cortex
Venue: arXiv cs.SC 2024
Year: 2024

We introduce a data-driven framework to automatically identify interpretable and physically meaningful hyperelastic constitutive models from sparse data. Leveraging symbolic regression, an algorithm based on genetic programming, our approach generates elegant hyperelastic models that achieve accurate data fitting through parsimonious mathematic formulae, while strictly adhering to hyperelasticity constraints such as polyconvexity. Our investigation spans three distinct hyperelastic models -- invariant-based, principal stretch-based, and normal strain-based -- and highlights the versatility of symbolic regression. We validate our new approach using synthetic data from five classic hyperelastic models and experimental data from the human brain to demonstrate algorithmic efficacy. Our results suggest that our symbolic regression robustly discovers accurate models with succinct mathematic expressions in invariant-based, stretch-based, and strain-based scenarios. Strikingly, the strain-based model exhibits superior accuracy, while both stretch- and strain-based models effectively capture the nonlinearity and tension-compression asymmetry inherent to human brain tissue. Polyconvexity examinations affirm the rigor of convexity within the training regime and demonstrate excellent extrapolation capabilities beyond this regime for all three models. However, the stretch-based models raise concerns regarding potential convexity loss under large deformations. Finally, robustness tests on noise-embedded data underscore the reliability of our symbolic regression algorithms. Our study confirms the applicability and accuracy of symbolic regression in the automated discovery of hyperelastic models for the human brain and gives rise to a wide variety of applications in other soft matter systems.