OpenRaiser/Intern-Atlas huggingface dataset:
Virtual Omnidirectional Perception for Downwash Prediction within a Team of Nano Multirotors Flying in Close Proximity
Venue: arXiv cs.RO 2023
Year: 2023

Teams of flying robots can be used for inspection, delivery, and construction tasks, in which they might be required to fly very close to each other. In such close-proximity cases, nonlinear aerodynamic effects can cause catastrophic crashes, necessitating each robots' awareness of the surrounding. Existing approaches rely on multiple, expensive or heavy perception sensors. Such perception methods are impractical to use on nano multirotors that are constrained with respect to weight, computation, and price. Instead, we propose to use the often ignored yaw degree-of-freedom of multirotors to spin a single, cheap and lightweight monocular camera at a high angular rate for omnidirectional awareness of the neighboring robots. We provide a dataset collected with real-world physical flights as well as with 3D-rendered scenes and compare two existing learning-based methods in different settings with respect to success rate, relative position estimation, and downwash prediction accuracy. We demonstrate that our proposed spinning camera is capable of predicting the presence of aerodynamic downwash with an $F_1$ score of over 80% in a challenging swapping task.
A Meta-Evaluation of Faithfulness Metrics for Long-Form Hospital-Course Summarization
Venue: arXiv cs.CL 2023
Year: 2023

Long-form clinical summarization of hospital admissions has real-world significance because of its potential to help both clinicians and patients. The faithfulness of summaries is critical to their safe usage in clinical settings. To better understand the limitations of abstractive systems, as well as the suitability of existing evaluation metrics, we benchmark faithfulness metrics against fine-grained human annotations for model-generated summaries of a patient's Brief Hospital Course. We create a corpus of patient hospital admissions and summaries for a cohort of HIV patients, each with complex medical histories. Annotators are presented with summaries and source notes, and asked to categorize manually highlighted summary elements (clinical entities like conditions and medications as well as actions like "following up") into one of three categories: ``Incorrect,'' ``Missing,'' and ``Not in Notes.'' We meta-evaluate a broad set of proposed faithfulness metrics and, across metrics, explore the importance of domain adaptation (e.g. the impact of in-domain pre-training and metric fine-tuning), the use of source-summary alignments, and the effects of distilling a single metric from an ensemble of pre-existing metrics. Off-the-shelf metrics with no exposure to clinical text correlate well yet overly rely on summary extractiveness. As a practical guide to long-form clinical narrative summarization, we find that most metrics correlate best to human judgments when provided with one summary sentence at a time and a minimal set of relevant source context.
Combining static analysis and dynamic symbolic execution in a toolchain to detect fault injection vulnerabilities
Venue: arXiv cs.SE 2023
Year: 2023

Certification through auditing allows to ensure that critical embedded systems are secure. This entails reviewing their critical components and checking for dangerous execution paths. This latter task requires the use of specialized tools which allow to explore and replay executions but are also difficult to use effectively within the context of the audit, where time and knowledge of the code are limited. Fault analysis is especially tricky as the attacker may actively influence execution, rendering some common methods unusable and increasing the number of possible execution paths exponentially. In this work, we present a new method which mitigates these issues by reducing the number of fault injection points considered to only the most relevant ones relatively to some security properties. We use fast and robust static analysis to detect injection points and assert their impactfulness. A more precise dynamic/symbolic method is then employed to validate attack paths. This way the insight required to find attacks is reduced and dynamic methods can better scale to realistically sized programs. Our method is implemented into a toolchain based on Frama-C and KLEE and validated on WooKey, a case-study proposed by the National Cybersecurity Agency of France.
Disturbance Estimation for High-Degree-of-Freedom Euler-Lagrangian Systems Using Sliding Mode Observer without Matching Conditions
Venue: arXiv cs.RO 2023
Year: 2023

This paper proposes a novel observer-based disturbance estimation method for high degree-of-freedom Euler-Lagrangian systems using an unknown input-output (UIO) sliding mode observer (SMO). Different from the previous SMO methods, this approach does not assume the matching condition of the disturbances. Besides, compared to the conventional disturbance estimation methods, the proposed method does not require the calculation of the inverse inertia matrices and accurate measurement of system velocities. This advantage resolves the concerns of heavy computational load and amplified noise for practical problems like external torque estimation of high degree-of-freedom manipulators in safe human-robot collaboration. We achieve this by defining a novel linearized model for the Euler-Lagrangian system and designing a sliding-mode-based disturbance observer. The estimation precision of the observer is ensured by the Lyapunov-based stability proof and the equivalent control theory of sliding mode. At the end of this paper, the method is implemented on a seven-degree-of-freedom robot manipulator, and the experimental results confirm its decent performance and potential to be applied in practice.
Notable: On-the-fly Assistant for Data Storytelling in Computational Notebooks
Venue: arXiv cs.HC 2023
Year: 2023

Computational notebooks are widely used for data analysis. Their interleaved displays of code and execution results (e.g., visualizations) are welcomed since they enable iterative analysis and preserve the exploration process. However, the communication of data findings remains challenging in computational notebooks. Users have to carefully identify useful findings from useless ones, document them with texts and visual embellishments, and then organize them in different tools. Such workflow greatly increases their workload, according to our interviews with practitioners. To address the challenge, we designed Notable to offer on-the-fly assistance for data storytelling in computational notebooks. It provides intelligent support to minimize the work of documenting and organizing data findings and diminishes the cost of switching between data exploration and storytelling. To evaluate Notable, we conducted a user study with 12 data workers. The feedback from user study participants verifies its effectiveness and usability.
Flow-Based Integrated Assignment and Path-Finding for Mobile Robot Sorting Systems
Venue: arXiv cs.RO 2023
Year: 2023

Express companies are deploying more robotic sorting systems, where mobile robots are used to sort incoming parcels by destination. In this study, we propose an integrated assignment and path-finding method for robots in such sorting systems. The method has two parts: offline and online. In the offline part, we represent the system as a traffic flow network, develop an approximate delay function using stochastic models, and solve the min-cost network flow problem. In the online part, robots are guided through the system according to the calculated optimal flow split probability. The online calculation of the method is decentralized and has linear complexity. Our method outperforms fast multi-agent path planning algorithms like prioritized planning because such algorithms lead to stochastic user equilibrium traffic assignment. In contrast, our method gives the approximated system-optimal traffic assignment. According to our simulations, our method can achieve 10%--20% higher throughput than zoning or random assignment. We also show that our method is robust even if the initial demand estimation is inaccurate.
NEPHELE: A Neural Platform for Highly Realistic Cloud Radiance Rendering
Venue: arXiv cs.GR 2023
Year: 2023

We have recently seen tremendous progress in neural rendering (NR) advances, i.e., NeRF, for photo-real free-view synthesis. Yet, as a local technique based on a single computer/GPU, even the best-engineered Instant-NGP or i-NGP cannot reach real-time performance when rendering at a high resolution, and often requires huge local computing resources. In this paper, we resort to cloud rendering and present NEPHELE, a neural platform for highly realistic cloud radiance rendering. In stark contrast with existing NR approaches, our NEPHELE allows for more powerful rendering capabilities by combining multiple remote GPUs and facilitates collaboration by allowing multiple people to view the same NeRF scene simultaneously. We introduce i-NOLF to employ opacity light fields for ultra-fast neural radiance rendering in a one-query-per-ray manner. We further resemble the Lumigraph with geometry proxies for fast ray querying and subsequently employ a small MLP to model the local opacity lumishperes for high-quality rendering. We also adopt Perfect Spatial Hashing in i-NOLF to enhance cache coherence. As a result, our i-NOLF achieves an order of magnitude performance gain in terms of efficiency than i-NGP, especially for the multi-user multi-viewpoint setting under cloud rendering scenarios. We further tailor a task scheduler accompanied by our i-NOLF representation and demonstrate the advance of our methodological design through a comprehensive cloud platform, consisting of a series of cooperated modules, i.e., render farms, task assigner, frame composer, and detailed streaming strategies. Using such a cloud platform compatible with neural rendering, we further showcase the capabilities of our cloud radiance rendering through a series of applications, ranging from cloud VR/AR rendering.
Marpa and nullable symbols
Venue: arXiv cs.CL 2023
Year: 2023

The Marpa parser was intended to make the best results in the academic literature on Earley's algorithm available as a practical general parser. Earley-based parsers have had issues handling nullable symbols. Initially, we dealt with nullable symbols by following the approach in Aycock and Horspool's 2002 paper. This paper reports our experience with the approach in that paper, and the approach to handling nullables that we settled on in reaction to that experience.
Predicted Embedding Power Regression for Large-Scale Out-of-Distribution Detection
Venue: arXiv cs.CV 2023
Year: 2023

Out-of-distribution (OOD) inputs can compromise the performance and safety of real world machine learning systems. While many methods exist for OOD detection and work well on small scale datasets with lower resolution and few classes, few methods have been developed for large-scale OOD detection. Existing large-scale methods generally depend on maximum classification probability, such as the state-of-the-art grouped softmax method. In this work, we develop a novel approach that calculates the probability of the predicted class label based on label distributions learned during the training process. Our method performs better than current state-of-the-art methods with only a negligible increase in compute cost. We evaluate our method against contemporary methods across $14$ datasets and achieve a statistically significant improvement with respect to AUROC (84.2 vs 82.4) and AUPR (96.2 vs 93.7).
Computing with Categories in Machine Learning
Venue: arXiv cs.LG 2023
Year: 2023

Category theory has been successfully applied in various domains of science, shedding light on universal principles unifying diverse phenomena and thereby enabling knowledge transfer between them. Applications to machine learning have been pursued recently, and yet there is still a gap between abstract mathematical foundations and concrete applications to machine learning tasks. In this paper we introduce DisCoPyro as a categorical structure learning framework, which combines categorical structures (such as symmetric monoidal categories and operads) with amortized variational inference, and can be applied, e.g., in program learning for variational autoencoders. We provide both mathematical foundations and concrete applications together with comparison of experimental performance with other models (e.g., neuro-symbolic models). We speculate that DisCoPyro could ultimately contribute to the development of artificial general intelligence.
SALSA PICANTE: a machine learning attack on LWE with binary secrets
Venue: arXiv cs.CR 2023
Year: 2023

Learning with Errors (LWE) is a hard math problem underpinning many proposed post-quantum cryptographic (PQC) systems. The only PQC Key Exchange Mechanism (KEM) standardized by NIST is based on module~LWE, and current publicly available PQ Homomorphic Encryption (HE) libraries are based on ring LWE. The security of LWE-based PQ cryptosystems is critical, but certain implementation choices could weaken them. One such choice is sparse binary secrets, desirable for PQ HE schemes for efficiency reasons. Prior work, SALSA, demonstrated a machine learning-based attack on LWE with sparse binary secrets in small dimensions ($n \le 128$) and low Hamming weights ($h \le 4$). However, this attack assumes access to millions of eavesdropped LWE samples and fails at higher Hamming weights or dimensions. We present PICANTE, an enhanced machine learning attack on LWE with sparse binary secrets, which recovers secrets in much larger dimensions (up to $n=350$) and with larger Hamming weights (roughly $n/10$, and up to $h=60$ for $n=350$). We achieve this dramatic improvement via a novel preprocessing step, which allows us to generate training data from a linear number of eavesdropped LWE samples ($4n$) and changes the distribution of the data to improve transformer training. We also improve the secret recovery methods of SALSA and introduce a novel cross-attention recovery mechanism allowing us to read off the secret directly from the trained models. While PICANTE does not threaten NIST's proposed LWE standards, it demonstrates significant improvement over SALSA and could scale further, highlighting the need for future investigation into machine learning attacks on LWE with sparse binary secrets.
EscherNet 101
Venue: arXiv cs.CV 2023
Year: 2023

A deep learning model, EscherNet 101, is constructed to categorize images of 2D periodic patterns into their respective 17 wallpaper groups. Beyond evaluating EscherNet 101 performance by classification rates, at a micro-level we investigate the filters learned at different layers in the network, capable of capturing second-order invariants beyond edge and curvature.
ConBaT: Control Barrier Transformer for Safe Policy Learning
Venue: arXiv cs.RO 2023
Year: 2023

Large-scale self-supervised models have recently revolutionized our ability to perform a variety of tasks within the vision and language domains. However, using such models for autonomous systems is challenging because of safety requirements: besides executing correct actions, an autonomous agent must also avoid the high cost and potentially fatal critical mistakes. Traditionally, self-supervised training mainly focuses on imitating previously observed behaviors, and the training demonstrations carry no notion of which behaviors should be explicitly avoided. In this work, we propose Control Barrier Transformer (ConBaT), an approach that learns safe behaviors from demonstrations in a self-supervised fashion. ConBaT is inspired by the concept of control barrier functions in control theory and uses a causal transformer that learns to predict safe robot actions autoregressively using a critic that requires minimal safety data labeling. During deployment, we employ a lightweight online optimization to find actions that ensure future states lie within the learned safe set. We apply our approach to different simulated control tasks and show that our method results in safer control policies compared to other classical and learning-based methods such as imitation learning, reinforcement learning, and model predictive control.
AI for Science: An Emerging Agenda
Venue: arXiv cs.AI 2023
Year: 2023

This report documents the programme and the outcomes of Dagstuhl Seminar 22382 "Machine Learning for Science: Bridging Data-Driven and Mechanistic Modelling". Today's scientific challenges are characterised by complexity. Interconnected natural, technological, and human systems are influenced by forces acting across time- and spatial-scales, resulting in complex interactions and emergent behaviours. Understanding these phenomena -- and leveraging scientific advances to deliver innovative solutions to improve society's health, wealth, and well-being -- requires new ways of analysing complex systems. The transformative potential of AI stems from its widespread applicability across disciplines, and will only be achieved through integration across research domains. AI for science is a rendezvous point. It brings together expertise from $\mathrm{AI}$ and application domains; combines modelling knowledge with engineering know-how; and relies on collaboration across disciplines and between humans and machines. Alongside technical advances, the next wave of progress in the field will come from building a community of machine learning researchers, domain experts, citizen scientists, and engineers working together to design and deploy effective AI tools. This report summarises the discussions from the seminar and provides a roadmap to suggest how different communities can collaborate to deliver a new wave of progress in AI and its application for scientific discovery.
Discovering substantive disagreement with review articles?
Venue: arXiv cs.DL 2023
Year: 2023

Disagreements help drive science. How does one identify and track them in scholarly literature? We ask the research question will searching review articles (RA) will be more time efficient for this purpose than searching non-review ones (NRA). This is especially so to the extent NRAs exceed RAs in a given field. We also discuss a metric for whether RAs report more substantive disagreements than NRAs.
SemEval-2023 Task 10: Explainable Detection of Online Sexism
Venue: arXiv cs.CL 2023
Year: 2023

Online sexism is a widespread and harmful phenomenon. Automated tools can assist the detection of sexism at scale. Binary detection, however, disregards the diversity of sexist content, and fails to provide clear explanations for why something is sexist. To address this issue, we introduce SemEval Task 10 on the Explainable Detection of Online Sexism (EDOS). We make three main contributions: i) a novel hierarchical taxonomy of sexist content, which includes granular vectors of sexism to aid explainability; ii) a new dataset of 20,000 social media comments with fine-grained labels, along with larger unlabelled datasets for model adaptation; and iii) baseline models as well as an analysis of the methods, results and errors for participant submissions to our task.
MEV in fixed gas price blockchains: Terra Classic as a case of study
Venue: arXiv cs.CR 2023
Year: 2023

Maximum extractable value (MEV) has been extensively studied. In most papers, the researchers have worked with the Ethereum blockchain almost exclusively. Even though, Ethereum and other blockchains have dynamic gas prices this is not the case for all blockchains; many of them have fixed gas prices. Extending the research to other blockchains with fixed gas price could broaden the scope of the existing studies on MEV. To our knowledge, there is not a vast understanding of MEV in fixed gas price blockchains. Therefore, we propose to study Terra Classic as an example to understand how MEV activities affect blockchains with fixed gas price. We first analysed the data from Terra Classic before the UST de-peg event in May 2022 and described the nature of the exploited arbitrage opportunities. We found more than 188K successful arbitrages, and most of them used UST as the initial token. The capital to perform the arbitrage was less than 1K UST in 50% of the cases, and 80% of the arbitrages had less than four swaps. Then, we explored the characteristics that attribute to higher MEV. We found that searchers who use more complex mechanisms, i.e. different contracts and accounts, made higher profits. Finally, we concluded that the most profitable searchers used a strategy of running bots in a multi-instance environment, i.e. running bots with different virtual machines. We measured the importance of the geographic distribution of the virtual machines that run the bots. We found that having good geographic coverage makes the difference between winning or losing the arbitrage opportunities. That is because, unlike MEV extraction in Ethereum, bots in fixed gas price blockchains are not battling a gas war; they are fighting in a latency war.
TMHOI: Translational Model for Human-Object Interaction Detection
Venue: arXiv cs.CV 2023
Year: 2023

Detecting human-object interactions (HOIs) is an intricate challenge in the field of computer vision. Existing methods for HOI detection heavily rely on appearance-based features, but these may not fully capture all the essential characteristics necessary for accurate detection. To overcome these challenges, we propose an innovative graph-based approach called TMGHOI (Translational Model for Human-Object Interaction Detection). Our method effectively captures the sentiment representation of HOIs by integrating both spatial and semantic knowledge. By representing HOIs as a graph, where the interaction components serve as nodes and their spatial relationships as edges. To extract crucial spatial and semantic information, TMGHOI employs separate spatial and semantic encoders. Subsequently, these encodings are combined to construct a knowledge graph that effectively captures the sentiment representation of HOIs. Additionally, the ability to incorporate prior knowledge enhances the understanding of interactions, further boosting detection accuracy. We conducted extensive evaluations on the widely-used HICO-DET datasets to demonstrate the effectiveness of TMGHOI. Our approach outperformed existing state-of-the-art graph-based methods by a significant margin, showcasing its potential as a superior solution for HOI detection. We are confident that TMGHOI has the potential to significantly improve the accuracy and efficiency of HOI detection. Its integration of spatial and semantic knowledge, along with its computational efficiency and practicality, makes it a valuable tool for researchers and practitioners in the computer vision community. As with any research, we acknowledge the importance of further exploration and evaluation on various datasets to establish the generalizability and robustness of our proposed method.