Miracle-dz/UniFAR-FTUs huggingface dataset:
NiCro: Purely Vision-based, Non-intrusive Cross-Device and Cross-Platform GUI Testing

To ensure app compatibility and smoothness of user experience across diverse devices and platforms, developers have to perform cross-device, cross-platform testing of their apps, which is laborious. There comes a recently increasing trend of using a record and replay approach to facilitate the testing process. However, the graphic user interface (GUI) of an app running on different devices and platforms differs dramatically. This complicates the record and replay process as the presence, appearance and layout of the GUI widgets in the recording phase and replaying phase can be inconsistent. Existing techniques resort to instrumenting into the underlying system to obtain the app metadata for widget identification and matching between various devices. But such intrusive practices are limited by the accessibility and accuracy of the metadata on different platforms. On the other hand, several recent works attempt to derive the GUI information by analyzing the GUI image. Nevertheless, their performance is curbed by the applied preliminary visual approaches and the failure to consider the divergence of the same GUI displayed on different devices. To address the challenge, we propose a non-intrusive cross-device and cross-platform system NiCro. NiCro utilizes the state-of-the-art GUI widget detector to detect widgets from GUI images and then analyses a set of comprehensive information to match the widgets across diverse devices. At the system level, NiCro can interact with a virtual device farm and a robotic arm system to perform cross-device, cross-platform testing non-intrusively. We first evaluated NiCro by comparing its multi-modal widget and GUI matching approach with 4 commonly used matching techniques. Then, we further examined its overall performance on 8 various devices, using it to record and replay 107 test cases of 28 popular apps and the home page to show its effectiveness.
Black-Box vs. Gray-Box: A Case Study on Learning Table Tennis Ball Trajectory Prediction with Spin and Impacts

In this paper, we present a method for table tennis ball trajectory filtering and prediction. Our gray-box approach builds on a physical model. At the same time, we use data to learn parameters of the dynamics model, of an extended Kalman filter, and of a neural model that infers the ball's initial condition. We demonstrate superior prediction performance of our approach over two black-box approaches, which are not supplied with physical prior knowledge. We demonstrate that initializing the spin from parameters of the ball launcher using a neural network drastically improves long-time prediction performance over estimating the spin purely from measured ball positions. An accurate prediction of the ball trajectory is crucial for successful returns. We therefore evaluate the return performance with a pneumatic artificial muscular robot and achieve a return rate of 29/30 (97.7%).
Dual-ratio approach for detection of point fluorophores in biological tissue

Abstract. Significance Diffuse in vivo flow cytometry (DiFC) is an emerging fluorescence sensing method to non-invasively detect labeled circulating cells in vivo. However, due to signal-to-noise ratio (SNR) constraints largely attributed to background tissue autofluorescence (AF), DiFC’s measurement depth is limited. Aim The dual ratio (DR)/dual slope is an optical measurement method that aims to suppress noise and enhance SNR to deep tissue regions. We aim to investigate the combination of DR and near-infrared (NIR) DiFC to improve circulating cells’ maximum detectable depth and SNR. Approach Phantom experiments were used to estimate the key parameters in a diffuse fluorescence excitation and emission model. This model and parameters were implemented in Monte Carlo to simulate DR DiFC while varying noise and AF parameters to identify the advantages and limitations of the proposed technique. Results Two key factors must be true to give DR DiFC an advantage over traditional DiFC: first, the fraction of noise that DR methods cannot cancel cannot be above the order of 10% for acceptable SNR. Second, DR DiFC has an advantage, in terms of SNR, if the distribution of tissue AF contributors is surface-weighted. Conclusions DR cancelable noise may be designed (e.g., through the use of source multiplexing), and indications point to the AF contributors’ distribution being truly surface-weighted in vivo. Successful and worthwhile implementation of DR DiFC depends on these considerations, but results point to DR DiFC having possible advantages over traditional DiFC.
Towards Biomechanics-Aware Design of a Steerable Drilling Robot for Spinal Fixation Procedures with Flexible Pedicle Screws

Towards reducing the failure rate of spinal fixation surgical procedures in osteoporotic patients, we propose a unique biomechanically-aware framework for the design of a novel concentric tube steerable drilling robot (CT-SDR). The proposed framework leverages a patient-specific finite element (FE) biomechanics model developed based on Quantitative Computed Tomography (QCT) scans of patient's vertebra to calculate a biomechanically-optimal and feasible drilling and implantation trajectory. The FE output is then used as a design requirement for the design and evaluation of the CT-SDR. Providing a balance between the necessary flexibility to create curved optimal trajectories obtained by the FE module with the required strength to not buckle during drilling through a hard simulated bone material, we showed that the CT-SDR can reliably recreate this drilling trajectory with errors between 1.7-2.2%.
Reformalizing the notion of autonomy as closure through category theory as an arrow-first mathematics

Life continuously changes its own components and states at each moment through interaction with the external world, while maintaining its own individuality in a cyclical manner. Such a property, known as"autonomy,"has been formulated using the mathematical concept of"closure. "We introduce a branch of mathematics called"category theory"as an"arrow-first"mathematics, which sees everything as an"arrow,"and use it to provide a more comprehensive and concise formalization of the notion of autonomy. More specifically, the concept of"monoid,"a category that has only one object, is used to formalize in a simpler and more fundamental way the structure that has been formalized as"operational closure. "By doing so, we show that category theory is a framework or"tool of thinking"that frees us from the habits of thinking to which we are prone and allows us to discuss things formally from a more dynamic perspective, and that it should also contribute to our understanding of living systems.
All Roads Lead to Rome? Exploring the Invariance of Transformers' Representations

Transformer models bring propelling advances in various NLP tasks, thus inducing lots of interpretability research on the learned representations of the models. However, we raise a fundamental question regarding the reliability of the representations. Specifically, we investigate whether transformers learn essentially isomorphic representation spaces, or those that are sensitive to the random seeds in their pretraining process. In this work, we formulate the Bijection Hypothesis, which suggests the use of bijective methods to align different models' representation spaces. We propose a model based on invertible neural networks, BERT-INN, to learn the bijection more effectively than other existing bijective methods such as the canonical correlation analysis (CCA). We show the advantage of BERT-INN both theoretically and through extensive experiments, and apply it to align the reproduced BERT embeddings to draw insights that are meaningful to the interpretability research. Our code is at https://github.com/twinkle0331/BERT-similarity.
Random Sampling of Mellin Band-Limited Signals

In this paper, we address the random sampling problem for the class of Mellin band-limited functions BT which is concentrated on a bounded cube. It is established that any function in BT can be approximated by an element in a finite-dimensional subspace of BT. Utilizing the notion of covering number and Bernstein's inequality to the sum of independent random variables, we prove that the random sampling inequality holds with an overwhelming probability provided the sampling size is large enough.
Using Scalarizations for the Approximation of Multiobjective Optimization Problems: Towards a General Theory

We study the approximation of general multiobjective optimization problems with the help of scalarizations. Existing results state that multiobjective minimization problems can be approximated well by norm-based scalarizations. However, for multiobjective maximization problems, only impossibility results are known so far. Countering this, we show that all multiobjective optimization problems can, in principle, be approximated equally well by scalarizations. In this context, we introduce a transformation theory for scalarizations that establishes the following: Suppose there exists a scalarization that yields an approximation of a certain quality for arbitrary instances of multiobjective optimization problems with a given decomposition specifying which objective functions are to be minimized/maximized. Then, for each other decomposition, our transformation yields another scalarization that yields the same approximation quality for arbitrary instances of problems with this other decomposition. In this sense, the existing results about the approximation via scalarizations for minimization problems carry over to any other objective decomposition—in particular, to maximization problems—when suitably adapting the employed scalarization. We further provide necessary and sufficient conditions on a scalarization such that its optimal solutions achieve a constant approximation quality. We give an upper bound on the best achievable approximation quality that applies to general scalarizations and is tight for the majority of norm-based scalarizations applied in the context of multiobjective optimization. As a consequence, none of these norm-based scalarizations can induce approximation sets for optimization problems with maximization objectives, which unifies and generalizes the existing impossibility results concerning the approximation of maximization problems.
Making Offline RL Online: Collaborative World Models for Offline Visual Reinforcement Learning

Training offline reinforcement learning (RL) models using visual inputs poses two significant challenges, i.e., the overfitting problem in representation learning and the overestimation bias for expected future rewards. Recent work has attempted to alleviate the overestimation bias by encouraging conservative behaviors. This paper, in contrast, tries to build more flexible constraints for value estimation without impeding the exploration of potential advantages. The key idea is to leverage off-the-shelf RL simulators, which can be easily interacted with in an online manner, as the"test bed"for offline policies. To enable effective online-to-offline knowledge transfer, we introduce CoWorld, a model-based RL approach that mitigates cross-domain discrepancies in state and reward spaces. Experimental results demonstrate the effectiveness of CoWorld, outperforming existing RL approaches by large margins.
Relating Implicit Bias and Adversarial Attacks through Intrinsic Dimension

Despite their impressive performance in classification, neural networks are known to be vulnerable to adversarial attacks. These attacks are small perturbations of the input data designed to fool the model. Naturally, a question arises regarding the potential connection between the architecture, settings, or properties of the model and the nature of the attack. In this work, we aim to shed light on this problem by focusing on the implicit bias of the neural network, which refers to its inherent inclination to favor specific patterns or outcomes. Specifically, we investigate one aspect of the implicit bias, which involves the essential Fourier frequencies required for accurate image classification. We conduct tests to assess the statistical relationship between these frequencies and those necessary for a successful attack. To delve into this relationship, we propose a new method that can uncover non-linear correlations between sets of coordinates, which, in our case, are the aforementioned frequencies. By exploiting the entanglement between intrinsic dimension and correlation, we provide empirical evidence that the network bias in Fourier space and the target frequencies of adversarial attacks are closely tied.
CongFu: Conditional Graph Fusion for Drug Synergy Prediction

Drug synergy, characterized by the amplified combined effect of multiple drugs, presents a critical phenomenon for optimizing therapeutic outcomes. However, limited data on drug synergy, arising from the vast number of possible drug combinations and computational costs, motivate the need for predictive methods. In this work, we introduce CongFu, a novel Conditional Graph Fusion Layer, designed to predict drug synergy. CongFu employs an attention mechanism and a bottleneck to extract local graph contexts and conditionally fuse graph data within a global context. Its modular architecture enables flexible replacement of layer modules, including readouts and graph encoders, facilitating customization for diverse applications. To evaluate the performance of CongFu, we conduct comprehensive experiments on four datasets, encompassing three distinct setups for drug synergy prediction. Remarkably, CongFu achieves state-of-the-art results on 11 out of 12 benchmark datasets, demonstrating its ability to capture intricate patterns of drug synergy. Through extensive ablation studies, we validate the significance of individual layer components, affirming their contributions to overall predictive performance. By addressing the challenge of predicting drug synergy in untested drug pairs, CongFu opens new avenues for optimizing drug combinations and advancing personalized medicine.
A Fair and In-Depth Evaluation of Existing End-to-End Entity Linking Systems

Existing evaluations of entity linking systems often say little about how the system is going to perform for a particular application. There are two fundamental reasons for this. One is that many evaluations only use aggregate measures (like precision, recall, and F1 score), without a detailed error analysis or a closer look at the results. The other is that all of the widely used benchmarks have strong biases and artifacts, in particular: a strong focus on named entities, an unclear or missing specification of what else counts as an entity mention, poor handling of ambiguities, and an over- or underrepresentation of certain kinds of entities. We provide a more meaningful and fair in-depth evaluation of a variety of existing end-to-end entity linkers. We characterize their strengths and weaknesses and also report on reproducibility aspects. The detailed results of our evaluation can be inspected under https://elevant.cs. uni-freiburg.de/emnlp2023 . Our evaluation is based on several widely used benchmarks, which exhibit the problems mentioned above to various degrees, as well as on two new benchmarks, which address the problems mentioned above. The new benchmarks can be found under https://github.com/ad-freiburg/fair-entity-linking-benchmarks .
Directed Message Passing Based on Attention for Prediction of Molecular Properties

Molecular representation learning (MRL) has long been crucial in the fields of drug discovery and materials science, and it has made significant progress due to the development of natural language processing (NLP) and graph neural networks (GNNs). NLP treats the molecules as one dimensional sequential tokens while GNNs treat them as two dimensional topology graphs. Based on different message passing algorithms, GNNs have various performance on detecting chemical environments and predicting molecular properties. Herein, we propose Directed Graph Attention Networks (D-GATs): the expressive GNNs with directed bonds. The key to the success of our strategy is to treat the molecular graph as directed graph and update the bond states and atom states by scaled dot-product attention mechanism. This allows the model to better capture the sub-structure of molecular graph, i.e., functional groups. Compared to other GNNs or Message Passing Neural Networks (MPNNs), D-GATs outperform the state-of-the-art on 13 out of 15 important molecular property prediction benchmarks.
Modeling the contrast-sensitivity function of the human visual system

Based on a representation of the receptive fields of the human retina, a mathematical model has been developed that describes the primary stage of the processing of visual information, taking into account the internal noise of the photoreceptors. The hypothesis is advanced that the visual perception power of an image is the result of the sum of the reactions of the ganglion cells to whose receptive fields the action of the stimulus propagates. Based on the proposed model, the contrast-sensitivity function (CSF) of the human visual system is calculated. It is shown that, for stimuli in the form of regular periodic gratings with alternating light and dark strips in the region of high spatial frequencies, the CSF has resonance minima whose spectral position and depth depend on the ratio between the sizes of the central and peripheral zones of the receptive fields.
GRB 201015A and the nature of low-luminosity soft gamma-ray bursts

GRB 201015A is a peculiarly low luminosity, spectrally soft gamma-ray burst (GRB), with T90 = 9.8 ± 3.5 s (time interval of detection of 90 per cent of photons from the GRB), and an associated supernova (likely to be type Ic or Ic-BL). GRB 201015A has an isotropic energy $E_{\gamma , \rm iso}$$= 1.75 ^{+0.60} _{-0.53} \times 10^{50}$ erg, and photon index $\Gamma = 3.00 ^{+0.50} _{-0.42}$ (15–150 keV). It follows the Amati relation, a correlation between $E_{\gamma , \rm iso}$ and spectral peak energy Ep followed by long GRBs. It appears exceptionally soft based on Γ, the hardness ratio of HR = 0.47 ± 0.24, and low-Ep, so we have compared it to other GRBs sharing these properties. These events can be explained by shock breakout, poorly collimated jets, and off-axis viewing. Follow-up observations of the afterglow taken in the X-ray, optical, and radio reveal a surprisingly late flattening in the X-ray from t = (2.61 ± 1.27) × 104 s to $t = 1.67 ^{+1.14} _{-0.65} \times 10^6$ s. We fit the data to closure relations describing the synchrotron emission, finding the electron spectral index to be $p = 2.42 ^{+0.44} _{-0.30}$ and evidence of late-time energy injection with coefficient $q = 0.24 ^{+0.24} _{-0.18}$. The jet half opening angle lower limit (θj ≥ 16°) is inferred from the non-detection of a jet break. The launch of SVOM and Einstein Probe in 2023 should enable detection of more low-luminosity events like this, providing a fuller picture of the variety of GRBs.
Negative Feedback Training: A Novel Concept to Improve Robustness of NVCiM DNN Accelerators

Compute-in-memory (CIM) accelerators built upon non-volatile memory (NVM) devices excel in energy efficiency and latency when performing Deep Neural Network (DNN) inference, thanks to their in-situ data processing capability. However, the stochastic nature and intrinsic variations of NVM devices often result in performance degradation in DNN inference. Introducing these non-ideal device behaviors during DNN training enhances robustness, but drawbacks include limited accuracy improvement, reduced prediction confidence, and convergence issues. This arises from a mismatch between the deterministic training and non-deterministic device variations, as such training, though considering variations, relies solely on the model's final output. In this work, we draw inspiration from the control theory and propose a novel training concept: Negative Feedback Training (NFT) leveraging the multi-scale noisy information captured from network. We develop two specific NFT instances, Oriented Variational Forward (OVF) and Intermediate Representation Snapshot (IRS). Extensive experiments show that our methods outperform existing state-of-the-art methods with up to a 46.71% improvement in inference accuracy while reducing epistemic uncertainty, boosting output confidence, and improving convergence probability. Their effectiveness highlights the generality and practicality of our NFT concept in enhancing DNN robustness against device variations.
Geometrical Study of the Cone of Sums of Squares plus Sums of Nonnegative Circuits

In this article, we combine sums of squares (SOS) and sums of nonnegative circuit (SONC) forms, two independent nonnegativity certificates for real homogeneous polynomials. We consider the convex cone SOS+SONC of forms that decompose into a sum of an SOS and a SONC form and study it from a geometric point of view. We show that the SOS+SONC cone is proper and neither closed under multiplications nor under linear transformation of variables. Moreover, we present an alternative proof of an analog of Hilbert's 1888 Theorem for the SOS+SONC cone and prove that in the non-Hilbert cases it provides a proper superset of both the SOS and the SONC cone. This follows by exploiting a new necessary condition for membership in the SONC cone.
Shapley-Folkman-type Theorem for Integrally Convex Sets

The Shapley-Folkman theorem is a statement about the Minkowski sum of (non-convex) sets, expressing the closeness of the Minkowski sum to convexity in a quantitative manner. This paper establishes similar theorems for integrally convex sets and M-natural-convex sets, which are major classes of discrete convex sets in discrete convex analysis.
Drafting Event Schemas using Language Models

Past work has studied event prediction and event language modeling, sometimes mediated through structured representations of knowledge in the form of event schemas. Such schemas can lead to explainable predictions and forecasting of unseen events given incomplete information. In this work, we look at the process of creating such schemas to describe complex events. We use large language models (LLMs) to draft schemas directly in natural language, which can be further refined by human curators as necessary. Our focus is on whether we can achieve sufficient diversity and recall of key events and whether we can produce the schemas in a sufficiently descriptive style. We show that large language models are able to achieve moderate recall against schemas taken from two different datasets, with even better results when multiple prompts and multiple samples are combined. Moreover, we show that textual entailment methods can be used for both matching schemas to instances of events as well as evaluating overlap between gold and predicted schemas. Our method paves the way for easier distillation of event knowledge from large language model into schemas.
Interpretable by Design Visual Question Answering

Model interpretability has long been a hard problem for the AI community especially in the multimodal setting, where vision and language need to be aligned and reasoned at the same time. In this paper, we specifically focus on the problem of Visual Question Answering (VQA). While previous researches try to probe into the network structures of black-box multimodal models, we propose to tackle the problem from a different angle -- to treat interpretability as an explicit additional goal. Given an image and question, we argue that an interpretable VQA model should be able to tell what conclusions it can get from which part of the image, and show how each statement help to arrive at an answer. We introduce InterVQA: Interpretable-by-design VQA, where we design an explicit intermediate dynamic reasoning structure for VQA problems and enforce symbolic reasoning that only use the structure for final answer prediction to take place. InterVQA produces high-quality explicit intermediate reasoning steps, while maintaining similar to the state-of-the-art (sota) end-task performance.