nils-herrmann/clean_scirepeval huggingface dataset:
Temporal Self-Attention Network for Medical Concept Embedding

In longitudinal electronic health records (EHRs), the event records of a patient are distributed over a long period of time and the temporal relations between the events reflect sufficient domain knowledge to benefit prediction tasks such as the rate of inpatient mortality. Medical concept embedding as a feature extraction method that transforms a set of medical concepts with a specific time stamp into a vector, which will be fed into a supervised learning algorithm. The quality of the embedding significantly determines the learning performance over the medical data. In this paper, we propose a medical concept embedding method based on applying a self-attention mechanism to represent each medical concept. We propose a novel attention mechanism which captures the contextual information and temporal relationships between medical concepts. A light-weight neural net, "Temporal Self-Attention Network (TeSAN)", is then proposed to learn medical concept embedding based solely on the proposed attention mechanism. To test the effectiveness of our proposed methods, we have conducted clustering and prediction tasks on two public EHRs datasets comparing TeSAN against five state-of-the-art embedding methods. The experimental results demonstrate that the proposed TeSAN model is superior to all the compared methods. To the best of our knowledge, this work is the first to exploit temporal self-attentive relations between medical events.
Exploiting Fast Decaying and Locality in Multi-Agent MDP with Tree Dependence Structure

This paper considers a multi-agent Markov Decision Process (MDP), where there are n agents and each agent i is associated with a state si and action ai taking values from a finite set. Though the global state space size and action space size are exponential in n, we impose local dependence structures and focus on local policies that only depend on local states, and we propose a method that finds nearly optimal local policies in polynomial time (in n) when the dependence structure is a one directional tree. The algorithm builds on approximated reward functions which are evaluated using locally truncated Markov process. Further, under some special conditions, we prove that the gap between the approximated reward function and the true reward function is decaying exponentially fast as the length of the truncated Markov process gets longer. The intuition behind this is that under some assumptions, the effect of agent interactions decays exponentially in the distance between agents, which we term "fast decaying property". Results in this paper are our preliminary steps towards designing efficient reinforcement learning algorithms with optimality guarantees in large multi-agent MDP problems whose state (action) space size is exponentially large in n.
X-ToM: Explaining with Theory-of-Mind for Gaining Justified Human Trust

We present a new explainable AI (XAI) framework aimed at increasing justified human trust and reliance in the AI machine through explanations. We pose explanation as an iterative communication process, i.e. dialog, between the machine and human user. More concretely, the machine generates sequence of explanations in a dialog which takes into account three important aspects at each dialog turn: (a) human's intention (or curiosity); (b) human's understanding of the machine; and (c) machine's understanding of the human user. To do this, we use Theory of Mind (ToM) which helps us in explicitly modeling human's intention, machine's mind as inferred by the human as well as human's mind as inferred by the machine. In other words, these explicit mental representations in ToM are incorporated to learn an optimal explanation policy that takes into account human's perception and beliefs. Furthermore, we also show that ToM facilitates in quantitatively measuring justified human trust in the machine by comparing all the three mental representations. 
We applied our framework to three visual recognition tasks, namely, image classification, action recognition, and human body pose estimation. We argue that our ToM based explanations are practical and more natural for both expert and non-expert users to understand the internal workings of complex machine learning models. To the best of our knowledge, this is the first work to derive explanations using ToM. Extensive human study experiments verify our hypotheses, showing that the proposed explanations significantly outperform the state-of-the-art XAI methods in terms of all the standard quantitative and qualitative XAI evaluation metrics including human trust, reliance, and explanation satisfaction.
On the Separability of Classes with the Cross-Entropy Loss Function

In this paper, we focus on the separability of classes with the cross-entropy loss function for classification problems by theoretically analyzing the intra-class distance and inter-class distance (i.e. the distance between any two points belonging to the same class and different classes, respectively) in the feature space, i.e. the space of representations learnt by neural networks. Specifically, we consider an arbitrary network architecture having a fully connected final layer with Softmax activation and trained using the cross-entropy loss. We derive expressions for the value and the distribution of the squared L2 norm of the product of a network dependent matrix and a random intra-class and inter-class distance vector (i.e. the vector between any two points belonging to the same class and different classes), respectively, in the learnt feature space (or the transformation of the original data) just before Softmax activation, as a function of the cross-entropy loss value. The main result of our analysis is the derivation of a lower bound for the probability with which the inter-class distance is more than the intra-class distance in this feature space, as a function of the loss value. We do so by leveraging some empirical statistical observations with mild assumptions and sound theoretical analysis. As per intuition, the probability with which the inter-class distance is more than the intra-class distance decreases as the loss value increases, i.e. the classes are better separated when the loss value is low. To the best of our knowledge, this is the first work of theoretical nature trying to explain the separability of classes in the feature space learnt by neural networks trained with the cross-entropy loss function.
Self-Supervised Correspondence in Visuomotor Policy Learning

In this letter, we explore using self-supervised correspondence for improving the generalization performance and sample efficiency of visuomotor policy learning. Prior work has primarily used approaches such as autoencoding, pose-based losses, and end-to-end policy optimization in order to train the visual portion of visuomotor policies. We instead propose an approach using self-supervised dense visual correspondence training and show that this enables visuomotor policy learning with surprisingly high generalization performance with modest amounts of data. Using imitation learning, we demonstrate extensive hardware validation on challenging manipulation tasks with as few as 50 demonstrations. Our learned policies can generalize across classes of objects, react to deformable object configurations, and manipulate textureless symmetrical objects in a variety of backgrounds, all with closed-loop, real-time vision-based policies. Simulated imitation learning experiments suggest that correspondence training offers sample complexity and generalization benefits compared to autoencoding and end-to-end training.
PSGAN: Pose and Expression Robust Spatial-Aware GAN for Customizable Makeup Transfer

In this paper, we address the makeup transfer task, which aims to transfer the makeup from a reference image to a source image. Existing methods have achieved promising progress in constrained scenarios, but transferring between images with large pose and expression differences is still challenging. Besides, they cannot realize customizable transfer that allows a controllable shade of makeup or specifies the part to transfer, which limits their applications. To address these issues, we propose Pose and expression robust Spatial-aware GAN (PSGAN). It first utilizes Makeup Distill Network to disentangle the makeup of the reference image as two spatial-aware makeup matrices. Then, Attentive Makeup Morphing module is introduced to specify how the makeup of a pixel in the source image is morphed from the reference image. With the makeup matrices and the source image, Makeup Apply Network is used to perform makeup transfer. Our PSGAN not only achieves state-of-the-art results even when large pose and expression differences exist but also is able to perform partial and shade-controllable makeup transfer. Both the code and a newly collected dataset containing facial images with various poses and expressions will be available at https://github.com/wtjiang98/PSGAN.
The impact of (n,γ) reaction rate uncertainties on the predicted abundances of i-process elements with 32 ≤ Z ≤ 48 in the metal-poor star HD94028

Several anomalous elemental abundance ratios have been observed in the metal-poor star HD94028. We assume that its high [As/Ge] ratio is a product of a weak intermediate (i) neutron-capture process. Given that observational errors are usually smaller than predicted nuclear physics uncertainties, we have first set-up a benchmark one-zone i-process nucleosynthesis simulation results of which provide the best fit to the observed abundances. We have then performed Monte Carlo simulations in which 113 relevant (n,γ) reaction rates of unstable species were randomly varied within Hauser–Feshbach model uncertainty ranges for each reaction to estimate the impact on the predicted stellar abundances. One of the interesting results of these simulations is a double-peaked distribution of the As abundance, which is caused by the variation of the 75Ga (n,γ) cross-section. This variation strongly anticorrelates with the predicted As abundance, confirming the necessity for improved theoretical or experimental bounds on this cross-section. The 66Ni (n,γ) reaction is found to behave as a major bottleneck for the i-process nucleosynthesis. Our analysis finds the Pearson product–moment correlation coefficient rP > 0.2 for all of the i-process elements with 32 ≤ Z ≤ 42, with significant changes in their predicted abundances showing up when the rate of this reaction is reduced to its theoretically constrained lower bound. Our results are applicable to any other stellar nucleosynthesis site with the similar i-process conditions, such as Sakurai’s object (V4334 Sagittarii) or rapidly accreting white dwarfs.
Towards Distributed Logic Programming based on Computability Logic

{\em Computability logic} (CoL) is a powerful computational model which views computational problems as games played by a machine and its environment. It uses formulas to represent computational problems. In this paper, we show that CoL naturally supports multiagent programming models with distributed control. To be specific, we discuss a web-based implemention of a distributed logic programming model based on CoL (CL1 to be exact).
Global Autoregressive Models for Data-Efficient Sequence Learning

Standard autoregressive seq2seq models are easily trained by max-likelihood, but tend to show poor results under small-data conditions. We introduce a class of seq2seq models, GAMs (Global Autoregressive Models), which combine an autoregressive component with a log-linear component, allowing the use of global a priori features to compensate for lack of data. We train these models in two steps. In the first step, we obtain an unnormalized GAM that maximizes the likelihood of the data, but is improper for fast inference or evaluation. In the second step, we use this GAM to train (by distillation) a second autoregressive model that approximates the normalized distribution associated with the GAM, and can be used for fast inference and evaluation. Our experiments focus on language modelling under synthetic conditions and show a strong perplexity reduction of using the second autoregressive model over the standard one.
Modular decomposition of Markov chain: detecting hierarchical organization of pervasive communities

In network science, a group of nodes connected with each other at higher probability than with those outside the group is referred to as a community. From the perspective that individual communities are associated with functional modules constituting complex systems described by networks, discovering communities is primarily important for understanding overall functions of these systems. Much effort has been devoted to developing methods to detect communities in networks since the early days of network science. Nevertheless, the method to reveal key characteristics of communities in real-world network remains to be established. Here we formulate decomposition of a random walk spreading over the entire network into local modules as proxy for communities. This formulation will reveal the pervasive structure of communities and their hierarchical organization, which are the hallmarks of real-world networks but are out of reach of most existing methods.
Pose Neural Fabrics Search

Neural Architecture Search (NAS) technologies have been successfully performed for efficient neural architectures for tasks such as image classification and semantic segmentation. However, existing works implement NAS for target tasks independently of domain knowledge and focus only on searching for an architecture to replace the human-designed network in a common pipeline. \emph{Can we exploit human prior knowledge to guide NAS?} To address it, we propose a framework, named Pose Neural Fabrics Search (PNFS), introducing prior knowledge of body structure into NAS for human pose estimation. We lead a new neural architecture search space, by parameterizing Cell-based Neural Fabric, to learn micro as well as macro neural architecture using a differentiable search strategy. To take advantage of part-based structural knowledge of the human body and learning capability of NAS, global pose constraint relationships are modeled as multiple part representations, each of which is predicted by a personalized Cell-based Neural Fabric. In part representation, we view human skeleton keypoints as entities by representing them as vectors at image locations, expecting it to capture keypoint's feature in a relaxed vector space. The experiments on MPII and MS-COCO datasets demonstrate that PNFS\footnote{Code is available at \url{this https URL}} can achieve comparable performance to state-of-the-art methods, with fewer parameters and lower computational complexity.
Chromosome compaction and chromatin stiffness enhance diffusive loop extrusion by slip-link proteins.

We use Brownian dynamics simulations to study the formation of chromatin loops through diffusive sliding of slip-link-like proteins, mimicking the behaviour of cohesin molecules. We recently proposed that diffusive sliding is sufficient to explain the extrusion of chromatin loops of hundreds of kilo-base-pairs (kbp), which may then be stabilised by interactions between cohesin and CTCF proteins. Here we show that the flexibility of the chromatin fibre strongly affects this dynamical process, and find that diffusive loop extrusion is more efficient on stiffer chromatin regions. We also show that the dynamics of loop formation are faster in confined and collapsed chromatin conformations but that this enhancement is counteracted by the increased crowding. We provide a simple theoretical argument explaining why stiffness and collapsed conformations favour diffusive extrusion. In light of the heterogeneous physical and conformational properties of eukaryotic chromatin, we suggest that our results are relevant to understand the looping and organisation of interphase chromosomes in vivo.
TextSR: Content-Aware Text Super-Resolution Guided by Recognition

Scene text recognition has witnessed rapid development with the advance of convolutional neural networks. Nonetheless, most of the previous methods may not work well in recognizing text with low resolution which is often seen in natural scene images. An intuitive solution is to introduce super-resolution techniques as pre-processing. However, conventional super-resolution methods in the literature mainly focus on reconstructing the detailed texture of natural images, which typically do not work well for text due to the unique characteristics of text. To tackle these problems, in this work, we propose a content-aware text super-resolution network to generate the information desired for text recognition. In particular, we design an end-to-end network that can perform super-resolution and text recognition simultaneously. Different from previous super-resolution methods, we use the loss of text recognition as the Text Perceptual Loss to guide the training of the super-resolution network, and thus it pays more attention to the text content, rather than the irrelevant background area. Extensive experiments on several challenging benchmarks demonstrate the effectiveness of our proposed method in restoring a sharp high-resolution image from a small blurred one, and show that the recognition performance clearly boosts up the performance of text recognizer. To our knowledge, this is the first work focusing on text super-resolution. Code will be released in this https URL.
Dark matter and collider signals in supersymmetric 
U(1)′
 models with nonuniversal 
Z′
 couplings

We analyse supersymmetric models augmented by an extra $U(1)$ gauge group. To avoid anomalies in these models without introducing exotics, we allow for family-dependent $U(1)^\prime$ charges, and choose a simple form for these, dependent on one $U(1)^\prime$ charge parameter only. With this choice, $Z^\prime$ decays into di-taus but not di-leptons, weakening considerably the constraints on its mass. In the supersymmetric sector, the effect is to lower the singlino mass, allowing it to be the dark matter candidate. We investigate the dark matter constraints and collider implications of such models, with mostly singlino, or mostly higgsinos, or a mixture of the two as lightest supersymmetric particles. In these scenarios, $Z^\prime$ decays significantly into chargino or neutralino pairs, and thus indirectly into final state leptons. We devise benchmarks which, with adequate cuts, can yield signals visible at the high-luminosity LHC.
How humans learn and represent networks

Humans receive information from the world around them in sequences of discrete items—from words in language or notes in music to abstract concepts in books and websites on the Internet. To model their environment, from a young age people are tasked with learning the network structures formed by these items (nodes) and the connections between them (edges). But how do humans uncover the large-scale structures of networks when they experience only sequences of individual items? Moreover, what do people’s internal maps and models of these networks look like? Here, we introduce graph learning, a growing and interdisciplinary field studying how humans learn and represent networks in the world around them. Specifically, we review progress toward understanding how people uncover the complex webs of relationships underlying sequences of items. We begin by describing established results showing that humans can detect fine-scale network structure, such as variations in the probabilities of transitions between items. We next present recent experiments that directly control for differences in transition probabilities, demonstrating that human behavior depends critically on the mesoscale and macroscale properties of networks. Finally, we introduce computational models of human graph learning that make testable predictions about the impact of network structure on people’s behavior and cognition. Throughout, we highlight open questions in the study of graph learning that will require creative insights from cognitive scientists and network scientists alike.
DeepTIO: A Deep Thermal-Inertial Odometry With Visual Hallucination

Visual odometry shows excellent performance in a wide range of environments. However, in visually-denied scenarios (e.g. heavy smoke or darkness), pose estimates degrade or even fail. Thermal cameras are commonly used for perception and inspection when the environment has low visibility. However, their use in odometry estimation is hampered by the lack of robust visual features. In part, this is as a result of the sensor measuring the ambient temperature profile rather than scene appearance and geometry. To overcome this issue, we propose a Deep Neural Network model for thermal-inertial odometry (DeepTIO) by incorporating a visual hallucination network to provide the thermal network with complementary information. The hallucination network is taught to predict fake visual features from thermal images by using Huber loss. We also employ selective fusion to attentively fuse the features from three different modalities, i.e thermal, hallucination, and inertial features. Extensive experiments are performed in hand-held and mobile robot data in benign and smoke-filled environments, showing the efficacy of the proposed model.
Novel methods for the analysis of stepped wedge cluster randomized trials

Stepped wedge cluster randomized trials (SW‐CRTs) have become increasingly popular and are used for a variety of interventions and outcomes, often chosen for their feasibility advantages. SW‐CRTs must account for time trends in the outcome because of the staggered rollout of the intervention. Robust inference procedures and nonparametric analysis methods have recently been proposed to handle such trends without requiring strong parametric modeling assumptions, but these are less powerful than model‐based approaches. We propose several novel analysis methods that reduce reliance on modeling assumptions while preserving some of the increased power provided by the use of mixed effects models. In one method, we use the synthetic control approach to find the best matching clusters for a given intervention cluster. Another method makes use of within‐cluster crossover information to construct an overall estimator. We also consider methods that combine these approaches to further improve power. We test these methods on simulated SW‐CRTs, describing scenarios in which these methods have increased power compared with existing nonparametric methods while preserving nominal validity when mixed effects models are misspecified. We also demonstrate theoretical properties of these estimators with less restrictive assumptions than mixed effects models. Finally, we propose avenues for future research on the use of these methods; motivation for such research arises from their flexibility, which allows the identification of specific causal contrasts of interest, their robustness, and the potential for incorporating covariates to further increase power. Investigators conducting SW‐CRTs might well consider such methods when common modeling assumptions may not hold.
Place Recognition for Stereo VisualOdometry using LiDAR Descriptors

Place recognition is a core component in SLAM, and in most visual SLAM systems, it is based on the similarity between 2D images. However, the 3D points generated by visual odometry, and the structure information embedded within, are not exploited. In this paper, we adapt place recognition methods for 3D point clouds into stereo visual odometry. Stereo visual odometry generates 3D point clouds with a consistent scale. Thus, we are able to use global LiDAR descriptors for 3D point clouds to determine the similarity between places. 3D point clouds are more reliable than 2D visual cues (e.g., 2D features) against environmental changes such as varying illumination and can benefit visual SLAM systems in long-term deployment scenarios. Extensive evaluation on a public dataset (Oxford RobotCar) demonstrates the accuracy and efficiency of using 3D point clouds for place recognition over 2D methods.