nils-herrmann/clean_scirepeval huggingface dataset:
Effective Ways to Build and Evaluate Individual Survival Distributions

An accurate model of a patient's individual survival distribution can help determine the appropriate treatment for terminal patients. Unfortunately, risk scores (e.g., from Cox Proportional Hazard models) do not provide survival probabilities, single-time probability models (e.g., the Gail model, predicting 5 year probability) only provide for a single time point, and standard Kaplan-Meier survival curves provide only population averages for a large class of patients meaning they are not specific to individual patients. This motivates an alternative class of tools that can learn a model which provides an individual survival distribution which gives survival probabilities across all times - such as extensions to the Cox model, Accelerated Failure Time, an extension to Random Survival Forests, and Multi-Task Logistic Regression. This paper first motivates such "individual survival distribution" (ISD) models, and explains how they differ from standard models. It then discusses ways to evaluate such models - namely Concordance, 1-Calibration, Brier score, and various versions of L1-loss - and then motivates and defines a novel approach "D-Calibration", which determines whether a model's probability estimates are meaningful. We also discuss how these measures differ, and use them to evaluate several ISD prediction tools, over a range of survival datasets.
Multi-label classification search space in the MEKA software

This supplementary material aims to describe the proposed multi-label classification (MLC) search spaces based on the MEKA and WEKA softwares. First, we overview 26 MLC algorithms and meta-algorithms in MEKA, presenting their main characteristics, such as hyper-parameters, dependencies and constraints. Second, we review 28 single-label classification (SLC) algorithms, preprocessing algorithms and meta-algorithms in the WEKA software. These SLC algorithms were also studied because they are part of the proposed MLC search spaces. Fundamentally, this occurs due to the problem transformation nature of several MLC algorithms used in this work. These algorithms transform an MLC problem into one or several SLC problems in the first place and solve them with SLC model(s) in a next step. Therefore, understanding their main characteristics is crucial to this work. Finally, we present a formal description of the search spaces by proposing a context-free grammar that encompasses the 54 learning algorithms. This grammar basically comprehends the possible combinations, the constraints and dependencies among the learning algorithms.
Robust neural circuit reconstruction from serial electron microscopy with convolutional recurrent networks

Recent successes in deep learning have started to impact neuroscience. Of particular significance are claims that current segmentation algorithms achieve "super-human" accuracy in an area known as connectomics. However, as we will show, these algorithms do not effectively generalize beyond the particular source and brain tissues used for training -- severely limiting their usability by the broader neuroscience community. To fill this gap, we describe a novel connectomics challenge for source- and tissue-agnostic reconstruction of neurons (STAR), which favors broad generalization over fitting specific datasets. We first demonstrate that current state-of-the-art approaches to neuron segmentation perform poorly on the challenge. We further describe a novel convolutional recurrent neural network module that combines short-range horizontal connections within a processing stage and long-range top-down connections between stages. The resulting architecture establishes the state of the art on the STAR challenge and represents a significant step towards widely usable and fully-automated connectomics analysis.
Unsupervised Control Through Non-Parametric Discriminative Rewards

Learning to control an environment without hand-crafted rewards or expert data remains challenging and is at the frontier of reinforcement learning research. We present an unsupervised learning algorithm to train agents to achieve perceptually-specified goals using only a stream of observations and actions. Our agent simultaneously learns a goal-conditioned policy and a goal achievement reward function that measures how similar a state is to the goal state. This dual optimization leads to a co-operative game, giving rise to a learned reward function that reflects similarity in controllable aspects of the environment instead of distance in the space of observations. We demonstrate the efficacy of our agent to learn, in an unsupervised manner, to reach a diverse set of goals on three domains -- Atari, the DeepMind Control Suite and DeepMind Lab.
Context-Aware Dialog Re-Ranking for Task-Oriented Dialog Systems

Dialog response ranking is used to rank response candidates by considering their relation to the dialog history. Although researchers have addressed this concept for open-domain dialogs, little attention has been focused on task-oriented dialogs. Furthermore, no previous studies have analyzed whether response ranking can improve the performance of existing dialog systems in real human–computer dialogs with speech recognition errors. In this paper, we propose a context-aware dialog response re-ranking system. Our system reranks responses in two steps: (1) it calculates matching scores for each candidate response and the current dialog context; (2) it combines the matching scores and a probability distribution of the candidates from an existing dialog system for response re-ranking. By using neural word embedding-based models and handcrafted or logistic regression-based ensemble models, we have improved the performance of a recently proposed end-to-end task-oriented dialog system on real dialogs with speech recognition errors.
Beyond Pham's algorithm for joint diagonalization

The approximate joint diagonalization of a set of matrices consists in finding a basis in which these matrices are as diagonal as possible. This problem naturally appears in several statistical learning tasks such as blind signal separation. We consider the diagonalization criterion studied in a seminal paper by Pham (2001), and propose a new quasi-Newton method for its optimization. Through numerical experiments on simulated and real datasets, we show that the proposed method outper-forms Pham's algorithm.
Image Reconstruction with Predictive Filter Flow

We propose a simple, interpretable framework for solving a wide range of image reconstruction problems such as denoising and deconvolution. Given a corrupted input image, the model synthesizes a spatially varying linear filter which, when applied to the input image, reconstructs the desired output. The model parameters are learned using supervised or self-supervised training. We test this model on three tasks: non-uniform motion blur removal, lossy-compression artifact reduction and single image super resolution. We demonstrate that our model substantially outperforms state-of-the-art methods on all these tasks and is significantly faster than optimization-based approaches to deconvolution. Unlike models that directly predict output pixel values, the predicted filter flow is controllable and interpretable, which we demonstrate by visualizing the space of predicted filters for different tasks.
One-Shot Instance Segmentation

We tackle the problem of one-shot instance segmentation: Given an example image of a novel, previously unknown object category, find and segment all objects of this category within a complex scene. To address this challenging new task, we propose Siamese Mask R-CNN. It extends Mask R-CNN by a Siamese backbone encoding both reference image and scene, allowing it to target detection and segmentation towards the reference category. We demonstrate empirical results on MS Coco highlighting challenges of the one-shot setting: while transferring knowledge about instance segmentation to novel object categories works very well, targeting the detection network towards the reference category appears to be more difficult. Our work provides a first strong baseline for one-shot instance segmentation and will hopefully inspire further research into more powerful and flexible scene analysis algorithms. Code is available at: this https URL
Core-fringe link prediction

Data collection often involves the partial measurement of a larger system. A common example arises in the process collecting network data: we often obtain network datasets by recording all of the interactions among a small set of core nodes, so that we end up with a measurement of the network consisting of these core nodes together with a potentially much larger set of fringe nodes that have links to the core. Given the ubiquity of this process for assembling network data, it becomes crucial to understand the role of such a core-fringe structure. 
Here we study how the inclusion of fringe nodes affects the standard task of network link prediction. One might initially think the inclusion of any additional data is useful, and hence that it should be beneficial to include all fringe nodes that are available. However, we find that this is not true; in fact, there is substantial variability in the value of the fringe nodes for prediction. In some datasets, once an algorithm is selected, including any additional data from the fringe can actually hurt prediction performance; in other datasets, including some amount of fringe information is useful before prediction performance saturates or even declines; and in further cases, including the entire fringe leads to the best performance. While such variety might seem surprising, we show that these behaviors are exhibited by simple random graph models.
Escaping Plato’s Cave: 3D Shape From Adversarial Rendering

We introduce PlatonicGAN to discover the 3D structure of an object class from an unstructured collection of 2D images, i.e., where no relation between photos is known, except that they are showing instances of the same category. The key idea is to train a deep neural network to generate 3D shapes which, when rendered to images, are indistinguishable from ground truth images (for a discriminator) under various camera poses. Discriminating 2D images instead of 3D shapes allows tapping into unstructured 2D photo collections instead of relying on curated (e.g., aligned, annotated, etc.) 3D data sets. To establish constraints between 2D image observation and their 3D interpretation, we suggest a family of rendering layers that are effectively differentiable. This family includes visual hull, absorption-only (akin to x-ray), and emission-absorption. We can successfully reconstruct 3D shapes from unstructured 2D images and extensively evaluate PlatonicGAN on a range of synthetic and real data sets achieving consistent improvements over baseline methods. We further show that PlatonicGAN can be combined with 3D supervision to improve on and in some cases even surpass the quality of 3D-supervised methods.
Kalman Filter Demystified: From Intuition to Probabilistic Graphical Model to Real Case in Financial Markets

In this paper, we revisit the Kalman filter theory. After giving the intuition on a simplified financial markets example, we revisit the maths underlying it. We then show that Kalman filter can be presented in a very different fashion using graphical models. This enables us to establish the connection between Kalman filter and Hidden Markov Models. We then look at their application in financial markets and provide various intuitions in terms of their applicability for complex systems such as financial markets. Although this paper has been written more like a self contained work connecting Kalman filter to Hidden Markov Models and hence revisiting well known and establish results, it contains new results and brings additional contributions to the field. First, leveraging on the link between Kalman filter and HMM, it gives new algorithms for inference for extended Kalman filters. Second, it presents an alternative to the traditional estimation of parameters using EM algorithm thanks to the usage of CMA-ES optimization. Third, it examines the application of Kalman filter and its Hidden Markov models version to financial markets, providing various dynamics assumptions and tests. We conclude by connecting Kalman filter approach to trend following technical analysis system and showing their superior performances for trend following detection.
Multi-Level Multimodal Common Semantic Space for Image-Phrase Grounding

We address the problem of phrase grounding by learning a multi-level common semantic space shared by the textual and visual modalities. We exploit multiple levels of feature maps of a Deep Convolutional Neural Network, as well as contextualized word and sentence embeddings extracted from a character-based language model. Following dedicated non-linear mappings for visual features at each level, word, and sentence embeddings, we obtain multiple instantiations of our common semantic space in which comparisons between any target text and the visual content is performed with cosine similarity. We guide the model by a multi-level multimodal attention mechanism which outputs attended visual features at each level. The best level is chosen to be compared with text content for maximizing the pertinence scores of image-sentence pairs of the ground truth. Experiments conducted on three publicly available datasets show significant performance gains (20%-60% relative) over the state-of-the-art in phrase localization and set a new performance record on those datasets. We provide a detailed ablation study to show the contribution of each element of our approach and release our code on GitHub.
Few-Shot Generalization Across Dialogue Tasks

Machine-learning based dialogue managers are able to learn complex behaviors in order to complete a task, but it is not straightforward to extend their capabilities to new domains. We investigate different policies' ability to handle uncooperative user behavior, and how well expertise in completing one task (such as restaurant reservations) can be reapplied when learning a new one (e.g. booking a hotel). We introduce the Recurrent Embedding Dialogue Policy (REDP), which embeds system actions and dialogue states in the same vector space. REDP contains a memory component and attention mechanism based on a modified Neural Turing Machine, and significantly outperforms a baseline LSTM classifier on this task. We also show that both our architecture and baseline solve the bAbI dialogue task, achieving 100% test accuracy.
Neural probabilistic motor primitives for humanoid control

We focus on the problem of learning a single motor module that can flexibly express a range of behaviors for the control of high-dimensional physically simulated humanoids. To do this, we propose a motor architecture that has the general structure of an inverse model with a latent-variable bottleneck. We show that it is possible to train this model entirely offline to compress thousands of expert policies and learn a motor primitive embedding space. The trained neural probabilistic motor primitive system can perform one-shot imitation of whole-body humanoid behaviors, robustly mimicking unseen trajectories. Additionally, we demonstrate that it is also straightforward to train controllers to reuse the learned motor primitive space to solve tasks, and the resulting movements are relatively naturalistic. To support the training of our model, we compare two approaches for offline policy cloning, including an experience efficient method which we call linear feedback policy cloning. We encourage readers to view a supplementary video ( this https URL ) summarizing our results.
The origins of the circumgalactic medium in the FIRE simulations

We use a particle tracking analysis to study the origins of the circumgalactic medium (CGM), separating it into (1) accretion from the intergalactic medium (IGM), (2) wind from the central galaxy, and (3) gas ejected from other galaxies. Our sample consists of 21 FIRE-2 simulations, spanning the halo mass range Mh ∼ 1010–1012 M⊙, and we focus on z = 0.25 and z = 2. Owing to strong stellar feedback, only ∼L⋆ haloes retain a baryon mass $\gtrsim\! 50\hbox{ per cent}$ of their cosmic budget. Metals are more efficiently retained by haloes, with a retention fraction $\gtrsim\! 50\hbox{ per cent}$. Across all masses and redshifts analysed $\gtrsim \!60\hbox{ per cent}$ of the CGM mass originates as IGM accretion (some of which is associated with infalling haloes). Overall, the second most important contribution is wind from the central galaxy, though gas ejected or stripped from satellites can contribute a comparable mass in ∼L⋆ haloes. Gas can persist in the CGM for billions of years, resulting in well mixed-halo gas. Sightlines through the CGM are therefore likely to intersect gas of multiple origins. For low-redshift ∼L⋆ haloes, cool gas (T < 104.7 K) is distributed on average preferentially along the galaxy plane, however with strong halo-to-halo variability. The metallicity of IGM accretion is systematically lower than the metallicity of winds (typically by ≳1 dex), although CGM and IGM metallicities depend significantly on the treatment of subgrid metal diffusion. Our results highlight the multiple physical mechanisms that contribute to the CGM and will inform observational efforts to develop a cohesive picture.
On the correlation between the local dark matter and stellar velocities

The dark matter velocity distribution in the Solar neighbourhood is an important astrophysical input which enters in the predicted event rate of dark matter direct detection experiments. It has been recently suggested that the local dark matter velocity distribution can be inferred from that of old or metal-poor stars in the Milky Way. We investigate this potential relation using six high resolution magneto-hydrodynamical simulations of Milky Way-like galaxies of the Auriga project. We do not find any correlation between the velocity distributions of dark matter and old stars in the Solar neighbourhood. Likewise, there are no strong correlations between the local velocity distributions of dark matter and metal-poor stars selected by applying reasonable cuts on metallicity. In some simulated galaxies, extremely metal-poor stars have a velocity distribution that is statistically consistent with that of the dark matter, but the sample of such stars is so small that we cannot draw any strong conclusions.
Dwarf galaxies in CDM, WDM, and SIDM: disentangling baryons and dark matter physics

We present a suite of FIRE-2 cosmological zoom-in simulations of isolated field dwarf galaxies, all with masses of $M_{\rm halo} \approx 10^{10}\, {\rm M}_{\odot }$ at z = 0, across a range of dark matter models. For the first time, we compare how both self-interacting dark matter (SIDM) and/or warm dark matter (WDM) models affect the assembly histories as well as the central density structure in fully hydrodynamical simulations of dwarfs. Dwarfs with smaller stellar half-mass radii (r1/2 < 500 pc) have lower σ⋆/Vmax ratios, reinforcing the idea that smaller dwarfs may reside in haloes that are more massive than is naively expected. The majority of dwarfs simulated with self-interactions actually experience contraction of their inner density profiles with the addition of baryons relative to the cores produced in dark-matter-only runs, though the simulated dwarfs are always less centrally dense than in ΛCDM. The V1/2–r1/2 relation across all simulations is generally consistent with observations of Local Field dwarfs, though compact objects such as Tucana provide a unique challenge. Overall, the inclusion of baryons substantially reduces any distinct signatures of dark matter physics in the observable properties of dwarf galaxies. Spatially resolved rotation curves in the central regions (<400 pc) of small dwarfs could provide a way to distinguish between CDM, WDM, and SIDM, however: at the masses probed in this simulation suite, cored density profiles in dwarfs with small r1/2 values can only originate from dark matter self-interactions.
Unrepresentative video data: A review and evaluation

It is well known that the quality and quantity of training data are significant factors which affect the development and performance of machine intelligence algorithms. Without representative data, neither scientists nor algorithms would be able to accurately capture the visual details of objects, actions or scenes. An evaluation methodology which filters data quality does not yet exist, and currently, the validation of the data depends solely on human factor. This study reviews several public datasets and discusses their limitations and issues regarding quality, feasibility, adaptation and availability of training data. A simple approach to evaluate (i.e. automatically "clean" samples) training data is proposed with the use of real events recorded on the YouTube platform. This study focuses on action recognition data and particularly on human fall detection datasets. However, the limitations described in this paper apply in virtually all datasets.
Semantic Part Detection via Matching: Learning to Generalize to Novel Viewpoints From Limited Training Data

Detecting semantic parts of an object is a challenging task, particularly because it is hard to annotate semantic parts and construct large datasets. In this paper, we present an approach which can learn from a small annotated dataset containing a limited range of viewpoints and generalize to detect semantic parts for a much larger range of viewpoints. The approach is based on our matching algorithm, which is used for finding accurate spatial correspondence between two images and transplanting semantic parts annotated on one image to the other. Images in the training set are matched to synthetic images rendered from a 3D CAD model, following which a clustering algorithm is used to automatically annotate semantic parts of the CAD model. During the testing period, this CAD model can synthesize annotated images under every viewpoint. These synthesized images are matched to images in the testing set to detect semantic parts in novel viewpoints. Our algorithm is simple, intuitive, and contains very few parameters. Experiments show our method outperforms standard deep learning approaches and, in particular, performs much better on novel viewpoints. For facilitating the future research, code is available: