nils-herrmann/clean_scirepeval huggingface dataset:
Automatic Hip Fracture Identification and Functional Subclassification with Deep Learning

To investigate the feasibility of automatic identification and classification of hip fractures using deep learning, which may improve outcomes by reducing diagnostic errors and decreasing time to operation.



Hip and pelvic radiographs from 1118 studies were reviewed, and 3026 hips were labeled via bounding boxes and classified as normal, displaced femoral neck fracture, nondisplaced femoral neck fracture, intertrochanteric fracture, previous open reduction and internal fixation, or previous arthroplasty. A deep learning-based object detection model was trained to automate the placement of the bounding boxes. A Densely Connected Convolutional Neural Network (or DenseNet) was trained on a subset of the bounding box images, and its performance was evaluated on a held-out test set and by comparison on a 100-image subset with two groups of human observers: fellowship-trained radiologists and orthopedists; senior residents in emergency medicine, radiology, and orthopedics.



The binary accuracy for detecting a fracture of this model was 93.7% (95% confidence interval [CI]: 90.8%, 96.5%), with a sensitivity of 93.2% (95% CI: 88.9%, 97.1%) and a specificity of 94.2% (95% CI: 89.7%, 98.4%). Multiclass classification accuracy was 90.8% (95% CI: 87.5%, 94.2%). When compared with the accuracy of human observers, the accuracy of the model achieved an expert-level classification, at the very least, under all conditions. Additionally, when the model was used as an aid, human performance improved, with aided resident performance approximating unaided fellowship-trained expert performance in the multiclass classification.



A deep learning model identified and classified hip fractures with expert-level performance, at the very least, and when used as an aid, improved human performance, with aided resident performance approximating that of unaided fellowship-trained attending physicians.Supplemental material is available for this article.
Characteristic Polynomials of Complex Random Matrices and Painlevé Transcendents

We study expectations of powers and correlation functions for characteristic polynomials of N×N non-Hermitian random matrices. For the 1-point and 2-point correlation function, we obtain several characterizations in terms of Painleve transcendents, both at finite-N and asymptotically as N→∞. In the asymptotic analysis, two regimes of interest are distinguished: boundary asymptotics where parameters of the correlation function can touch the boundary of the limiting eigenvalue support and bulk asymptotics where they are strictly inside the support. For the complex Ginibre ensemble this involves Painleve IV at the boundary as N \to \infty. Our approach, together with the results in \cite{HW17} suggests that this should arise in a much broader class of planar models. For the bulk asymptotics, one of our results can be interpreted as the merging of two `planar Fisher-Hartwig singularities' where Painleve V arises in the asymptotics. We also discuss the correspondence of our results with a normal matrix model with d-fold rotational symmetries known as the \textit{lemniscate ensemble}, recently studied in \cite{BGM, BGG18}. Our approach is flexible enough to apply to non-Gaussian models such as the truncated unitary ensemble or induced Ginibre ensemble; we show that in the former case Painleve VI arises at finite-N. Scaling near the boundary leads to Painleve V, in contrast to the Ginibre ensemble.
Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification

Federated Learning enables visual models to be trained in a privacy-preserving way using real-world data from mobile devices. Given their distributed nature, the statistics of the data across these devices is likely to differ significantly. In this work, we look at the effect such non-identical data distributions has on visual classification via Federated Learning. We propose a way to synthesize datasets with a continuous range of identicalness and provide performance measures for the Federated Averaging algorithm. We show that performance degrades as distributions differ more, and propose a mitigation strategy via server momentum. Experiments on CIFAR-10 demonstrate improved classification performance over a range of non-identicalness, with classification accuracy improved from 30.1% to 76.9% in the most skewed settings.
Slice-based Learning: A Programming Model for Residual Learning in Critical Data Slices

In real-world machine learning applications, data subsets correspond to especially critical outcomes: vulnerable cyclist detections are safety-critical in an autonomous driving task, and "question" sentences might be important to a dialogue agent's language understanding for product purposes. While machine learning models can achieve high quality performance on coarse-grained metrics like F1-score and overall accuracy, they may underperform on critical subsets-we define these as slices, the key abstraction in our approach. To address slice-level performance, practitioners often train separate "expert" models on slice subsets or use multi-task hard parameter sharing. We propose Slice-based Learning, a new programming model in which the slicing function (SF), a programming interface, specifies critical data subsets for which the model should commit additional capacity. Any model can leverage SFs to learn slice expert representations, which are combined with an attention mechanism to make slice-aware predictions. We show that our approach maintains a parameter-efficient representation while improving over baselines by up to 19.0 F1 on slices and 4.6 F1 overall on datasets spanning language understanding (e.g. SuperGLUE), computer vision, and production-scale industrial systems.
Probing the Information Encoded in X-Vectors

Deep neural network based speaker embeddings, such as x-vectors, have been shown to perform well in text-independent speaker recognition/verification tasks. In this paper, we use simple classifiers to investigate the contents encoded by x-vector embeddings. We probe these embeddings for information related to the speaker, channel, transcription (sentence, words, phones), and meta information about the utterance (duration and augmentation type), and compare these with the information encoded by i-vectors across a varying number of dimensions. We also study the effect of data augmentation during extractor training on the information captured by x-vectors. Experiments on the RedDots data set show that x-vectors capture spoken content and channel-related information, while performing well on speaker verification tasks.
Addressing Semantic Drift in Question Generation for Semi-Supervised Question Answering

Text-based Question Generation (QG) aims at generating natural and relevant questions that can be answered by a given answer in some context. Existing QG models suffer from a “semantic drift” problem, i.e., the semantics of the model-generated question drifts away from the given context and answer. In this paper, we first propose two semantics-enhanced rewards obtained from downstream question paraphrasing and question answering tasks to regularize the QG model to generate semantically valid questions. Second, since the traditional evaluation metrics (e.g., BLEU) often fall short in evaluating the quality of generated questions, we propose a QA-based evaluation method which measures the QG model’s ability to mimic human annotators in generating QA training data. Experiments show that our method achieves the new state-of-the-art performance w.r.t. traditional metrics, and also performs best on our QA-based evaluation metrics. Further, we investigate how to use our QG model to augment QA datasets and enable semi-supervised QA. We propose two ways to generate synthetic QA pairs: generate new questions from existing articles or collect QA pairs from new articles. We also propose two empirically effective strategies, a data filter and mixing mini-batch training, to properly use the QG-generated data for QA. Experiments show that our method improves over both BiDAF and BERT QA baselines, even without introducing new articles.
Optical, J and K light curves of XTE J1118+480 = KV UMa: the mass of the black hole and the spectrum of the non-stellar component

Optical, J and K photometric observations of the KV UMa black hole X-ray nova in its quiescent state obtained in 2017–2018 are presented. A significant flickering within light curves was not detected, although the average brightness of the system faded by ≈0.1m over 350 d. Changes in the average brightness were not accompanied with an increase or decrease in the flickering. From the modelling of five light curves the inclination of the KV UMa orbit and the black hole mass were obtained: i = 74 ± 4°, $M_\mathrm{BH}=(7.06\rm {-}7.24)\, \mathrm{M}_{\odot }$, depending on the mass ratio used. The non-stellar component of the spectrum in the range λ = 6400–22 000 Å can be fitted by a power law Fλ ∼ λα, α ≈ −1.8. The accretion disc orientation angle changed from one epoch to another. The model with spots on the star was inadequate. Evolutionary calculations using the scenario machine code were performed for low-mass X-ray binaries, with a recently discovered anomalously rapid decrease of the orbital period taken into account. We show that the observed decrease can be consistent with the magnetic stellar wind of the optical companion, whose magnetic field was increased during the common-envelope stage. Several constraints on evolutionary scenario parameters were developed.
Crank up the Volume: Preference Bias Amplification in Collaborative Recommendation

Recommender systems are personalized: we expect the results given to a particular user to reflect that user's preferences. Some researchers have studied the notion of calibration, how well recommendations match users' stated preferences, and bias disparity the extent to which mis-calibration affects different user groups. In this paper, we examine bias disparity over a range of different algorithms and for different item categories and demonstrate significant differences between model-based and memory-based algorithms.
Sample Complexity of Probabilistic Roadmaps via $\epsilon$-nets

We study fundamental theoretical aspects of probabilistic roadmaps (PRM) in the finite time (non-asymptotic) regime. In particular, we investigate how completeness and optimality guarantees of the approach are influenced by the underlying deterministic sampling distribution X and connection radius r > 0. We develop the notion of (δ, )-completeness of the parameters X , r, which indicates that for every motionplanning problem of clearance at least δ > 0, PRM using X , r returns a solution no longer than 1+ times the shortest δ-clear path. Leveraging the concept of -nets, we characterize in terms of lower and upper bounds the number of samples needed to guarantee (δ, )-completeness. This is in contrast with previous work which mostly considered the asymptotic regime in which the number of samples tends to infinity. In practice, we propose a sampling distribution inspired by -nets that achieves nearly the same coverage as grids while using fewer samples.
Supervised Learning for Physical Layer Based Message Authentication in URLLC Scenarios

PHYSEC based message authentication can, as an alternative to conventional security schemes, be applied within Ultra Reliable Low Latency Communication (URLLC) scenarios in order to meet the requirement of secure user data transmissions in the sense of authenticity and integrity. In this work, we investigate the performance of supervised learning classifiers for discriminating legitimate transmitters from illegimate ones in such scenarios. We further present our methodology of data collection using Software Defined Radio (SDR) platforms and the data processing pipeline including e.g. necessary preprocessing steps. Finally, the performance of the considered supervised learning schemes under different side conditions is presented.
Using deep learning to localize gravitational wave sources

In this paper, we report on the construction of a deep Artificial Neural Network (ANN) to localize simulated gravitational wave signals in the sky with high accuracy. We have modelled the sky as a sphere and have considered cases where the sphere is divided into 18, 50, 128, 1024, 2048 and 4096 sectors. The sky direction of the gravitational wave source is estimated by classifying the signal into one of these sectors based on it's right ascension and declination values for each of these cases. In order to do this, we have injected simulated binary black hole gravitational wave signals of component masses sampled uniformly between 30-80 solar mass into Gaussian noise and used the whitened strain values to obtain the input features for training our ANN. We input features such as the delays in arrival times, phase differences and amplitude ratios at each of the three detectors Hanford, Livingston and Virgo, from the raw time-domain strain values as well as from analytical versions of these signals, obtained through Hilbert transformation. We show that our model is able to classify gravitational wave samples, not used in the training process, into their correct sectors with very high accuracy (>90%) for coarse angular resolution using 18, 50 and 128 sectors. We also test our localization on test samples with injection parameters of the published LIGO binary black hole merger events GW150914, GW170818 and GW170823 for 1024, 2048 and 4096 sectors and compare the result with that from BAYESTAR and Parameter Estimation (PE). In addition, we report that the time taken by our model to localize one GW signal is around 0.018 secs on 14 Intel Xeon CPU cores.
Machine-learning interpolation of population-synthesis simulations to interpret gravitational-wave observations: A case study

We report on advances to interpret current and future gravitational-wave events in light of astrophysical simulations. A machine-learning emulator is trained on numerical population-synthesis predictions and inserted into a Bayesian hierarchical framework. In this case study, a modest but state-of-the-art suite of simulations of isolated binary stars is interpolated across two event parameters and one population parameter. The validation process of our pipelines highlights how omitting some of the event parameters might cause errors in estimating selection effects, which propagates as systematics to the final population inference. Using LIGO/Virgo data from O1 and O2 we infer that black holes in binaries are most likely to receive natal kicks with one-dimensional velocity dispersion $\sigma$ = 105+44 km/s. Our results showcase potential applications of machine-learning tools in conjunction with population-synthesis simulations and gravitational-wave data.
A Forward Modeling Approach to AGN Variability--Method Description and Early Applications

We present a numerical framework for the variability of active galactic nuclei (AGNs), which links the variability of AGNs over a broad range of timescales and luminosities to the observed properties of the AGN population as a whole, and particularly the Eddington ratio distribution function. We implemented our framework on GPU architecture, relying on previously published time-series-generating algorithms. After extensive tests that characterize several intrinsic and numerical aspects of the simulations, we describe some applications used for current and future time-domain surveys and for the study of extremely variable sources (e.g., “changing-look” or flaring AGNs). Specifically, we define a simulation setup that reproduces the AGN variability observed in the (intermediate) Palomar Transient Factory survey and use it to forward model longer light curves of the kind that may be observed within the Large Synoptic Survey Telescope (LSST) main survey. Thanks to our efficient implementations, these simulations are able to cover, for example, over 1 Myr with a roughly weekly cadence. We envision that this framework will become highly valuable to prepare for, and best exploit, data from upcoming time-domain surveys, such as, for example, LSST.
Dynamical friction in a Fuzzy Dark Matter universe

We present an in-depth exploration of the phenomenon of dynamical friction in a universe where the dark matter is composed entirely of so-called Fuzzy Dark Matter (FDM), ultralight bosons of mass m∼𝒪(10−22) eV . We review the classical treatment of dynamical friction before presenting analytic results in the case of FDM for point masses, extended mass distributions, and FDM backgrounds with finite velocity dispersion. We then test these results against a large suite of fully non-linear simulations that allow us to assess the regime of applicability of the analytic results. We apply these results to a variety of astrophysical problems of interest, including infalling satellites in a galactic dark matter background, and determine that (1) for FDM masses m≳ 10−21 eV c−2, the timing problem of the Fornax dwarf spheroidal's globular clusters is no longer solved and (2) the effects of FDM on the process of dynamical friction for satellites of total mass M and relative velocity vrel should require detailed numerical simulations for (M/109 M⊙) (m/10−22 eV)(100 km s−1/vrel) ∼ 1, parameters which would lie outside the validated range of applicability of any currently developed analytic theory, due to transient wave structures in the time-dependent regime.
Finding the Brightest Cosmic Beacons in the Southern Hemisphere

The study of absorptions along the lines of sight to bright high-z QSOs is an invaluable cosmological tool that provides a wealth of information on the inter-/circum-galactic medium, dark matter, primordial elements, reionization, fundamental constants, and general relativity. Unfortunately, the number of bright (i ≲ 18) QSOs at z ≳ 2 in the southern hemisphere is much lower than that in the north, due to the lack of wide multiwavelength surveys at decl. δ < 0°, hampering the effectiveness of observations from southern observatories. In this work we present a new method based on Canonical Correlation Analysis to identify such objects, taking advantage of a number of available databases: Skymapper, Gaia DR2, WISE, and 2MASS. Our QSO candidate sample lists 1476 sources with i < 18 over 12,400 square degrees in the southern hemisphere. With a preliminary campaign we observed spectroscopically 69 of them, confirming 54 new bright QSOs at z > 2.5, corresponding to a success rate of our method of ∼80%. Furthermore, we estimate a completeness of ∼90% of our sample at completion of our observation campaign. The new QSOs confirmed by this first and the forthcoming campaigns (QUBRICS survey) will be the targets of subsequent studies using higher resolution spectrographs, like ESPRESSO, UVES, and (in the long term) ELT/HIRES.
Magnetic Resonance Fingerprinting Reconstruction Using Recurrent Neural Networks

Magnetic Resonance Fingerprinting (MRF) is an imaging technique acquiring unique time signals for different tissues. Although the acquisition is highly accelerated, the reconstruction time remains a problem, as the state-of-the-art template matching compares every signal with a set of possible signals. To overcome this limitation, deep learning based approaches, e.g. Convolutional Neural Networks (CNNs) have been proposed. In this work, we investigate the applicability of Recurrent Neural Networks (RNNs) for this reconstruction problem, as the signals are correlated in time. Compared to previous methods based on CNNs, RNN models yield significantly improved results using in-vivo data.
Horizontal Flows and Manifold Stochastics in Geometric Deep Learning

We introduce two constructions in geometric deep learning for 1) transporting orientation-dependent convolutional filters over a manifold in a continuous way and thereby defining a convolution operator that naturally incorporates the rotational effect of holonomy; and 2) allowing efficient evaluation of manifold convolution layers by sampling manifold valued random variables that center around a weighted diffusion mean. Both methods are inspired by stochastics on manifolds and geometric statistics, and provide examples of how stochastic methods – here horizontal frame bundle flows and non-linear bridge sampling schemes, can be used in geometric deep learning. We outline the theoretical foundation of the two methods, discuss their relation to Euclidean deep networks and existing methodology in geometric deep learning, and establish important properties of the proposed constructions.
Dynamic Graph Algorithms and Graph Sparsification: New Techniques and Connections

Graphs naturally appear in several real-world contexts including social networks, the web network, and telecommunication networks. While the analysis and the understanding of graph structures have been a central area of study in algorithm design, the rapid increase of data sets over the last decades has posed new challenges for designing efficient algorithms that process large-scale graphs. These challenges arise from two usual assumptions in classical algorithm design, namely that graphs are static and that they fit into a single machine. However, in many application domains, graphs are subject to frequent changes over time, and their massive size makes them infeasible to be stored in the memory of a single machine. 
Driven by the need to devise new tools for overcoming such challenges, this thesis focuses on two areas of modern algorithm design that directly deal with processing massive graphs, namely dynamic graph algorithms and graph sparsification. We develop new algorithmic techniques from both dynamic and sparsification perspective for a multitude of graph-based optimization problems which lie at the core of Spectral Graph Theory, Graph Partitioning, and Metric Embeddings. Our algorithms are faster than any previous one and design smaller sparsifiers with better (approximation) quality. More importantly, this work introduces novel reduction techniques that show unexpected connections between seemingly different areas such as dynamic graph algorithms and graph sparsification.
Learning Household Task Knowledge from WikiHow Descriptions

Commonsense procedural knowledge is important for AI agents and robots that operate in a human environment. While previous attempts at constructing procedural knowledge are mostly rule- and template-based, recent advances in deep learning provide the possibility of acquiring such knowledge directly from natural language sources. As a first step in this direction, we propose a model to learn embeddings for tasks, as well as the individual steps that need to be taken to solve them, based on WikiHow articles. We learn these embeddings such that they are predictive of both step relevance and step ordering. We also experiment with the use of integer programming for inferring consistent global step orderings from noisy pairwise predictions.
Enabling Intuitive Human-Robot Teaming Using Augmented Reality and Gesture Control

Human-robot teaming offers great potential because of the opportunities to combine strengths of heterogeneous agents. However, one of the critical challenges in realizing an effective human-robot team is efficient information exchange - both from the human to the robot as well as from the robot to the human. In this work, we present and analyze an augmented reality-enabled, gesture-based system that supports intuitive human-robot teaming through improved information exchange. Our proposed system requires no external instrumentation aside from human-wearable devices and shows promise of real-world applicability for service-oriented missions. Additionally, we present preliminary results from a pilot study with human participants, and highlight lessons learned and open research questions that may help direct future development, fielding, and experimentation of autonomous HRI systems.