nils-herrmann/clean_scirepeval huggingface dataset:
Empirical Likelihood for Contextual Bandits

We apply empirical likelihood techniques to contextual bandit policy value estimation, confidence intervals, and learning. We propose a tighter estimator for off-policy evaluation with improved statistical performance over previous proposals. Coupled with this estimator is a confidence interval which also improves over previous proposals. We then harness these to improve learning from contextual bandit data. Each of these is empirically evaluated to show good performance against strong baselines in finite sample regimes.
Approximately Strategyproof Tournament Rules: On Large Manipulating Sets and Cover-Consistence

We consider the manipulability of tournament rules, in which $n$ teams play a round robin tournament and a winner is (possibly randomly) selected based on the outcome of all $\binom{n}{2}$ matches. Prior work defines a tournament rule to be $k$-SNM-$\alpha$ if no set of $\leq k$ teams can fix the $\leq \binom{k}{2}$ matches among them to increase their probability of winning by $>\alpha$ and asks: for each $k$, what is the minimum $\alpha(k)$ such that a Condorcet-consistent (i.e. always selects a Condorcet winner when one exists) $k$-SNM-$\alpha(k)$ tournament rule exists? 
A simple example witnesses that $\alpha(k) \geq \frac{k-1}{2k-1}$ for all $k$, and [Schneider et al., 2017] conjectures that this is tight (and prove it is tight for $k=2$). Our first result refutes this conjecture: there exists a sufficiently large $k$ such that no Condorcet-consistent tournament rule is $k$-SNM-$1/2$. Our second result leverages similar machinery to design a new tournament rule which is $k$-SNM-$2/3$ for all $k$ (and this is the first tournament rule which is $k$-SNM-$(<1)$ for all $k$). 
Our final result extends prior work, which proves that single-elimination bracket with random seeding is $2$-SNM-$1/3$([Schneider et al., 2017]), in a different direction by seeking a stronger notion of fairness than Condorcet-consistence. We design a new tournament rule, which we call Randomized-King-of-the-Hill, which is $2$-SNM-$1/3$ and \emph{cover-consistent} (the winner is an uncovered team with probability $1$).
Efficient Project Gradient Descent for Ensemble Adversarial Attack

Recent advances show that deep neural networks are not robust to deliberately crafted adversarial examples which many are generated by adding human imperceptible perturbation to clear input. Consider $l_2$ norms attacks, Project Gradient Descent (PGD) and the Carlini and Wagner (C\&W) attacks are the two main methods, where PGD control max perturbation for adversarial examples while C\&W approach treats perturbation as a regularization term optimized it with loss function together. If we carefully set parameters for any individual input, both methods become similar. In general, PGD attacks perform faster but obtains larger perturbation to find adversarial examples than the C\&W when fixing the parameters for all inputs. In this report, we propose an efficient modified PGD method for attacking ensemble models by automatically changing ensemble weights and step size per iteration per input. This method generates smaller perturbation adversarial examples than PGD method while remains efficient as compared to C\&W method. Our method won the first place in IJCAI19 Targeted Adversarial Attack competition.
Photon sphere and reentrant phase transition of charged Born-Infeld-AdS black holes

In this paper, we extend the study of the relationship between the photon sphere and the thermodynamic phase transition, especially the reentrant phase transition, to this black hole background. According to the number of the thermodynamic critical points, the black hole systems are divided into four cases with different values of Born-Infeld parameter $b$, where the black hole systems can have no phase transition, reentrant phase transition, or Van der Waals-like phase transition. For these different cases, we obtain the corresponding phase structures in pressure-temperature diagram and temperature-specific volume diagram. The tiny differences between these cases are clearly displayed. On the other hand, the radius $r_{\rm ps}$ and the minimal impact parameter $u_{\rm ps}$ of the photon sphere are calculated via the effective potential of the radial motion of photons. For different cases, $r_{\rm ps}$ and $u_{\rm ps}$ are found to have different behaviors. In particular, with the increase of $r_{\rm ps}$ or $u_{\rm ps}$, the temperature possesses a decrease-increase-decrease-increase behavior for fixed pressure if there exists the reentrant phase transition. While for fixed temperature, the pressure will show an increase-decrease-increase-decrease behavior instead. These behaviors are quite different from that of the Van der Waals-like phase transition. Near the critical point, the changes of $r_{\rm ps}$ and $u_{\rm ps}$ among the black hole phase transition confirm an universal critical exponent $\frac{1}{2}$. Therefore, all the results indicate that, for the charged Born-Infeld-AdS black holes, not only the Van der Waals-like phase transition, but also the reentrant phase transition can be reflected through the photon sphere.
Benchmarking Minimax Linkage

Minimax linkage was first introduced by Ao et al.  in 2004, as an alternative to standard linkage methods used in hierarchical clustering. Minimax linkage relies on distances to a prototype for each cluster; this prototype can be thought of as a representative object in the cluster, hence improving the interpretability of clustering results. Bien and Tibshirani analyzed properties of this method in 2011 , popularizing the method within the statistics community. Additionally, they performed comparisons of minimax linkage to standard linkage methods, making use of five data sets and two different evaluation metrics (distance to prototype and misclassification rate). In an effort to expand upon their work and evaluate minimax linkage more comprehensively, our benchmark study focuses on thorough method evaluation via multiple performance metrics on several well-described data sets. We also make all code and data publicly available through an R package, for full reproducibility. Similarly to , we find that minimax linkage often produces the smallest maximum minimax radius of all linkage methods, meaning that minimax linkage produces clusters where objects in a cluster are tightly clustered around their prototype. This is true across a range of values for the total number of clusters (k). However, this is not always the case, and special attention should be paid to the case when k is the true known value. For true k, minimax linkage does not always perform the best in terms of all the evaluation metrics studied, including maximum minimax radius. This paper was motivated by the IFCS Cluster Benchmarking Task Force's call for clustering benchmark studies and the white paper , which put forth guidelines and principles for comprehensive benchmarking in clustering. Our work is designed to be a neutral benchmark study of minimax linkage.
Extension of Rough Set Based on Positive Transitive Relation

The application of rough set theory in incomplete information systems is a key problem in practice since missing values almost always occur in knowledge acquisition due to the error of data measuring, the limitation of data collection, or the limitation of data comprehension, etc. An incomplete information system is mainly processed by compressing the indiscernibility relation. The existing rough set extension models based on tolerance or symmetric similarity relations typically discard one relation among the reflexive, symmetric and transitive relations, especially the transitive relation. In order to overcome the limitations of the current rough set extension models, we define a new relation called the positive transitive relation and then propose a novel rough set extension model built upon which. The new model holds the merit of the existing rough set extension models while avoids their limitations of discarding transitivity or symmetry. In comparison to the existing extension models, the proposed model has a better performance in processing the incomplete information systems while substantially reducing the computational complexity, taking into account the relation of tolerance and similarity of positive transitivity, and supplementing the related theories in accordance to the intuitive classification of incomplete information. In summary, the positive transitive relation can improve current theoretical analysis of incomplete information systems and the newly proposed extension model is more suitable for processing incomplete information systems and has a broad application prospect.
Dissecting Content and Context in Argumentative Relation Analysis

When assessing relations between argumentative units (e.g., support or attack), computational systems often exploit disclosing indicators or markers that are not part of elementary argumentative units (EAUs) themselves, but are gained from their context (position in paragraph, preceding tokens, etc.). We show that this dependency is much stronger than previously assumed. In fact, we show that by completely masking the EAU text spans and only feeding information from their context, a competitive system may function even better. We argue that an argument analysis system that relies more on discourse context than the argument’s content is unsafe, since it can easily be tricked. To alleviate this issue, we separate argumentative units from their context such that the system is forced to model and rely on an EAU’s content. We show that the resulting classification system is more robust, and argue that such models are better suited for predicting argumentative relations across documents.
When Unseen Domain Generalization is Unnecessary? Rethinking Data Augmentation

Recent advances in deep learning for medical image segmentation demonstrate expert-level accuracy. However, in clinically realistic environments, such methods have marginal performance due to differences in image domains, including different imaging protocols, device vendors and patient populations. Here we consider the problem of domain generalization, when a model is trained once, and its performance generalizes to unseen domains. Intuitively, within a specific medical imaging modality the domain differences are smaller relative to natural images domain variability. We rethink data augmentation for medical 3D images and propose a deep stacked transformations (DST) approach for domain generalization. Specifically, a series of n stacked transformations are applied to each image in each mini-batch during network training to account for the contribution of domain-specific shifts in medical images. We comprehensively evaluate our method on three tasks: segmentation of whole prostate from 3D MRI, left atrial from 3D MRI, and left ventricle from 3D ultrasound. We demonstrate that when trained on a small source dataset, (i) on average, DST models on unseen datasets degrade only by 11% (Dice score change), compared to the conventional augmentation (degrading 39%) and CycleGAN-based domain adaptation method (degrading 25%); (ii) when evaluation on the same domain, DST is also better albeit only marginally. (iii) When training on large-sized data, DST on unseen domains reaches performance of state-of-the-art fully supervised models. These findings establish a strong benchmark for the study of domain generalization in medical imaging, and can be generalized to the design of robust deep segmentation models for clinical deployment.
Classifying the reported ability in clinical mobility descriptions

Assessing how individuals perform different activities is key information for modeling health states of individuals and populations. Descriptions of activity performance in clinical free text are complex, including syntactic negation and similarities to textual entailment tasks. We explore a variety of methods for the novel task of classifying four types of assertions about activity performance: Able, Unable, Unclear, and None (no information). We find that ensembling an SVM trained with lexical features and a CNN achieves 77.9% macro F1 score on our task, and yields nearly 80% recall on the rare Unclear and Unable samples. Finally, we highlight several challenges in classifying performance assertions, including capturing information about sources of assistance, incorporating syntactic structure and negation scope, and handling new modalities at test time. Our findings establish a strong baseline for this novel task, and identify intriguing areas for further research.
Video Modeling With Correlation Networks

Motion is a salient cue to recognize actions in video. Modern action recognition models leverage motion information either explicitly by using optical flow as input or implicitly by means of 3D convolutional filters that simultaneously capture appearance and motion information. This paper proposes an alternative approach based on a learnable correlation operator that can be used to establish frame-to-frame matches over convolutional feature maps in the different layers of the network. The proposed architecture enables the fusion of this explicit temporal matching information with traditional appearance cues captured by 2D convolution. Our correlation network compares favorably with widely-used 3D CNNs for video modeling, and achieves competitive results over the prominent two-stream network while being much faster to train. We empirically demonstrate that correlation networks produce strong results on a variety of video datasets, and outperform the state of the art on four popular benchmarks for action recognition: Kinetics, Something-Something, Diving48, and Sports1M.
Understanding Heating in Active Region Cores through Machine Learning. I. Numerical Modeling and Predicted Observables

To adequately constrain the frequency of energy deposition in active region cores in the solar corona, systematic comparisons between detailed models and observational data are needed. In this paper, we describe a pipeline for forward modeling active region emission using magnetic field extrapolations and field-aligned hydrodynamic models. We use this pipeline to predict time-dependent emission from active region NOAA 1158 for low-, intermediate-, and high-frequency nanoflares. In each pixel of our predicted multi-wavelength, time-dependent images, we compute two commonly used diagnostics: the emission measure slope and the time lag. We find that signatures of the heating frequency persist in both of these diagnostics. In particular, our results show that the distribution of emission measure slopes narrows and the mean decreases with decreasing heating frequency and that the range of emission measure slopes is consistent with past observational and modeling work. Furthermore, we find that the time lag becomes increasingly spatially coherent with decreasing heating frequency while the distribution of time lags across the whole active region becomes more broad with increasing heating frequency. In a follow-up paper, we train a random forest classifier on these predicted diagnostics and use this model to classify real observations of NOAA 1158 in terms of the underlying heating frequency.
Real or Fake? Learning to Discriminate Machine from Human Generated Text

Energy-based models (EBMs), a.k.a. un-normalized models, have had recent successes in continuous spaces. However, they have not been successfully applied to model text sequences. While decreasing the energy at training samples is straightforward, mining (negative) samples where the energy should be increased is difficult. In part, this is because standard gradient-based methods are not readily applicable when the input is high-dimensional and discrete. Here, we side-step this issue by generating negatives using pre-trained auto-regressive language models. The EBM then works in the residual of the language model; and is trained to discriminate real text from text generated by the auto-regressive models. We investigate the generalization ability of residual EBMs, a pre-requisite for using them in other applications. We extensively analyze generalization for the task of classifying whether an input is machine or human generated, a natural task given the training loss and how we mine negatives. Overall, we observe that EBMs can generalize remarkably well to changes in the architecture of the generators producing negatives. However, EBMs exhibit more sensitivity to the training set used by such generators.
Watch, Try, Learn: Meta-Learning from Demonstrations and Reward

Imitation learning allows agents to learn complex behaviors from demonstrations. However, learning a complex vision-based task may require an impractical number of demonstrations. Meta-imitation learning is a promising approach towards enabling agents to learn a new task from one or a few demonstrations by leveraging experience from learning similar tasks. In the presence of task ambiguity or unobserved dynamics, demonstrations alone may not provide enough information; an agent must also try the task to successfully infer a policy. In this work, we propose a method that can learn to learn from both demonstrations and trial-and-error experience with sparse reward feedback. In comparison to meta-imitation, this approach enables the agent to effectively and efficiently improve itself autonomously beyond the demonstration data. In comparison to meta-reinforcement learning, we can scale to substantially broader distributions of tasks, as the demonstration reduces the burden of exploration. Our experiments show that our method significantly outperforms prior approaches on a set of challenging, vision-based control tasks.
Searching for dark matter in the Galactic halo with a wide field of view TeV gamma-ray observatory in the Southern Hemisphere

Despite mounting evidence that dark matter (DM) exists in the Universe, its fundamental nature remains unknown. We present sensitivity estimates to detect DM particles with a future very-high-energy (≳ TeV) wide field-of-view gamma-ray observatory in the Southern Hemisphere. This observatory would search for gamma rays from the annihilation or decay of DM particles in the Galactic halo. With a wide field of view, both the Galactic Center and a large fraction of the Galactic halo will be detectable with unprecedented sensitivity to DM in the mass range of ∼500 GeV to ∼2 PeV . These results, combined with those from other present and future gamma-ray observatories, will likely probe the thermal relic annihilation cross section of Weakly Interacting Massive Particles for all masses from ∼80 TeV down to the GeV range in most annihilation channels.
Deep Contextualized Biomedical Abbreviation Expansion

Automatic identification and expansion of ambiguous abbreviations are essential for biomedical natural language processing applications, such as information retrieval and question answering systems. In this paper, we present DEep Contextualized Biomedical Abbreviation Expansion (DECBAE) model. DECBAE automatically collects substantial and relatively clean annotated contexts for 950 ambiguous abbreviations from PubMed abstracts using a simple heuristic. Then it utilizes BioELMo to extract the contextualized features of words, and feed those features to abbreviation-specific bidirectional LSTMs, where the hidden states of the ambiguous abbreviations are used to assign the exact definitions. Our DECBAE model outperforms other baselines by large margins, achieving average accuracy of 0.961 and macro-F1 of 0.917 on the dataset. It also surpasses human performance for expanding a sample abbreviation, and remains robust in imbalanced, low-resources and clinical settings.
Robust Bi-Tempered Logistic Loss Based on Bregman Divergences

We introduce a temperature into the exponential function and replace the softmax output layer of neural nets by a high temperature generalization. Similarly, the logarithm in the log loss we use for training is replaced by a low temperature logarithm. By tuning the two temperatures we create loss functions that are non-convex already in the single layer case. When replacing the last layer of the neural nets by our bi-temperature generalization of logistic loss, the training becomes more robust to noise. We visualize the effect of tuning the two temperatures in a simple setting and show the efficacy of our method on large data sets. Our methodology is based on Bregman divergences and is superior to a related two-temperature method using the Tsallis divergence.
Partially Linear Additive Gaussian Graphical Models

We propose a partially linear additive Gaussian graphical model (PLA-GGM) for the estimation of associations between random variables distorted by observed confounders. Model parameters are estimated using an $L_1$-regularized maximal pseudo-profile likelihood estimator (MaPPLE) for which we prove $\sqrt{n}$-sparsistency. Importantly, our approach avoids parametric constraints on the effects of confounders on the estimated graphical model structure. Empirically, the PLA-GGM is applied to both synthetic and real-world datasets, demonstrating superior performance compared to competing methods.
Online Forecasting of Total-Variation-bounded Sequences

We consider the problem of online forecasting of sequences of length $n$ with total-variation at most $C_n$ using observations contaminated by independent $\sigma$-subgaussian noise. We design an $O(n\log n)$-time algorithm that achieves a cumulative square error of $\tilde{O}(n^{1/3}C_n^{2/3}\sigma^{4/3} + C_n^2)$ with high probability.We also prove a lower bound that matches the upper bound in all parameters (up to a $\log(n)$ factor). To the best of our knowledge, this is the first \emph{polynomial-time} algorithm that achieves the optimal $O(n^{1/3})$ rate in forecasting total variation bounded sequences and the first algorithm that \emph{adapts to unknown} $C_n$. Our proof techniques leverage the special localized structure of Haar wavelet basis and the adaptivity to unknown smoothness parameters in the classical wavelet smoothing [Donoho et al., 1998]. We also compare our model to the rich literature of dynamic regret minimization and nonstationary stochastic optimization, where our problem can be treated as a special case. We show that the workhorse in those settings --- online gradient descent and its variants with a fixed restarting schedule --- are instances of a class of \emph{linear forecasters} that require a suboptimal regret of $\tilde{\Omega}(\sqrt{n})$. This implies that the use of more adaptive algorithms is necessary to obtain the optimal rate.
Complex structure degenerations and collapsing of Calabi-Yau metrics

In this paper, we make progress on understanding the collapsing behavior of Calabi-Yau metrics on a degenerating family of polarized Calabi-Yau manifolds. In the case of a family of smooth Calabi-Yau hypersurfaces in projective space degenerating into the transversal union of two smooth Fano hypersurfaces in a generic way, we obtain a complete result in all dimensions establishing explicit and precise relationships between the metric collapsing and complex structure degenerations. This result is new even in complex dimension two. This is achieved via gluing and singular perturbation techniques, and a key geometric ingredient involving the construction of certain (not necessarily smooth) Kahler metrics with torus symmetry. We also discuss possible extensions of this result to more general settings.