nils-herrmann/clean_scirepeval huggingface dataset:
Differential Equation Units: Learning Functional Forms of Activation Functions from Data

Most deep neural networks use simple, fixed activation functions, such as sigmoids or rectified linear units, regardless of domain or network structure. We introduce differential equation units (DEUs), an improvement to modern neural networks, which enables each neuron to learn a particular nonlinear activation function from a family of solutions to an ordinary differential equation. Specifically, each neuron may change its functional form during training based on the behavior of the other parts of the network. We show that using neurons with DEU activation functions results in a more compact network capable of achieving comparable, if not superior, performance when is compared to much larger networks.
The Recent Burstiness of Star Formation in Galaxies at z ∼ 4.5 from Hα Measurements

The redshift range z = 4–6 marks a transition phase between primordial and mature galaxy formation in which galaxies considerably increase their stellar mass, metallicity, and dust content. The study of galaxies in this redshift range is therefore important to understanding early galaxy formation and the fate of galaxies at later times. Here, we investigate the burstiness of the recent star formation history (SFH) of 221z ∼ 4.5 main-sequence galaxies at by comparing their ultra-violet (UV) continuum, Hα luminosity, and Hα equivalent-width (EW). The Hα properties are derived from the Spitzer [3.6 μm]−[4.5 μm] broadband color, thereby properly taking into account model and photometric uncertainties. We find a significant scatter between Hα- and UV-derived luminosities and star formation rates (SFRs). About half of the galaxies show a significant excess in Hα compared to expectations from a constant smooth SFH. We also find a tentative anticorrelation between Hα EW and stellar mass, ranging from 1000 Å at to below 100 Å at . Consulting models suggests that most z ∼ 4.5 galaxies had a burst of star formation within the last 50 Myr, increasing their SFRs by a factor of >5. The most massive galaxies on the other hand might decrease their SFRs and may be transitioning to a quiescent stage by z = 4. We identify differential dust attenuation (f) between stars and nebular regions as the main contributor to the uncertainty. With local galaxies selected by increasing Hα EW (reaching values similar to high-z galaxies), we predict that f approaches unity at z > 4, consistent with the extrapolation of measurements out to z = 2.
The population of galaxies that contribute to the H i mass function

We look at the contribution of different galaxy populations to the atomic hydrogen (H i) mass function (HIMF) and the H i density parameter, ΩH i, in the local Universe. Our analysis is based on a sample of 7857 H i selected galaxies selected from a volume common to the Sloan Digital Sky Survey (SDSS) and Arecibo Fast Legacy ALFA (40 per cent catalogue – α.40) surveys. We define different populations of galaxies in the colour (u − r)–magnitude (Mr) plane and compute the HIMF for each of them. Additionally we compute the HIMF for dark galaxies; these are undetected in SDSS and represent $\sim 2{{\ \rm per\ cent}}$ of the total sample. We find that the luminous red population dominates the total HIMF for $\log _{10}(M_{\rm{H\,{\small I}}}h^2_{70}/M_{\odot }) \ge 10.4$. The full red population – luminous and faint – represents about $\sim 17{{\ \rm per\ cent}}$ of the ΩH i budget, while that of the dark population is $\sim 3{{\ \rm per\ cent}}$. The HIMF about the knee, $\log _{10}(M_{\rm{H\,{\small I}}}h^2_{70}/M_{\odot }) \in [8,10.4]$, is dominated by the faint and luminous blue populations, the latter dominating at larger masses in this interval. Their total contribution to ΩH i is $\sim 55\!-\!70{{\ \rm per\ cent}}$, the variation depending on the definition of population. The dominant populations at the low-mass end, $\log _{10}(M_{\rm{H\,{\small I}}}h^2_{70}/M_{\odot }) \le 8.0$ are the faint blue and faint bluer populations, the latter’s dominance being sensitive to its definition. The full blue (blue–bluer luminous and faint) population represents $\sim 80{{\ \rm per\ cent}}$ of ΩH i. A bimodal HIMF suggested by our results is however not seen since the amplitude of the HIMF of the luminous red population is small compared to that of the luminous blue population.
Learning to Discriminate Perturbations for Blocking Adversarial Attacks in Text Classification

Adversarial attacks against machine learning models have threatened various real-world applications such as spam filtering and sentiment analysis. In this paper, we propose a novel framework, learning to discriminate perturbations (DISP), to identify and adjust malicious perturbations, thereby blocking adversarial attacks for text classification models. To identify adversarial attacks, a perturbation discriminator validates how likely a token in the text is perturbed and provides a set of potential perturbations. For each potential perturbation, an embedding estimator learns to restore the embedding of the original word based on the context and a replacement token is chosen based on approximate kNN search. DISP can block adversarial attacks for any NLP model without modifying the model structure or training procedure. Extensive experiments on two benchmark datasets demonstrate that DISP significantly outperforms baseline methods in blocking adversarial attacks for text classification. In addition, in-depth analysis shows the robustness of DISP across different situations.
The Roger–Yang skein algebra and the decorated Teichmüller space

Based on hyperbolic geometric considerations, Roger and Yang introduced an extension of the Kauffman bracket skein algebra that includes arcs. In particular, their skein algebra is a deformation quantization of a certain commutative curve algebra, and there is a Poisson algebra homomorphism between the curve algebra and the algebra of smooth functions on decorated Teichmuller space. In this paper, we consider surfaces with punctures which is not the 3-holed sphere and which have an ideal triangulation without self-folded edges or triangles. For those surfaces, we prove that Roger and Yang's Poisson algebra homomorphism is injective, and the skein algebra they defined have no zero divisors. A section about generalized corner coordinates for normal arcs may be of independent interest.
ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn Comparisons

While dialogue remains an important end-goal of natural language research, the difficulty of evaluation is an oft-quoted reason why it remains troublesome to make real progress towards its solution. Evaluation difficulties are actually two-fold: not only do automatic metrics not correlate well with human judgments, but also human judgments themselves are in fact difficult to measure. The two most used human judgment tests, single-turn pairwise evaluation and multi-turn Likert scores, both have serious flaws as we discuss in this work. 
We instead provide a novel procedure involving comparing two full dialogues, where a human judge is asked to pay attention to only one speaker within each, and make a pairwise judgment. The questions themselves are optimized to maximize the robustness of judgments across different annotators, resulting in better tests. We also show how these tests work in self-play model chat setups, resulting in faster, cheaper tests. We hope these tests become the de facto standard, and will release open-source code to that end.
A framework for seizure detection using effective connectivity, graph theory and deep modular neural networks

The electrical characteristics of the EEG signals can be used for seizure detection. Statistical independence between different brain regions is measured by functional brain connectivity (FBC). Specific directional effects can't consider by FBC and thus effective brain connectivity (EBC) is used to measure causal intervention between one neuronal region and the rest of the neuronal regions. Our main purpose is to provide a reliable automatic seizure detection approach. 
 
In this study, three new methods are provided. Deep modular neural network (DMNN) is developed based on a combination of various EBC classification results in the different frequencies. Another method is named "modular effective neural networks (MENN)". This method combines the classification results of the three different EBC in the specific frequency. "Modular frequency neural networks (MFNN)" is another method that combines the classification results of the specific EBC in the seven different frequencies. 
 
The mean accuracy of the MFNN are 97.14%, 98.53%, and 97.91% using directed transfer function, directed coherence, and generalized partial directed coherence, respectively. Using the MENN, the highest mean accuracy is 98.34%. Finally, DMNN has the highest mean accuracy which is equal to 99.43. To our best knowledge, the proposed method is a new method that provides the high accuracy in comparison to other studies which used MIT-CHB database. 
 
The knowledge of structure-function relationships between different areas of the brain is necessary for characterizing the underlying dynamics. Hence, features based on EBC can provide a reliable automatic seizure detection approach.
Solving Interpretable Kernel Dimension Reduction

Kernel dimensionality reduction (KDR) algorithms find a low dimensional representation of the original data by optimizing kernel dependency measures that are capable of capturing nonlinear relationships. The standard strategy is to first map the data into a high dimensional feature space using kernels prior to a projection onto a low dimensional space. While KDR methods can be easily solved by keeping the most dominant eigenvectors of the kernel matrix, its features are no longer easy to interpret. Alternatively, Interpretable KDR (IKDR) is different in that it projects onto a subspace \textit{before} the kernel feature mapping, therefore, the projection matrix can indicate how the original features linearly combine to form the new features. Unfortunately, the IKDR objective requires a non-convex manifold optimization that is difficult to solve and can no longer be solved by eigendecomposition. Recently, an efficient iterative spectral (eigendecomposition) method (ISM) has been proposed for this objective in the context of alternative clustering. However, ISM only provides theoretical guarantees for the Gaussian kernel. This greatly constrains ISM's usage since any kernel method using ISM is now limited to a single kernel. This work extends the theoretical guarantees of ISM to an entire family of kernels, thereby empowering ISM to solve any kernel method of the same objective. In identifying this family, we prove that each kernel within the family has a surrogate $\Phi$ matrix and the optimal projection is formed by its most dominant eigenvectors. With this extension, we establish how a wide range of IKDR applications across different learning paradigms can be solved by ISM. To support reproducible results, the source code is made publicly available on \url{this https URL}.
Automatic Critical Mechanic Discovery in Video Games

We present a new method of automatic critical mechanic discovery for video games using a combination of game description parsing and playtrace information. This method is applied to several games within the General Video Game Artificial Intelligence (GVG-AI) framework. In a user study, human-identified mechanics are compared against system-identified critical mechanics to verify alignment between humans and the system. The results of the study demonstrate that the new method is able to match humans with higher consistency than baseline. Our system is further validated by comparing MCTS agents augmented with critical mechanics and vanilla MCTS agents on 4 games from GVG-AI. Our new playtrace method shows a significant performance improvement over the baseline for all 4 tested games. The proposed method also shows either matched or improved performance over the old method, demonstrating that playtrace information is responsible for more complete critical mechanic discovery.
Abductive Reasoning as Self-Supervision for Common Sense Question Answering

Question answering has seen significant advances in recent times, especially with the introduction of increasingly bigger transformer-based models pre-trained on massive amounts of data. While achieving impressive results on many benchmarks, their performances appear to be proportional to the amount of training data available in the target domain. In this work, we explore the ability of current question-answering models to generalize - to both other domains as well as with restricted training data. We find that large amounts of training data are necessary, both for pre-training as well as fine-tuning to a task, for the models to perform well on the designated task. We introduce a novel abductive reasoning approach based on Grenander's Pattern Theory framework to provide self-supervised domain adaptation cues or "pseudo-labels," which can be used instead of expensive human annotations. The proposed self-supervised training regimen allows for effective domain adaptation without losing performance compared to fully supervised baselines. Extensive experiments on two publicly available benchmarks show the efficacy of the proposed approach. We show that neural networks models trained using self-labeled data can retain up to $75\%$ of the performance of models trained on large amounts of human-annotated training data.
Attending the Emotions to Detect Online Abusive Language

In recent years, abusive behavior has become a serious issue in online social networks. In this paper, we present a new corpus for the task of abusive language detection that is collected from a semi-anonymous online platform, and unlike the majority of other available resources, is not created based on a specific list of bad words. We also develop computational models to incorporate emotions into textual cues to improve aggression identification. We evaluate our proposed methods on a set of corpora related to the task and show promising results with respect to abusive language detection.
Efficient Sentence Embedding using Discrete Cosine Transform

Vector averaging remains one of the most popular sentence embedding methods in spite of its obvious disregard for syntactic structure. While more complex sequential or convolutional networks potentially yield superior classification performance, the improvements in classification accuracy are typically mediocre compared to the simple vector averaging. As an efficient alternative, we propose the use of discrete cosine transform (DCT) to compress word sequences in an order-preserving manner. The lower order DCT coefficients represent the overall feature patterns in sentences, which results in suitable embeddings for tasks that could benefit from syntactic features. Our results in semantic probing tasks demonstrate that DCT embeddings indeed preserve more syntactic information compared with vector averaging. With practically equivalent complexity, the model yields better overall performance in downstream classification tasks that correlate with syntactic features, which illustrates the capacity of DCT to preserve word order information.
Cosmological Evidence Modelling: a new simulation-based approach to constrain cosmology on non-linear scales

Extracting accurate cosmological information from galaxy–galaxy and galaxy–matter correlation functions on non-linear scales (${\lesssim } 10 \, h^{-1}{\rm {Mpc}}$) requires cosmological simulations. Additionally, one has to marginalize over several nuisance parameters of the galaxy–halo connection. However, the computational cost of such simulations prohibits naive implementations of stochastic posterior sampling methods like Markov chain Monte Carlo (MCMC) that would require of order $\mathcal {O}(10^6)$ samples in cosmological parameter space. Several groups have proposed surrogate models as a solution: a so-called emulator is trained to reproduce observables for a limited number of realizations in parameter space. Afterwards, this emulator is used as a surrogate model in an MCMC analysis. Here, we demonstrate a different method called Cosmological Evidence Modelling (CEM). First, for each simulation, we calculate the Bayesian evidence marginalized over the galaxy–halo connection by repeatedly populating the simulation with galaxies. We show that this Bayesian evidence is directly related to the posterior probability of cosmological parameters. Finally, we build a physically motivated model for how the evidence depends on cosmological parameters as sampled by the simulations. We demonstrate the feasibility of CEM by using simulations from the Aemulus simulation suite and forecasting cosmological constraints from BOSS CMASS measurements of redshift-space distortions. Our analysis includes exploration of how galaxy assembly bias affects cosmological inference. Overall, CEM has several potential advantages over the more common approach of emulating summary statistics, including the ability to easily marginalize over highly complex models of the galaxy–halo connection and greater accuracy, thereby reducing the number of simulations required.
High Resolution Medical Image Analysis with Spatial Partitioning

Medical images such as 3D computerized tomography (CT) scans and pathology images, have hundreds of millions or billions of voxels/pixels. It is infeasible to train CNN models directly on such high resolution images, because neural activations of a single image do not fit in the memory of a single GPU/TPU. Existing image analysis approaches alleviate this problem by cropping or down-sampling input images, which leads to complicated implementation and sub-optimal performance due to information loss. In this paper, we implement spatial partitioning, which internally distributes the input and output of convolutional layers across GPUs/TPUs. Our implementation is based on the Mesh-TensorFlow framework and the computation distribution is transparent to end users. With this technique, we train a 3D Unet on up to 512 by 512 by 512 resolution data. To the best of our knowledge, this is the first work for handling such high resolution images end-to-end.
Machine learning approaches in Detecting the Depression from Resting-state Electroencephalogram (EEG): A Review Study

In this paper we aimed at reviewing several different approaches present today in search for more accurate diagnostic and treatment management in mental healthcare. Our focus is on mood disorders, and in particular on major depressive disorder (MDD). We are reviewing and discussing findings based on neuroimaging studies (MRI and fMRI) first to get the impression of the body of knowledge about the anatomical and functional differences in depression. Then, we are focusing on less expensive data-driven approach, applicable for everyday clincal practice, in particular those based on electroencephalographic (EEG) recordings. Among those studies utilizing EEG, we are discussing a group of applications used for detecting of depression based on the resting state EEG (detection studies) and interventional studies (using stimulus in their protocols or aiming to predict the outcome of therapy). We conclude with a discussion and review of guidelines to improve the reliability of developed models that could serve improvement of diagnostic of depression in psychiatry.
Steenrod operations and A-module extensions

Explicit extensions representing cocycles $x \in Ext_{A}^{s,t}(F_2,F_2)$ are useful in calculating Steenrod operations $Sq^i : Ext_{A}^{s,t}(F_2,F_2) \longrightarrow Ext_{A}^{s+i,2t}(F_2,F_2)$ by a method devised by the second author. This can be used to identify explicit cocycles in the minimal resolutions produced by the first author's computer programs, and this information is useful in determining differentials in the Adams spectral sequence.
Trading-Off Static and Dynamic Regret in Online Least-Squares and Beyond

Recursive least-squares algorithms often use forgetting factors as a heuristic to adapt to non-stationary data streams. The first contribution of this paper rigorously characterizes the effect of forgetting factors for a class of online Newton algorithms. For exp-concave and strongly convex objectives, the algorithms achieve the dynamic regret of $\max\{O(\log T),O(\sqrt{TV})\}$, where $V$ is a bound on the path length of the comparison sequence. In particular, we show how classic recursive least-squares with a forgetting factor achieves this dynamic regret bound. By varying $V$, we obtain a trade-off between static and dynamic regret. In order to obtain more computationally efficient algorithms, our second contribution is a novel gradient descent step size rule for strongly convex functions. Our gradient descent rule recovers the order optimal dynamic regret bounds described above. For smooth problems, we can also obtain static regret of $O(T^{1-\beta})$ and dynamic regret of $O(T^\beta V^*)$, where $\beta \in (0,1)$ and $V^*$ is the path length of the sequence of minimizers. By varying $\beta$, we obtain a trade-off between static and dynamic regret.
Automating Cluster Management with Weave

Modern cluster management systems like Kubernetes and Openstack grapple with hard combinatorial optimization problems: load balancing, placement, scheduling, and configuration. Currently, developers tackle these problems by designing custom application-specific algorithms---an approach that is proving unsustainable, as ad-hoc solutions both perform poorly and introduce overwhelming complexity to the system, making it challenging to add important new features. 
We propose a radically different architecture, where programmers drive cluster management tasks declaratively, using SQL queries over cluster state stored in a relational database. These queries capture in a natural way both constraints on the cluster configuration as well as optimization objectives. When a cluster reconfiguration is required at runtime, our tool, called Weave, synthesizes an encoding of these queries into an optimization model, which it solves using an off-the-shelf solver. 
We demonstrate Weave's efficacy by powering three production-grade systems with it: a Kubernetes scheduler, a virtual machine management solution, and a distributed transactional datastore. Using Weave, we expressed complex cluster management policies in under 20 lines of SQL, easily added new features to these existing systems, and significantly improved placement quality and convergence times.
Geometry-Aware Video Object Detection for Static Cameras

In this paper we propose a geometry-aware model for video object detection. Specifically, we consider the setting that cameras can be well approximated as static, e.g. in video surveillance scenarios, and scene pseudo depth maps can therefore be inferred easily from the object scale on the image plane. We make the following contributions: First, we extend the recent anchor-free detector (CornerNet ) to video object detections. In order to exploit the spatial-temporal information while maintaining high efficiency, the proposed model accepts video clips as input, and only makes predictions for the starting and the ending frames, i.e. heatmaps of object bounding box corners and the corresponding embeddings for grouping. Second, to tackle the challenge from scale variations in object detection, scene geometry information, e.g. derived depth maps, is explicitly incorporated into deep networks for multi-scale feature selection and for the network prediction. Third, we validate the proposed architectures on an autonomous driving dataset generated from the Carla simulator , and on a real dataset for human detection (DukeMTMC dataset [28]). When comparing with the existing competitive single-stage or two-stage detectors, the proposed geometry-aware spatio-temporal network achieves significantly better results.