nils-herrmann/clean_scirepeval huggingface dataset:
Additive function approximation in the brain

Many biological learning systems such as the mushroom body, hippocampus, and cerebellum are built from sparsely connected networks of neurons. For a new understanding of such networks, we study the function spaces induced by sparse random features and characterize what functions may and may not be learned. A network with $d$ inputs per neuron is found to be equivalent to an additive model of order $d$, whereas with a degree distribution the network combines additive terms of different orders. We identify three specific advantages of sparsity: additive function approximation is a powerful inductive bias that limits the curse of dimensionality, sparse networks are stable to outlier noise in the inputs, and sparse random features are scalable. Thus, even simple brain architectures can be powerful function approximators. Finally, we hope that this work helps popularize kernel theories of networks among computational neuroscientists.
Fate of scalar dark matter solitons around supermassive galactic black holes

In scalar-field dark matter scenarios, a scalar-field soliton could form at the center of galactic halos, around the supermassive black holes that sit at the center of galaxies. Focusing on the large scalar-mass limit, where the soliton is formed by the balance between self-gravity and a repulsive self-interaction, we study the infall of the scalar field onto the central Schwarzschild black hole. We derive the scalar-field profile, from the Schwarzschild radius to the large radii dominated by the scalar cloud. We show that the steady state solution selects the maximum allowed flux, with a critical profile that is similar to the transonic solution obtained for the hydrodynamic case. This finite flux, which scales as the inverse of the self-interaction coupling, is small enough to allow the dark matter soliton to survive for many Hubble times.
MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance

A robust evaluation metric has a profound impact on the development of text generation systems. A desirable metric compares system output against references based on their semantics rather than surface forms. In this paper we investigate strategies to encode system and reference texts to devise a metric that shows a high correlation with human judgment of text quality. We validate our new metric, namely MoverScore, on a number of text generation tasks including summarization, machine translation, image captioning, and data-to-text generation, where the outputs are produced by a variety of neural and non-neural systems. Our findings suggest that metrics combining contextualized representations with a distance measure perform the best. Such metrics also demonstrate strong generalization capability across tasks. For ease-of-use we make our metrics available as web service.
On the Acceleration of Deep Learning Model Parallelism With Staleness

Training the deep convolutional neural network for computer vision problems is slow and inefficient, especially when it is large and distributed across multiple devices. The inefficiency is caused by the backpropagation algorithm's forward locking, backward locking, and update locking problems. Existing solutions for acceleration either can only handle one locking problem or lead to severe accuracy loss or memory inefficiency. Moreover, none of them consider the straggler problem among devices. In this paper, we propose Layer-wise Staleness and a novel efficient training algorithm, Diversely Stale Parameters (DSP), to address these challenges. We also analyze the convergence of DSP with two popular gradient-based methods and prove that both of them are guaranteed to converge to critical points for non-convex problems. Finally, extensive experimental results on training deep learning models demonstrate that our proposed DSP algorithm can achieve significant training speedup with stronger robustness than compared methods.
Random Sampling for Group-By Queries

Random sampling has been widely used in approximate query processing on large databases, due to its potential to significantly reduce resource usage and response times, at the cost of a small approximation error. We consider random sampling for answering the ubiquitous class of group-by queries, which first group data according to one or more attributes, and then aggregate within each group after filtering through a predicate. The challenge with group-by queries is that a sampling method cannot focus on optimizing the quality of a single answer (e.g. the mean of selected data), but must simultaneously optimize the quality of a set of answers (one per group).We present CVOPT, a query- and data-driven sampling framework for a set of group-by queries. To evaluate the quality of a sample, CVOPT defines a metric based on the norm (e.g. ℓ2 or ℓ∞) of the coefficients of variation (CVs) of different answers, and constructs a stratified sample that provably optimizes the metric. CVOPT can handle group-by queries on data where groups have vastly different statistical characteristics, such as frequencies, means, or variances. CVOPT jointly optimizes for multiple aggregations and multiple group-by clauses, and provides a way to prioritize specific groups or aggregates. It can be tuned to cases when partial information about a query workload is known, such as a data warehouse where queries are run periodically.Our experimental results show that CVOPT outperforms the current state-of-the-art on sample quality and estimation accuracy for group-by queries. On a set of queries on two real-world data sets, CVOPT yields relative errors that are 5× smaller than competing approaches, under the same space budget.
Effective Use of Transformer Networks for Entity Tracking

Tracking entities in procedural language requires understanding the transformations arising from actions on entities as well as those entities’ interactions. While self-attention-based pre-trained language encoders like GPT and BERT have been successfully applied across a range of natural language understanding tasks, their ability to handle the nuances of procedural texts is still unknown. In this paper, we explore the use of pre-trained transformer networks for entity tracking tasks in procedural text. First, we test standard lightweight approaches for prediction with pre-trained transformers, and find that these approaches underperforms even simple baselines. We show that much stronger results can be attained by restructuring the input to guide the model to focus on a particular entity. Second, we assess the degree to which the transformer networks capture the process dynamics, investigating such factors as merged entities and oblique entity references. On two different tasks, ingredient detection in recipes and QA over scientific processes, we achieve state-of-the-art results, but our models still largely attend to shallow context clues and do not form complex representations of intermediate process state.
Contextual Minimum-Norm Estimates (CMNE): A Deep Learning Method for Source Estimation in Neuronal Networks

Magnetoencephalography (MEG) and Electroencephalography (EEG) source estimates have thus far mostly been derived sample by sample, i.e., independent of each other in time. However, neuronal assemblies are heavily interconnected, constraining the temporal evolution of neural activity in space as detected by MEG and EEG. The observed neural currents are thus highly context dependent. Here, a new method is presented which integrates predictive deep learning networks with the Minimum-Norm Estimates (MNE) approach. Specifically, we employ Long Short-Term Memory (LSTM) networks, a type of recurrent neural network, for predicting brain activity. Because we use past activity (context) in the estimation, we call our method Contextual MNE (CMNE). We demonstrate that these contextual algorithms can be used for predicting activity based on previous brain states and when used in conjunction with MNE, they lead to more accurate source estimation. To evaluate the performance of CMNE, it was tested on simulated and experimental data from human auditory evoked response experiments.
(Un)informed Consent: Studying GDPR Consent Notices in the Field

Since the adoption of the General Data Protection Regulation (GDPR) in May 2018 more than 60 % of popular websites in Europe display cookie consent notices to their visitors. This has quickly led to users becoming fatigued with privacy notifications and contributed to the rise of both browser extensions that block these banners and demands for a solution that bundles consent across multiple websites or in the browser. In this work, we identify common properties of the graphical user interface of consent notices and conduct three experiments with more than 80,000 unique users on a German website to investigate the influence of notice position, type of choice, and content framing on consent. We find that users are more likely to interact with a notice shown in the lower (left) part of the screen. Given a binary choice, more users are willing to accept tracking compared to mechanisms that require them to allow cookie use for each category or company individually. We also show that the wide-spread practice of nudging has a large effect on the choices users make. Our experiments show that seemingly small implementation decisions can substantially impact whether and how people interact with consent notices. Our findings demonstrate the importance for regulation to not just require consent, but also provide clear requirements or guidance for how this consent has to be obtained in order to ensure that users can make free and informed choices.
Intensity augmentation for domain transfer of whole breast segmentation in MRI

The segmentation of the breast from the chest wall is an important first step in the analysis of breast magnetic resonance images. 3D U-nets have been shown to obtain high segmentation accuracy and appear to generalize well when trained on one scanner type and tested on another scanner, provided that a very similar T1-weighted MR protocol is used. There has, however, been little work addressing the problem of domain adaptation when image intensities or patient orientation differ markedly between the training set and an unseen test set. To overcome the domain shift we propose to apply extensive intensity augmentation in addition to geometric augmentation during training. We explored both style transfer and a novel intensity remapping approach as intensity augmentation strategies. For our experiments, we trained a 3D U-net on T1-weighted scans and tested on T2-weighted scans. By applying intensity augmentation we increased segmentation performance from a DSC of 0.71 to 0.90. This performance is very close to the baseline performance of training and testing on T2-weighted scans (0.92). Furthermore, we applied our network to an independent test set made up of publicly available scans acquired using a T1-weighted TWIST sequence and a different coil configuration. On this dataset we obtained a performance of 0.89, close to the inter-observer variability of the ground truth segmentations (0.92). Our results show that using intensity augmentation in addition to geometric augmentation is a suitable method to overcome the intensity domain shift and we expect it to be useful for a wide range of segmentation tasks.
Nanoscale structural alterations in cancer cells to assess anti-cancerous drug effectiveness in ovarian cancer treatment using TEM imaging.

Understanding the nanoscale structural changes can provide the physical state of cells/tissues. It has been now shown that increases in nanoscale structural alterations are associated with the progress of carcinogenesis in most of the cancer cases, including early carcinogenesis. Anti-cancerous therapies are intended for the growth inhibition of cancer cells; however, it is challenging to detect the efficacy of such drugs in early stages of treatment. A unique method to assess the impact of anti-cancerous drugs on cancerous cells/tissues is to probe the nanoscale structural alterations. In this paper, we study the effect of different anti-cancerous drugs on ovarian tumorigenic cells, using their nanoscale structural alterations as a biomarker. Transmission electron microscopy (TEM) imaging on thin cell sections is performed to obtain their nanoscale structures. The degree of nanoscale structural alterations of tumorigenic cells and anti-cancerous drug treated tumorigenic cells are quantified by using the recently developed inverse participation ratio (IPR) technique. Results show an increase in the degree of nanoscale fluctuations in tumorigenic cells relative to non-tumorigenic cells; then a nearly reverse of the degree of fluctuation of tumorigenic cells to that of non-tumorigenic cells, after the anti-cancerous drugs treatment. These results support that the effect of anti-cancerous drugs in cancer treatment can be quantified by using the degree of nanoscale fluctuations of the cells via TEM imaging. Potential applications of the technique for cancer treatment are also discussed.
In Plain Sight: Media Bias Through the Lens of Factual Reporting

The increasing prevalence of political bias in news media calls for greater public awareness of it, as well as robust methods for its detection. While prior work in NLP has primarily focused on the lexical bias captured by linguistic attributes such as word choice and syntax, other types of bias stem from the actual content selected for inclusion in the text. In this work, we investigate the effects of informational bias: factual content that can nevertheless be deployed to sway reader opinion. We first produce a new dataset, BASIL, of 300 news articles annotated with 1,727 bias spans and find evidence that informational bias appears in news articles more frequently than lexical bias. We further study our annotations to observe how informational bias surfaces in news articles by different media outlets. Lastly, a baseline model for informational bias prediction is presented by fine-tuning BERT on our labeled data, indicating the challenges of the task and future directions.
Finite Euclidean and Non-Euclidean Geometries

The purpose of this book is to give an exposition of geometry, from a point of view which complements Klein's Erlangen program. The emphasis is on extending the classical Euclidean geometry to the finite case, but it goes beyond that. After a brief introduction, which gives the main theme, I present the main results, according to a synthetic view of the subject, rather that chronologically. First, I give some variation on the axiomatic treatment of projective geometry, followed by new results on quaternionian geometry, followed by results in geometry over the reals which are generalized over arbitrary fields, then those which depend on properties of finite fields. I then present results in finite mechanics. The role of the computer, which was essential for these inquiries, is briefly surveyed. The methodology to obtain illustrations by drawings is described. I end with a table which enumerates enclosed additional material.
AutoGMM: Automatic and Hierarchical Gaussian Mixture Modeling in Python

Gaussian mixture modeling is a fundamental tool in clustering, as well as discriminant analysis and semiparametric density estimation. However, estimating the optimal model for any given number of components is an NP-hard problem, and estimating the number of components is in some respects an even harder problem.  In R, a popular package called mclust addresses both of these problems. However, Python has lacked such a package. We therefore introduce AutoGMM, a Python algorithm for automatic Gaussian mixture modeling, and its hierarchical version, HGMM. AutoGMM builds upon scikit-learn’s AgglomerativeClustering and GaussianMixture classes, with certain modifications to make the results more stable. Empirically, on several different applications, AutoGMM performs approximately as well as mclust, and sometimes better.  AutoMM, a freely available Python package, enables efficient Gaussian mixture modeling by automatically selecting the initialization, number of clusters and covariance constraints.
Efficient Multivariate Bandit Algorithm with Path Planning

We solve the arms exponential exploding issues in the Multivariate-Mabwhen the arm dimension hierarchy is considered. We propose a framework called path planning, which utilizes paths in a graph to model reward success rate with m-way dimension interaction and adopts Thompson Sampling (TS) for a heuristic search. It is straightforward to combat the curse of dimensionality using a serial process that operates sequentially by focusing on one dimension per each process. Our proposed method utilizing tree models has advantages comparing with traditional models such as general linear regression. Real data and simulation studies validate our claim by achieving faster convergence speed, better efficient optimal arm allocation, and lower cumulative regret.
Restricted Minimum Error Entropy Criterion for Robust Classification.

The minimum error entropy (MEE) criterion is a powerful approach for non-Gaussian signal processing and robust machine learning. However, the instantiation of MEE on robust classification is a rather vacancy in the literature. The original MEE purely focuses on minimizing Renyi's quadratic entropy of the prediction errors, which could exhibit inferior capability in noisy classification tasks. To this end, we analyze the optimal error distribution with adverse outliers and introduce a specific codebook for restriction, which optimizes the error distribution toward the optimal case. Half-quadratic-based optimization and convergence analysis of the proposed learning criterion, called restricted MEE (RMEE), are provided. The experimental results considering logistic regression and extreme learning machine on synthetic data and UCI datasets, respectively, are presented to demonstrate the superior robustness of RMEE. Furthermore, we evaluate RMEE on a noisy electroencephalogram dataset, so as to strengthen its practical impact.
Automatic Weight Estimation of Harvested Fish from Images

Approximately 2,500 weights and corresponding images of harvested Lates calcarifer (Asian seabass or barramundi) were collected at three different locations in Queensland, Australia. Two instances of the LinkNet-34 segmentation Convolutional Neural Network (CNN) were trained. The first one was trained on 200 manually segmented fish masks with excluded fins and tails. The second was trained on 100 whole-fish masks. The two CNNs were applied to the rest of the images and yielded automatically segmented masks. The one-factor and two-factor simple mathematical weight-from-area models were fitted on 1072 area-weight pairs from the first two locations, where area values were extracted from the automatically segmented masks. When applied to 1,400 test images (from the third location), the one-factor whole-fish mask model achieved the best mean absolute percentage error (MAPE), MAPE = 4.36%. Direct weight-from-image regression CNNs were also trained, where the no-fins based CNN performed best on the test images with MAPE = 4.28%.
Decentralized Stochastic Gradient Tracking for Empirical Risk Minimization

Recent works have shown superiorities of decentralized SGD to centralized counterparts in large-scale machine learning, but their theoretical gap is still not fully understood. In this paper, we propose a decentralized stochastic gradient tracking (DSGT) algorithm over peer-to-peer networks for empirical risk minimization problems, and explicitly evaluate its convergence rate in terms of key parameters of the problem, e.g., algebraic connectivity of the communication network, mini-batch size, and gradient variance. Importantly, it is the first theoretical result that can exactly recover the rate of the centralized SGD, and has optimal dependence on the algebraic connectivity of the networks when using stochastic gradients. Moreover, we explicitly quantify how the network affects speedup and the rate improvement over existing works. Interestingly, we also point out for the first time that both linear and sublinear speedup can be possible. We empirically validate DSGT on neural networks and logistic regression problems, and show its advantage over the state-of-the-art algorithms.
A view on elliptic integrals from primitive forms (Period integrals of type $\mathrm{A_2, B_2}$ and $\mathrm{G_2}$

Elliptic integrals, since Euler's finding of addition theorem 1751, has been studied extensively from various view points. Present paper gives a short summery of a view point from primitive integrals of types $\mathrm{A_2}, \mathrm{B_2}$ and $\mathrm{G_2}$. We solve Jacobi inversion problem for the period maps in the sense explained in the introduction (see [Siegel] Chap.1,13) by introducing certain generalized Eisenstein series of types $\mathrm{A_2}, \mathrm{B_2}$ and $\mathrm{\mathrm{G_2}}$, which generate the ring of invariant functions on the period domain for the congruence subgroups $\Gamma_1(N)$ ($N=1,2$ and $3$). In particular, Eisenstein series of type $\mathrm{B_2}$ includes the case of weight two, and Eisenstein series of type $\mathrm{G_2}$ includes the cases of weight one and two, which seem to be of new feature. The goal of the paper is a partial answer to the discriminant conjecture, which claims an existence of certain cusp form of weight 1 with character of topological origin, giving a power root of the discriminant form ( E36,p.\ 265-320.\ 2004) Concluding Remarks for more about back grounds of the present paper.
Probing Majorana neutrinos with double-β decay

Looking for an exotic decay Neutrinos—elementary fermionic particles with no electrical charge—defy the standard model of particle physics by having a tiny, but nonzero mass. One explanation for their properties is that they are Majorana fermions, which are particles equal to their antiparticles. If neutrinos were Majorana fermions, a process called neutrinoless double-β decay would become possible: an unstable nucleus could decay by turning two of its neutrons into protons with the emission of two electrons but no antineutrinos. The GERDA Collaboration searched for this decay in a particular isotope of germanium. Housed deep underground to reduce the background signal, the experiment did not detect the elusive process but did place improved boundaries on its half-life. Science, this issue p. 1445 A germanium detector–based experiment places improved boundaries on the half-life of an exotic nuclear decay. A discovery that neutrinos are Majorana fermions would have profound implications for particle physics and cosmology. The Majorana character of neutrinos would make possible the neutrinoless double-β (0νββ) decay, a matter-creating process without the balancing emission of antimatter. The GERDA Collaboration searches for the 0νββ decay of 76Ge by operating bare germanium detectors in an active liquid argon shield. With a total exposure of 82.4 kg⋅year, we observe no signal and derive a lower half-life limit of T1/2 > 0.9 × 1026 years (90% C.L.). Our T1/2 sensitivity, assuming no signal, is 1.1 × 1026 years. Combining the latter with those from other 0νββ decay searches yields a sensitivity to the effective Majorana neutrino mass of 0.07 to 0.16 electron volts.