nils-herrmann/clean_scirepeval huggingface dataset:
How Developers Iterate on Machine Learning Workflows - A Survey of the Applied Machine Learning Literature

Machine learning workflow development is anecdotally regarded to be an iterative process of trial-and-error with humans-in-the-loop. However, we are not aware of quantitative evidence corroborating this popular belief. A quantitative characterization of iteration can serve as a benchmark for machine learning workflow development in practice, and can aid the development of human-in-the-loop machine learning systems. To this end, we conduct a small-scale survey of the applied machine learning literature from five distinct application domains. We collect and distill statistics on the role of iteration within machine learning workflow development, and report preliminary trends and insights from our investigation, as a starting point towards this benchmark. Based on our findings, we finally describe desiderata for effective and versatile human-in-the-loop machine learning systems that can cater to users in diverse domains.
A Formal TLS Handshake Model in LNT

Testing of network services represents one of the biggest challenges in cyber security. Because new vulnerabilities are detected on a regular basis, more research is needed. These faults have their roots in the software development cycle or because of intrinsic leaks in the system specification. Conformance testing checks whether a system behaves according to its specification. Here model-based testing provides several methods for automated detection of shortcomings. The formal specification of a system behavior represents the starting point of the testing process. In this paper, a widely used cryptographic protocol is specified and tested for conformance with a test execution framework. The first empirical results are presented and discussed.
Gender Bias in Sharenting: Both Men and Women Mention Sons More Often Than Daughters on Social Media

Gender inequality starts before birth. Parents tend to prefer boys over girls, which is manifested in reproductive behavior, marital life, and parents’ pastimes and investments in their children. While social media and sharing information about children (so-called “sharenting”) have become an integral part of parenthood, it is not well-known if and how gender preference shapes online behavior of users. In this paper, we investigate public mentions of daughters and sons on social media. We use data from a popular social networking site on public posts from 635,665 users. We find that both men and women mention sons more often than daughters in their posts. We also find that posts featuring sons get more “likes” on average. Our results indicate that girls are underrepresented in parents’ digital narratives about their children. This gender imbalance may send a message that girls are less important than boys, or that they deserve less attention, thus reinforcing gender inequality.
What deep learning can tell us about higher cognitive functions like mindreading?

Can deep learning (DL) guide our understanding of computations happening in biological brain? We will first briefly consider how DL has contributed to the research on visual object recognition. In the main part we will assess whether DL could also help us to clarify the computations underlying higher cognitive functions such as Theory of Mind. In addition, we will compare the objectives and learning signals of brains and machines, leading us to conclude that simply scaling up the current DL algorithms will most likely not lead to human level Theory of Mind.
Giant Catalytic Effect of Altruists in Schelling's Segregation Model.

We study the effect of introducing altruistic agents in a Schelling-like model of residential segregation. We find that even an infinitesimal proportion of altruists has dramatic catalytic effects on the collective utility of the system. Altruists provide pathways that move the system away from the suboptimal equilibrium it would reach if the system included only egoist agents, allowing it to reach the optimal steady state.
Bundled fragments of first-order modal logic: (un)decidability

Quantified modal logic provides a natural logical language for reasoning about modal attitudes even while retaining the richness of quantification for referring to predicates over domains. But then most fragments of the logic are undecidable, over many model classes. Over the years, only a few fragments (such as the monodic) have been shown to be decidable. In this paper, we study fragments that bundle quantifiers and modalities together, inspired by earlier work on epistemic logics of know-how/why/what. As always with quantified modal logics, it makes a significant difference whether the domain stays the same across worlds, or not. In particular, we show that the bundle $\forall \Box$ is undecidable over constant domain interpretations, even with only monadic predicates, whereas $\exists \Box$ bundle is decidable. On the other hand, over increasing domain interpretations, we get decidability with both $\forall \Box$ and $\exists \Box$ bundles with unrestricted predicates. In these cases, we also obtain tableau based procedures that run in \PSPACE. We further show that the $\exists \Box$ bundle cannot distinguish between constant domain and increasing domain interpretations.
Feed-forward Uncertainty Propagation in Belief and Neural Networks

We propose a feed-forward inference method applicable to belief and neural networks. In a belief network, the method estimates an approximate factorized posterior of all hidden units given the input. In neural networks the method propagates uncertainty of the input through all the layers. In neural networks with injected noise, the method analytically takes into account uncertainties resulting from this noise. Such feed-forward analytic propagation is differentiable in parameters and can be trained end-to-end. Compared to standard NN, which can be viewed as propagating only the means, we propagate the mean and variance. The method can be useful in all scenarios that require knowledge of the neuron statistics, e.g. when dealing with uncertain inputs, considering sigmoid activations as probabilities of Bernoulli units, training the models regularized by injected noise (dropout) or estimating activation statistics over the dataset (as needed for normalization methods). In the experiments we show the possible utility of the method in all these tasks as well as its current limitations.
Self-Representation on Twitter Using Emoji Skin Color Modifiers

Since 2015, it has been possible to modify certain emoji with a skin tone. The five different skin tones were introduced with the aim of representing more human diversity, but some commentators feared they might be used as a way to negatively represent other users/groups. This paper presents a quantitative analysis of the use of skin tone modifiers on emoji on Twitter, showing that users with darker-skinned profile photos employ them more often than users with lighter-skinned profile photos, and the vast majority of skin tone usage matches the color of a user's profile photo - i.e., tones represent the self, rather than the other. In the few cases where users do use opposite-toned emoji, we find no evidence of negative racial sentiment. Thus, the introduction of skin tones seems to have met the goal of better representing human diversity.
Deeply Supervised Semantic Model for Click-Through Rate Prediction in Sponsored Search

In sponsored search it is critical to match ads that are relevant to a query and to accurately predict their likelihood of being clicked. Commercial search engines typically use machine learning models for both query-ad relevance matching and click-through-rate (CTR) prediction. However, matching models are based on the similarity between a query and an ad, ignoring the fact that a retrieved ad may not attract clicks, while click models rely on click history, being of limited use for new queries and ads. We propose a deeply supervised architecture that jointly learns the semantic embeddings of a query and an ad as well as their corresponding CTR.We also propose a novel cohort negative sampling technique for learning implicit negative signals. We trained the proposed architecture using one billion query-ad pairs from a major commercial web search engine. This architecture improves the best-performing baseline deep neural architectures by 2\% of AUC for CTR prediction and by statistically significant 0.5\% of NDCG for query-ad matching.
Primordial Anisotropies in the Gravitational Wave Background from Cosmological Phase Transitions.

Phase transitions in the early Universe can readily create an observable stochastic gravitational wave background. We show that such a background necessarily contains anisotropies analogous to those of the cosmic microwave background (CMB) of photons, and that these too may be within reach of proposed gravitational wave detectors. Correlations within the gravitational wave anisotropies and their cross-correlations with the CMB can provide new insights into the mechanism underlying primordial fluctuations, such as multifield inflation, as well as reveal the existence of nonstandard "hidden sectors" of particle physics in earlier eras.
Fast and Robust Subspace Clustering Using Random Projections

Over the past several decades, subspace clustering has been receiving increasing interest and continuous progress. However, due to the lack of scalability and/or robustness, existing methods still have difficulty in dealing with the data that possesses simultaneously three characteristics: high-dimensional, massive and grossly corrupted. To tackle the scalability and robustness issues simultaneously, in this paper we suggest to consider a problem called compressive robust subspace clustering, which is to perform robust subspace clustering with the compressed data, and which is generated by projecting the original high-dimensional data onto a lower-dimensional subspace chosen at random. Given these random projections, the proposed method, row space pursuit (RSP), recovers not only the authentic row space, which provably leads to correct clustering results under certain conditions, but also the gross errors possibly existing in data. The compressive nature of the random projections gives our RSP high computational and storage efficiency, and the recovery property enables the ability for RSP to deal with the grossly corrupted data. Extensive experiments on high-dimensional and/or large-scale datasets show that RSP can maintain comparable accuracies to to prevalent methods with significant reductions in the computational time.
DeepScores-A Dataset for Segmentation, Detection and Classification of Tiny Objects

We present the DeepScores dataset with the goal of advancing the state-of-the-art in small object recognition by placing the question of object recognition in the context of scene understanding. DeepScores contains high quality images of musical scores, partitioned into 300, 000 sheets of written music that contain symbols of different shapes and sizes. With close to a hundred million small objects, this makes our dataset not only unique, but also the largest public dataset. DeepScores comes with ground truth for object classification, detection and semantic segmentation. DeepScores thus poses a relevant challenge for computer vision in general, and optical music recognition (OMR) research in particular. We present a detailed statistical analysis of the dataset, comparing it with other computer vision datasets like PASCAL VOC, SUN, SVHN, ImageNet, MS-COCO, as well as with other OMR datasets. Finally, we provide baseline performances for object classification, intuition for the inherent difficulty that DeepScores poses to state-of-the-art object detectors like YOLO or R-CNN, and give pointers to future research based on this dataset.
MegaDepth: Learning Single-View Depth Prediction from Internet Photos

Single-view depth prediction is a fundamental problem in computer vision. Recently, deep learning methods have led to significant progress, but such methods are limited by the available training data. Current datasets based on 3D sensors have key limitations, including indoor-only images (NYU), small numbers of training examples (Make3D), and sparse sampling (KITTI). We propose to use multi-view Internet photo collections, a virtually unlimited data source, to generate training data via modern structure-from-motion and multi-view stereo (MVS) methods, and present a large depth dataset called MegaDepth based on this idea. Data derived from MVS comes with its own challenges, including noise and unreconstructable objects. We address these challenges with new data cleaning methods, as well as automatically augmenting our data with ordinal depth relations generated using semantic segmentation. We validate the use of large amounts of Internet data by showing that models trained on MegaDepth exhibit strong generalization-not only to novel scenes, but also to other diverse datasets including Make3D, KITTI, and DIW, even when no images from those datasets are seen during
Deflection of light by rotating regular black holes using the Gauss-Bonnet theorem

In this paper, we study the weak gravitational lensing in the spacetime of rotating regular black hole geometries such as Ayon-Beato-Garc\'ia (ABG), Bardeen, and Hayward black holes. We calculate the deflection angle of light using the Gauss-Bonnet theorem (GBT) and show that the deflection of light can be viewed as a partially topological effect in which the deflection angle can be calculated by considering a domain outside of the light ray applied to the black hole optical geometries. Then, we demonstrate also the deflection angle via the geodesics formalism for these black holes to verify our results and explore the differences with the Kerr solution. These black holes have in addition to the total mass and rotation parameter, different parameters as electric charge, magnetic charge, and deviation parameter. Newsworthy, we find that the deflection of light has correction terms coming from these parameters which generalizes the Kerr deflection angle.
Clinical Concept Embeddings Learned from Massive Sources of Medical Data

Word embeddings have emerged as a popular approach to unsupervised learning of word relationships in machine learning and natural language processing. In this article, we benchmark two of the most popular algorithms, GloVe and word2vec, to assess their suitability for capturing medical relationships in large sources of biomedical data. Leaning on recent theoretical insights, we provide a unified view of these algorithms and demonstrate how different sources of data can be combined to construct the largest ever set of embeddings for 108,477 medical concepts using an insurance claims database of 60 million members, 20 million clinical notes, and 1.7 million full text biomedical journal articles. We evaluate our approach, called cui2vec, on a set of clinically relevant benchmarks and in many instances demonstrate state of the art performance relative to previous results. Finally, we provide a downloadable set of pre-trained embeddings for other researchers to use, as well as an online tool for interactive exploration of the cui2vec embeddings.
The Tsetlin Machine - A Game Theoretic Bandit Driven Approach to Optimal Pattern Recognition with Propositional Logic

Although simple individually, artificial neurons provide state-of-the-art performance when interconnected in deep networks. Arguably, the Tsetlin Automaton is an even simpler and more versatile learning mechanism, capable of solving the multi-armed bandit problem. Merely by means of a single integer as memory, it learns the optimal action in stochastic environments through increment and decrement operations. In this paper, we introduce the Tsetlin Machine, which solves complex pattern recognition problems with propositional formulas, composed by a collective of Tsetlin Automata. To eliminate the longstanding problem of vanishing signal-to-noise ratio, the Tsetlin Machine orchestrates the automata using a novel game. Further, both inputs, patterns, and outputs are expressed as bits, while recognition and learning rely on bit manipulation, simplifying computation. Our theoretical analysis establishes that the Nash equilibria of the game align with the propositional formulas that provide optimal pattern recognition accuracy. This translates to learning without local optima, only global ones. In five benchmarks, the Tsetlin Machine provides competitive accuracy compared with SVMs, Decision Trees, Random Forests, Naive Bayes Classifier, Logistic Regression, and Neural Networks. We further demonstrate how the propositional formulas facilitate interpretation. We believe the combination of high accuracy, interpretability, and computational simplicity makes the Tsetlin Machine a promising tool for a wide range of domains.
Forming Super Star Clusters in the Central Starburst of NGC 253

NGC 253 hosts the nearest nuclear starburst. Previous observations show a region rich in molecular gas, with dense clouds associated with recent star formation. We used the Atacama Large Submillimeter/Millimeter Array (ALMA) to image the 350 GHz dust continuum and molecular line emission from this region at 2 pc resolution. Our observations reveal ∼14 bright, compact (∼2–3 pc FWHM) knots of dust emission. Most of these sources are likely to be forming super star clusters (SSCs) based on their inferred dynamical and gas masses, association with 36 GHz radio continuum emission, and coincidence with line emission tracing dense, excited gas. One source coincides with a known SSC, but the rest remain invisible in Hubble near-infrared (IR) imaging. Our observations imply that gas still constitutes a large fraction of the overall mass in these sources. Their high brightness temperature at 350 GHz also implies a large optical depth near the peak of the IR spectral energy distribution. As a result, these sources may have large IR photospheres, and the IR radiation force likely exceeds L/c. Still, their moderate observed velocity dispersions suggest that feedback from radiation, winds, and supernovae are not yet disrupting most sources. This mode of star formation appears to produce a large fraction of stars in the burst. We argue for a scenario in which this phase lasts ∼1 Myr, after which the clusters shed their natal cocoons but continue to produce ionizing photons. The strong feedback that drives the observed cold gas and X-ray outflows likely occurs after the clusters emerge from this early phase.
Phase transitions of the Moran process and algorithmic consequences

The Moran process is a random process that models the spread of genetic mutations through graphs. On connected graphs, the process eventually reaches “fixation,” where all vertices are mutants, or “extinction,” where none are. Our main result is an almost‐tight upper bound on expected absorption time. For all ϵ>0, we show that the expected absorption time on an n‐vertex graph is o(n3+ϵ). Specifically, it is at most n3eO((loglogn)3) , and there is a family of graphs where it is Ω(n3). In proving this, we establish a phase transition in the probability of fixation, depending on the mutants' fitness r. We show that no similar phase transition occurs for digraphs, where it is already known that the expected absorption time can be exponential. Finally, we give an improved fully polynomial randomized approximation scheme (FPRAS) for approximating the probability of fixation. On degree‐bounded graphs where some basic properties are given, its running time is independent of the number of vertices.
Deep Painterly Harmonization

Copying an element from a photo and pasting it into a painting is a challenging task. Applying photo compositing techniques in this context yields subpar results that look like a collage — and existing painterly stylization algorithms, which are global, perform poorly when applied locally. We address these issues with a dedicated algorithm that carefully determines the local statistics to be transferred. We ensure both spatial and inter‐scale statistical consistency and demonstrate that both aspects are key to generating quality results. To cope with the diversity of abstraction levels and types of paintings, we introduce a technique to adjust the parameters of the transfer depending on the painting. We show that our algorithm produces significantly better results than photo compositing or global stylization techniques and that it enables creative painterly edits that would be otherwise difficult to achieve.
Question Answering over Freebase via Attentive RNN with Similarity Matrix based CNN

With the rapid growth of knowledge bases (KBs), question answering over knowledge base, a.k.a. KBQA has drawn huge attention in recent years. Most of the existing KBQA methods follow so called encoder-compare framework. They map the question and the KB facts to a common embedding space, in which the similarity between the question vector and the fact vectors can be conveniently computed. This, however, inevitably loses original words interaction information. To preserve more original information, we propose an attentive recurrent neural network with similarity matrix based convolutional neural network (AR-SMCNN) model, which is able to capture comprehensive hierarchical information utilizing the advantages of both RNN and CNN. We use RNN to capture semantic-level correlation by its sequential modeling nature, and use an attention mechanism to keep track of the entities and relations simultaneously. Meanwhile, we use a similarity matrix based CNN with two-directions pooling to extract literal-level words interaction matching utilizing CNNs strength of modeling spatial correlation among data. Moreover, we have developed a new heuristic extension method for entity detection, which significantly decreases the effect of noise. Our method has outperformed the state-of-the-arts on SimpleQuestion benchmark in both accuracy and efficiency.
Analyzing Right-wing YouTube Channels: Hate, Violence and Discrimination

As of 2018, YouTube, the major online video sharing website, hosts multiple channels promoting right-wing content. In this paper, we observe issues related to hate, violence and discriminatory bias in a dataset containing more than 7,000 videos and 17 million comments. We investigate similarities and differences between users' comments and video content in a selection of right-wing channels and compare it to a baseline set using a three-layered approach, in which we analyze (a) lexicon, (b) topics and (c) implicit biases present in the texts. Among other results, our analyses show that right-wing channels tend to (a) contain a higher degree of words from "negative'' semantic fields, (b) raise more topics related to war and terrorism, and (c) demonstrate more discriminatory bias against Muslims (in videos) and towards LGBT people (in comments). Our findings shed light not only into the collective conduct of the YouTube community promoting and consuming right-wing content, but also into the general behavior of YouTube users.
Topological Phase Transition and Z_{2} Index for S=1 Quantum Spin Chains.

We study S=1 quantum spin systems on the infinite chain with short ranged Hamiltonians that have a certain rotational and discrete symmetry. We define a Z_{2} index for any gapped unique ground state, and prove that it is invariant under a smooth deformation. By using the index, we provide the first rigorous proof of the existence of a "topological" phase transition, which cannot be characterized by any conventional order parameters, between the Affleck-Kennedy-Lieb-Tasaki (AKLT) model and trivial models. This rigorously establishes that the AKLT model is in a nontrivial symmetry protected topological phase.