nils-herrmann/clean_scirepeval huggingface dataset:
The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations

We introduce a new test of how well language models capture meaning in children’s books. Unlike standard language modelling benchmarks, it distinguishes the task of predicting syntactic function words from that of predicting lowerfrequency words, which carry greater semantic content. We compare a range of state-of-the-art models, each with a different way of encoding what has been previously read. We show that models which store explicit representations of long-term contexts outperform state-of-the-art neural language models at predicting semantic content words, although this advantage is not observed for syntactic function words. Interestingly, we find that the amount of text encoded in a single memory representation is highly influential to the performance: there is a sweet-spot, not too big and not too small, between single words and full sentences that allows the most meaningful information in a text to be effectively retained and recalled. Further, the attention over such window-based memories can be trained effectively through self-supervision. We then assess the generality of this principle by applying it to the CNN QA benchmark, which involves identifying named entities in paraphrased summaries of news articles, and achieve state-of-the-art performance.
Learning Convolutional Neural Networks for Graphs

Numerous important problems can be framed as learning from graph data. We propose a framework for learning convolutional neural networks for arbitrary graphs. These graphs may be undirected, directed, and with both discrete and continuous node and edge attributes. Analogous to image-based convolutional networks that operate on locally connected regions of the input, we present a general approach to extracting locally connected regions from graphs. Using established benchmark data sets, we demonstrate that the learned feature representations are competitive with state of the art graph kernels and that their computation is highly efficient.
Prediction of long-term clinical outcomes using simple functional exercise performance tests in patients with COPD: a 5-year prospective cohort study

The 1-min sit-to-stand (1-min STS) test and handgrip strength test have been proposed as simple tests of functional exercise performance in chronic obstructive pulmonary disease (COPD) patients. We assessed the long-term (5-year) predictive performance of the 1-min sit-to-stand and handgrip strength tests for mortality, health-related quality of life (HRQoL) and exacerbations in COPD patients. In 409 primary care patients, we found the 1-min STS test to be strongly associated with long-term morality (hazard ratio per 3 more repetitions: 0.81, 95% CI 0.65 to 0.86) and moderately associated with long-term HRQoL. Neither test was associated with exacerbations. Our results suggest that the 1-min STS test may be useful for assessing the health status and long-term prognosis of COPD patients.
Emergence of scaling in random networks

Systems as diverse as genetic networks or the World Wide Web are best described as networks with complex topology. A common property of many large networks is that the vertex connectivities follow a scale-free power-law distribution. This feature was found to be a consequence of two generic mechanisms: (i) networks expand continuously by the addition of new vertices, and (ii) new vertices attach preferentially to sites that are already well connected. A model based on these two ingredients reproduces the observed stationary scale-free distributions, which indicates that the development of large networks is governed by robust self-organizing phenomena that go beyond the particulars of the individual systems.
Distributed Prioritized Experience Replay

We propose a distributed architecture for deep reinforcement learning at scale, that enables agents to learn effectively from orders of magnitude more data than previously possible. The algorithm decouples acting from learning: the actors interact with their own instances of the environment by selecting actions according to a shared neural network, and accumulate the resulting experience in a shared experience replay memory; the learner replays samples of experience and updates the neural network. The architecture relies on prioritized experience replay to focus only on the most significant data generated by the actors. Our architecture substantially improves the state of the art on the Arcade Learning Environment, achieving better final performance in a fraction of the wall-clock training time.
Attention on Attention: Architectures for Visual Question Answering (VQA)

Visual Question Answering (VQA) is an increasingly popular topic in deep learning research, requiring coordination of natural language processing and computer vision modules into a single architecture. We build upon the model which placed first in the VQA Challenge by developing thirteen new attention mechanisms and introducing a simplified classifier. We performed 300 GPU hours of extensive hyperparameter and architecture searches and were able to achieve an evaluation score of 64.78%, outperforming the existing state-of-the-art single model’s validation score of 63.15%. The code is available at github.com/SinghJasdeep/Attention-on-Attentionfor-VQA.
Sequential minimal optimization : A fast Algorithm for Training Support Vector machines

This paper proposes a new algorithm for training support vector machines: Sequential Minimal Optimization, or SMO. Training a support vector machine requires the solution of a very large quadratic programming (QP) optimization problem. SMO breaks this large QP problem into a series of smallest possible QP problems. These small QP problems are solved analytically, which avoids using a time-consuming numerical QP optimization as an inner loop. The amount of memory required for SMO is linear in the training set size, which allows SMO to handle very large training sets. Because matrix computation is avoided, SMO scales somewhere between linear and quadratic in the training set size for various test problems, while the standard chunking SVM algorithm scales somewhere between linear and cubic in the training set size. SMO’s computation time is dominated by SVM evaluation, hence SMO is fastest for linear SVMs and sparse data sets. On realworld sparse data sets, SMO can be more than 1000 times faster than the chunking algorithm.
An introduction to support vector machines for data mining

With increasing amounts of data being generated by businesses and researchers there is a need for fast, accurate and robust algorithms for data analysis. Improvements in databases technology, computing performance and artificial intelligence have contributed to the development of intelligent data analysis. The primary aim of data mining is to discover patterns in the data that lead to better understanding of the data generating process and to useful predictions. Examples of applications of data mining include detecting fraudulent credit card transactions, character recognition in automated zip code reading, and predicting compound activity in drug discovery. Real-world data sets are often characterized by having large numbers of examples, e.g. billions of credit card transactions and potential ‘drug-like’ compounds; being highly unbalanced, e.g. most transactions are not fraudulent, most compounds are not active against a given biological target; and, being corrupted by noise. The relationship between predictive variables, e.g. physical descriptors, and the target concept, e.g. compound activity, is often highly non-linear. One recent technique that has been developed to address these issues is the support vector machine. The support vector machine has been developed as robust tool for classification and regression in noisy, complex domains. The two key features of support vector machines are generalization theory, which leads to a principled way to choose an hypothesis; and, kernel functions, which introduce non-linearity in the hypothesis space without explicitly requiring a non-linear algorithm. In this tutorial I introduce support vector machines and highlight the advantages thereof over existing data analysis techniques, also are noted some important points for the data mining practitioner who wishes to use support vector machines.
Mind Over Matter , The Development of The Mental Toughness Scale ( MTS )

Mental Toughness (MT), which refers to an inner focus and commitment to rise above challenges when facing adversity, is viewed as one of the most important psychological attributes in determining success in sport. However, there is little consensus on key components of MT, and existing measures vary greatly while focusing on elite athletes. The purpose of this research was to develop a measure of MT for use with college athletes. Collegiate and noncollegiate athletes (N = 271) completed the original 54-item Mental Toughness Scale (MTS) in study 1. Factor analysis (PCA) results reduced the scale to an 11-item scale, with good reliability and validity as demonstrated by its positive correlations with self-esteem and flow. A second study of college basketball players (N = 143) was conducted to establish the psychometric properties of the MTS. Study 2 demonstrated convergent, divergent and criterion validity through correlations with related measures, and a CFA provided moderate support for the MTS as a one-dimensional measure of mental toughness in sport.
Segmental Recurrent Neural Networks for End-to-end Speech Recognition

We study the segmental recurrent neural network for end-to-end acoustic modelling. This model connects the segmental conditional random field (CRF) with a recurrent neural network (RNN) used for feature extraction. Compared to most previous CRF-based acoustic models, it does not rely on an external system to provide features or segmentation boundaries. Instead, this model marginalises out all the possible segmentations, and features are extracted from the RNN trained together with the segmental CRF. In essence, this model is self-contained and can be trained end-to-end. In this paper, we discuss practical training and decoding issues as well as the method to speed up the training in the context of speech recognition. We performed experiments on the TIMIT dataset. We achieved 17.3% phone error rate (PER) from the first-pass decoding — the best reported result using CRFs, despite the fact that we only used a zeroth-order CRF and without using any language model.
The periosteum. Part 1: Anatomy, histology and molecular biology.

The periosteum is a thin layer of connective tissue that covers the outer surface of a bone in all places except at joints (which are protected by articular cartilage). As opposed to bone itself, it has nociceptive nerve endings, making it very sensitive to manipulation. It also provides nourishment in the form of blood supply to the bone. The periosteum is connected to the bone by strong collagenous fibres called Sharpey's fibres, which extend to the outer circumferential and interstitial lamellae of bone. The periosteum consists of an outer "fibrous layer" and inner "cambium layer". The fibrous layer contains fibroblasts while the cambium layer contains progenitor cells which develop into osteoblasts that are responsible for increasing bone width. After a bone fracture the progenitor cells develop into osteoblasts and chondroblasts which are essential to the healing process. This review discusses the anatomy, histology and molecular biology of the periosteum in detail.
Convolutional Patch Representations for Image Retrieval: An Unsupervised Approach

Convolutional neural networks (CNNs) are able to model local stationary structures in natural images in a multi-scale fashion, when learning all model parameters with supervision. While excellent performance was achieved for image classification when large amounts of labeled visual data are available, their success for unsupervised tasks such as image retrieval has been moderate so far.Our paper focuses on this latter setting and explores several methods for learning patch descriptors without supervision with application to matching and instance-level retrieval. To that effect, we propose a new family of patch representations, based on the recently introduced convolutional kernel networks. We show that our descriptor, named Patch-CKN, performs better than SIFT as well as other convolutional networks learned by artificially introducing supervision and is significantly faster to train. To demonstrate its effectiveness, we perform an extensive evaluation on standard benchmarks for patch and image retrieval where we obtain state-of-the-art results. We also introduce a new dataset called RomePatches, which allows to simultaneously study descriptor performance for patch and image retrieval.
An Adaptive Network-Based Fuzzy Inference System (ANFIS) for the prediction of stock market return: The case of the Istanbul Stock Exchange

Stock market prediction is important and of great interest because successful prediction of stock prices may promise attractive benefits. These tasks are highly complicated and very difficult. In this paper, we investigate the predictability of stock market return with Adaptive Network-Based Fuzzy Inference System (ANFIS). The objective of this study is to determine whether an ANFIS algorithm is capable of accurately predicting stock market return. We attempt to model and predict the return on stock price index of the Istanbul Stock Exchange (ISE) with ANFIS. We use six macroeconomic variables and three indices as input variables. The experimental results reveal that the model successfully forecasts the monthly return of ISE National 100 Index with an accuracy rate of 98.3%. ANFIS provides a promising alternative for stock market prediction. ANFIS can be a useful tool for economists and practitioners dealing with the forecasting of the stock price index return.
Web Phishing detection based on graph mining

Web Phishing (Phishing) uses social engineering technique through short messages, emails and IMs to induce users to visit faked website to get sensitive information. With detecting method for phishing continually proposed and applied, the threat of web phishing has already reduced at a great extent. However, since each type of detection has limitation, phishing attackers can modify their strategies at a relatively low cost to avoid detection accordingly. Facing the defects of current detection, we mainly focus on the behavior pattern of phishing websites. We analyze real IP flows from ISP and propose a detecting method based on Graph Mining with Belief Propagation. The experiment suggested that our algorithm has decent accuracy and runtime efficiency. As we have considered distributed computation while designing the algorithm, it will be easy to replicate our model in popular distributed processing frameworks.
DeepSign: Deep learning for automatic malware signature generation and classification

This paper presents a novel deep learning based method for automatic malware signature generation and classification. The method uses a deep belief network (DBN), implemented with a deep stack of denoising autoencoders, generating an invariant compact representation of the malware behavior. While conventional signature and token based methods for malware detection do not detect a majority of new variants for existing malware, the results presented in this paper show that signatures generated by the DBN allow for an accurate classification of new malware variants. Using a dataset containing hundreds of variants for several major malware families, our method achieves 98.6% classification accuracy using the signatures generated by the DBN. The presented method is completely agnostic to the type of malware behavior that is logged (e.g., API calls and their parameters, registry entries, websites and ports accessed, etc.), and can use any raw input from a sandbox to successfully train the deep neural network which is used to generate malware signatures.
An Empirical Comparison of Machine Learning Models for Time Series Forecasting

In this work we present a large scale comparison study for the major machine learning models for time series forecasting. Specifically, we apply the models on the monthly M3 time series competition data (around a thousand time series). There have been very few, if any, large scale comparison studies for machine learning models for the regression or the time series forecasting problems, so we hope this study would fill this gap. The models considered are multilayer perceptron, Bayesian neural networks, radial basis functions, generalized regression neural networks (also called kernel regression), K-nearest neighbor regression, CART regression trees, support vector regression, and Gaussian processes. The study reveals significant differences between the different methods. The best two methods turned out to be the multilayer perceptron and the Gaussian process regression. In addition to model comparisons, we have tested different preprocessing methods and have shown that they have different impacts on the performance.
6D Object Pose Estimation with Depth Images: A Seamless Approach for Robotic Interaction and Augmented Reality

To determine the 3D orientation and 3D location of objects in the surroundings of a camera mounted on a robot or mobile device, we developed two powerful algorithms in object detection and temporal tracking that are combined seamlessly for robotic perception and interaction as well as Augmented Reality (AR). A separate evaluation of, respectively, the object detection and the temporal tracker demonstrates the important stride in research as well as the impact on industrial robotic applications and AR. When evaluated on a standard dataset, the detector produced the highest f1score with a large margin while the tracker generated the best accuracy at a very low latency of approximately 2 ms per frame with one CPU core – both algorithms outperforming the state of the art. When combined, we achieve a powerful framework that is robust to handle multiple instances of the same object under occlusion and clutter while attaining real-time performance. Aiming at stepping beyond the simple scenarios used by current systems, often constrained by having a single object in absence of clutter, averting to touch the object to prevent close-range partial occlusion, selecting brightly colored objects to easily segment them individually or assuming that the object has simple geometric structure, we demonstrate the capacity to handle challenging cases under clutter, partial occlusion and varying lighting conditions with objects of different shapes and sizes.
A hierarchical classification approach to automated essay scoring

This study evaluates the use of a hierarchical classification approach to automated assessment of essays. Automated essay scoring (AES) generally relies on machine learning techniques that compute essay scores using a set of text variables. Unlike previous studies that rely on regression models, this study computes essay scores using a hierarchical approach, analogous to an incremental algorithm for hierarchical classification. The corpus in this study consists of 1243 argumentative (persuasive) essays written on 14 different prompts, across 3 different grade levels (9th grade, 11th grade, college freshman), and four different time limits for writing or temporal conditions (untimed essays and essays written in 10, 15, and 25 minute increments). The features included in the analysis are computed using the automated tools, Coh-Metrix, the Writing Assessment Tool (WAT), and Linguistic Inquiry and Word Count (LIWC). Overall, the models developed to score all the essays in the data set report 55% exact accuracy and 92% adjacent accuracy between the predicted essay scores and the human scores. The results indicate that this is a promising approach to AES that could provide more specific feedback to writers and may be relevant to other natural language computations, such as the scoring of short answers in comprehension or knowledge assessments.
Describing Patterns and Disruptions in Large Scale Mobile App Usage Data

The advertising industry is seeking to use the unique data provided by the increasing usage of mobile devices and mobile applications (apps) to improve targeting and the experience with apps. As a consequence, understanding user behaviours with apps has gained increased interests from both academia and industry. In this paper we study user app engagement patterns and disruptions of those patterns in a data set unique in its scale and coverage of user activity. First, we provide a detailed account of temporal user activity patterns with apps and compare these to previous studies on app usage behavior. Then, in the second part, and the main contribution of this work, we take advantage of the scale and coverage of our sample and show how app usage behavior is disrupted through major political, social, and sports events.
Attachment, self-compassion, empathy, and subjective well-being among college students and community adults.

Research on subjective well-being suggests that it is only partly a function of environmental circumstances. There may be a personality characteristic or a resilient disposition toward experiencing high levels of well-being even in unfavorable circumstances. Adult attachment may contribute to this resilient disposition. This study examined whether the association between attachment anxiety and subjective well-being was mediated by Neff's (2003a, 2003b) concept of self-compassion. It also examined empathy toward others as a mediator in the association between attachment avoidance and subjective well-being. In Study 1, 195 college students completed self-report surveys. In Study 2, 136 community adults provided a cross-validation of the results. As expected, across these 2 samples, findings suggested that self-compassion mediated the association between attachment anxiety and subjective well-being, and emotional empathy toward others mediated the association between attachment avoidance and subjective well-being.
Divided They Tweet: The Network Structure of Political Microbloggers and Discussion Topics

In the context of a national election, this study explores more than 69,000 Twitter messages containing mentions of political parties and about 2,500 related user profiles to investigate the network structure of political microbloggers with respect to, first, their party preference and, second, the topics they discuss. We find that political microbloggers tend to follow like-minded peers. Microbloggers in a cohesive group tend to have the same political preferences. In addition, we conduct a content analysis of the political debate on Twitter to explore which topics and politicians are discussed and whether this debate reflects an ideological divide among participating users. While there are some discussion topics that are dominated by politically like-minded microbloggers, the majority of topics is discussed by a diverse group of microbloggers with various political