nils-herrmann/clean_scirepeval huggingface dataset:
How Good Are My Predictions? Efficiently Approximating Precision-Recall Curves for Massive Datasets

Large scale machine learning produces massive datasets whose items are often associated with a confidence level and can thus be ranked. However, computing the precision of these resources requires human annotation, which is often prohibitively expensive and is therefore skipped. We consider the problem of cost-effectively approximating precisionrecall (PR) or ROC curves for such systems. Our novel approach, called PAULA, provides theoretically guaranteed lower and upper bounds on the underlying precision function while relying on only O(logN) annotations for a resource with N items. This contrasts favorably with Θ( √ N logN) annotations needed by commonly used sampling based methods. Our key insight is to capitalize on a natural monotonicity property of the underlying confidence-based ranking. PAULA provides tight bounds for PR curves using, e.g., only 17K annotations for resources with 200K items and 48K annotations for resources with 2B items. We use PAULA to evaluate a subset of the much utilized PPDB paraphrase database and a recent Science knowledge base.
A Survey on Data Collection for Machine Learning: a Big Data - AI Integration Perspective

Data collection is a major bottleneck in machine learning and an active research topic in multiple communities. There are largely two reasons data collection has recently become a critical issue. First, as machine learning is becoming more widely-used, we are seeing new applications that do not necessarily have enough labeled data. Second, unlike traditional machine learning where feature engineering is the bottleneck, deep learning techniques automatically generate features, but instead require large amounts of labeled data. Interestingly, recent research in data collection comes not only from the machine learning, natural language, and computer vision communities, but also from the data management community due to the importance of handling large amounts of data. In this survey, we perform a comprehensive study of data collection from a data management point of view. Data collection largely consists of data acquisition, data labeling, and improvement of existing data or models. We provide a research landscape of these operations, provide guidelines on which technique to use when, and identify interesting research challenges. The integration of machine learning and data management for data collection is part of a larger trend of Big data and Artificial Intelligence (AI) integration and opens many opportunities for new research.
Publication Trends over 10 Years of Computational Thinking Research.

The current study aimed to review studies on computational thinking (CT) indexed in Web of Science (WOS) and ERIC databases. A thorough search in electronic databases revealed 96 studies on computational thinking which were published between 2006 and 2016. Studies were exposed to a quantitative content analysis through using an article control form developed by the researchers. Studies were summarized under several themes including the research purpose, design, methodology, sampling characteristics, data analysis, and main findings. The findings were reported using descriptive statistics to see the trends. It was observed that there was an increase in the number of CT studies in recent years, and these were mainly conducted in the field of computer sciences. In addition, CT studies were mostly published in journals in the field of Education and Instructional Technologies. Theoretical paradigm and literature review design were preferred more in previous studies. The most commonly used sampling method was the purposive sampling. It was also revealed that samples of previous CT studies were generally pre-college students. Written data collection tools and quantitative analysis were mostly used in reviewed papers. Findings mainly focused on CT skills. Based on current findings, recommendations and implications for further researches were provided.
Security Analytics: Big Data Analytics for cybersecurity: A review of trends, techniques and tools

The rapid growth of the Internet has brought with it an exponential increase in the type and frequency of cyber attacks. Many well-known cybersecurity solutions are in place to counteract these attacks. However, the generation of Big Data over computer networks is rapidly rendering these traditional solutions obsolete. To cater for this problem, corporate research is now focusing on Security Analytics, i.e., the application of Big Data Analytics techniques to cybersecurity. Analytics can assist network managers particularly in the monitoring and surveillance of real-time network streams and real-time detection of both malicious and suspicious (outlying) patterns. Such a behavior is envisioned to encompass and enhance all traditional security techniques. This paper presents a comprehensive survey on the state of the art of Security Analytics, i.e., its description, technology, trends, and tools. It hence aims to convince the reader of the imminent application of analytics as an unparalleled cybersecurity solution in the near future.
A non-local algorithm for image denoising

We propose a new measure, the method noise, to evaluate and compare the performance of digital image denoising methods. We first compute and analyze this method noise for a wide class of denoising algorithms, namely the local smoothing filters. Second, we propose a new algorithm, the nonlocal means (NL-means), based on a nonlocal averaging of all pixels in the image. Finally, we present some experiments comparing the NL-means algorithm and the local smoothing filters.
Analysis and optimization on the angular color shift of RGB OLED displays

Microcavites contribute to enhancing the optical efficiency and color saturation of an organic light emitting diode (OLED) display. A major tradeoff of the strong cavity effect is its apparent color shift, especially for RGB-based OLED displays, due to their mismatched angular intensity distributions. To mitigate the color shift, in this work we first analyze the emission spectrum shifts and angular distributions for the OLEDs with strong and weak cavities, both theoretically and experimentally. Excellent agreement between simulation and measurement is obtained. Next, we propose a systematic approach for RGB-OLED displays based on multi-objective optimization algorithms. Three objectives, namely external quantum efficiency (EQE), color gamut coverage, and angular color shift of primary and mixed colors, can be optimized simultaneously. Our optimization algorithm is proven to be effective for suppressing color shift while keeping a relatively high optical efficiency and wide color gamut.
Automatic Segmentation of MR Brain Images With a Convolutional Neural Network

Automatic segmentation in MR brain images is important for quantitative analysis in large-scale studies with images acquired at all ages. This paper presents a method for the automatic segmentation of MR brain images into a number of tissue classes using a convolutional neural network. To ensure that the method obtains accurate segmentation details as well as spatial consistency, the network uses multiple patch sizes and multiple convolution kernel sizes to acquire multi-scale information about each voxel. The method is not dependent on explicit features, but learns to recognise the information that is important for the classification based on training data. The method requires a single anatomical MR image only. The segmentation method is applied to five different data sets: coronal T2-weighted images of preterm infants acquired at 30 weeks postmenstrual age (PMA) and 40 weeks PMA, axial T2-weighted images of preterm infants acquired at 40 weeks PMA, axial T1-weighted images of ageing adults acquired at an average age of 70 years, and T1-weighted images of young adults acquired at an average age of 23 years. The method obtained the following average Dice coefficients over all segmented tissue classes for each data set, respectively: 0.87, 0.82, 0.84, 0.86, and 0.91. The results demonstrate that the method obtains accurate segmentations in all five sets, and hence demonstrates its robustness to differences in age and acquisition protocol.
Cannabidiol in Humans—The Quest for Therapeutic Targets

Cannabidiol (CBD), a major phytocannabinoid constituent of cannabis, is attracting growing attention in medicine for its anxiolytic, antipsychotic, antiemetic and anti-inflammatory properties. However, up to this point, a comprehensive literature review of the effects of CBD in humans is lacking. The aim of the present systematic review is to examine the randomized and crossover studies that administered CBD to healthy controls and to clinical patients. A systematic search was performed in the electronic databases PubMed and EMBASE using the key word "cannabidiol". Both monotherapy and combination studies (e.g., CBD + ∆9-THC) were included. A total of 34 studies were identified: 16 of these were experimental studies, conducted in healthy subjects, and 18 were conducted in clinical populations, including multiple sclerosis (six studies), schizophrenia and bipolar mania (four studies), social anxiety disorder (two studies), neuropathic and cancer pain (two studies), cancer anorexia (one study), Huntington's disease (one study), insomnia (one study), and epilepsy (one study). Experimental studies indicate that a high-dose of inhaled/intravenous CBD is required to inhibit the effects of a lower dose of ∆9-THC. Moreover, some experimental and clinical studies suggest that oral/oromucosal CBD may prolong and/or intensify ∆9-THC-induced effects, whereas others suggest that it may inhibit ∆9-THC-induced effects. Finally, preliminary clinical trials suggest that high-dose oral CBD (150-600 mg/d) may exert a therapeutic effect for social anxiety disorder, insomnia and epilepsy, but also that it may cause mental sedation. Potential pharmacokinetic and pharmacodynamic explanations for these results are discussed.
SyntaxSQLNet: Syntax Tree Networks for Complex and Cross-DomainText-to-SQL Task

Most existing studies in text-to-SQL tasks do not require generating complex SQL queries with multiple clauses or sub-queries, and generalizing to new, unseen databases. In this paper we propose SyntaxSQLNet, a syntax tree network to address the complex and crossdomain text-to-SQL generation task. SyntaxSQLNet employs a SQL specific syntax tree-based decoder with SQL generation path history and table-aware column attention encoders. We evaluate SyntaxSQLNet on a new large-scale text-to-SQL corpus containing databases with multiple tables and complex SQL queries containing multiple SQL clauses and nested queries. We use a database split setting where databases in the test set are unseen during training. Experimental results show that SyntaxSQLNet can handle a significantly greater number of complex SQL examples than prior work, outperforming the previous state-of-the-art model by 9.5% in exact matching accuracy. To our knowledge, we are the first to study this complex text-to-SQL task. Our task and models with the latest updates are available at https://yale-lily. github.io/seq2sql/spider.
A Non-Cooperative Game Approach to Autonomous Racing

We consider autonomous racing of two cars and present an approach to formulate the decision making as a non-cooperative non-zero-sum game. The game is formulated by restricting both players to fulfill static track constraints as well as collision constraints which depend on the combined actions of the two players. At the same time the players try to maximize their own progress. In the case where the action space of the players is finite, the racing game can be reformulated as a bimatrix game. For this bimatrix game, we show that the actions obtained by a sequential maximization approach where only the follower considers the action of the leader are identical to a Stackelberg and a Nash equilibrium in pure strategies. Furthermore, we propose a game promoting blocking, by additionally rewarding the leading car for staying ahead at the end of the horizon. We show that this changes the Stackelberg equilibrium, but has a minor influence on the Nash equilibria. For an online implementation, we propose to play the games in a moving horizon fashion, and we present two methods for guaranteeing feasibility of the resulting coupled repeated games. Finally, we study the performance of the proposed approaches in simulation for a set-up that replicates the miniature race car tested at the Automatic Control Laboratory of ETH Zürich. The simulation study shows that the presented games can successfully model different racing behaviors and generate interesting racing situations.
Texture segmentation with Fully Convolutional Networks

In the last decade, deep learning has contributed to advances in a wide range computer vision tasks including texture analysis. This paper explores a new approach for texture segmentation using deep convolutional neural networks, sharing important ideas with classic filter bank based texture segmentation methods. Several methods are developed to train Fully Convolutional Networks to segment textures in various applications. We show in particular that these networks can learn to recognize and segment a type of texture, e.g. wood and grass from texture recognition datasets (no training segmentation). We demonstrate that Fully Convolutional Networks can learn from repetitive patterns to segment a particular texture from a single image or even a part of an image. We take advantage of these findings to develop a method that is evaluated on a series of supervised and unsupervised experiments and improve the state of the art on the Prague texture segmentation datasets.
Meta-Learning: A Survey

Meta-learning, or learning to learn, is the science of systematically observing how different machine learning approaches perform on a wide range of learning tasks, and then learning from this experience, or meta-data, to learn new tasks much faster than otherwise possible. Not only does this dramatically speed up and improve the design of machine learning pipelines or neural architectures, it also allows us to replace hand-engineered algorithms with novel approaches learned in a data-driven way. In this chapter, we provide an overview of the state of the art in this fascinating and continuously evolving field.
Ginger and Propolis Exert Neuroprotective Effects against Monosodium Glutamate-Induced Neurotoxicity in Rats

Central nervous system cytotoxicity is linked to neurodegenerative disorders. The objective of the study was to investigate whether monosodium glutamate (MSG) neurotoxicity can be reversed by natural products, such as ginger or propolis, in male rats. Four different groups of Wistar rats were utilized in the study. Group A served as a normal control, whereas group B was orally administered with MSG (100 mg/kg body weight, via oral gavage). Two additional groups, C and D, were given MSG as group B along with oral dose (500 mg/kg body weight) of either ginger or propolis (600 mg/kg body weight) once a day for two months. At the end, the rats were sacrificed, and the brain tissue was excised and levels of neurotransmitters, ß-amyloid, and DNA oxidative marker 8-OHdG were estimated in the brain homogenates. Further, formalin-fixed and paraffin-embedded brain sections were used for histopathological evaluation. The results showed that MSG increased lipid peroxidation, nitric oxide, neurotransmitters, and 8-OHdG as well as registered an accumulation of ß-amyloid peptides compared to normal control rats. Moreover, significant depletions of glutathione, superoxide dismutase, and catalase as well as histopathological alterations in the brain tissue of MSG-treated rats were noticed in comparison with the normal control. In contrast, treatment with ginger greatly attenuated the neurotoxic effects of MSG through suppression of 8-OHdG and β-amyloid accumulation as well as alteration of neurotransmitter levels. Further improvements were also noticed based on histological alterations and reduction of neurodegeneration in the brain tissue. A modest inhibition of the neurodegenerative markers was observed by propolis. The study clearly indicates a neuroprotective effect of ginger and propolis against MSG-induced neurodegenerative disorders and these beneficial effects could be attributed to the polyphenolic compounds present in these natural products.
Frontomaxillary and mandibulomaxillary facial angles at 11 + 0 to 13 + 6 weeks in fetuses with trisomy 18.

To define the relative position of the maxilla and mandible in fetuses with trisomy 18 at 11 + 0 to 13 + 6 weeks of gestation.



A three-dimensional (3D) volume of the fetal head was obtained before karyotyping at 11 + 0 to 13 + 6 weeks of gestation in 36 fetuses subsequently found to have trisomy 18, and 200 chromosomally normal fetuses. The frontomaxillary facial (FMF) angle and the mandibulomaxillary facial (MMF) angle were measured in a mid-sagittal view of the fetal face.



In the chromosomally normal group both the FMF and MMF angles decreased significantly with crown-rump length (CRL). In the trisomy 18 fetuses the FMF angle was significantly greater and the angle was above the 95(th) centile of the normal range in 21 (58.3%) cases. In contrast, in trisomy 18 fetuses the MMF angle was significantly smaller than that in normal fetuses and the angle was below the 5(th) centile of the normal range in 12 (33.3%) cases.



Trisomy 18 at 11 + 0 to 13 + 6 weeks of gestation is associated with both mid-facial hypoplasia and micrognathia or retrognathia that can be documented by measurement of the FMF angle and MMF angle, respectively.
QUINT: Interpretable Question Answering over Knowledge Bases

We present QUINT, a live system for question answering over knowledge bases. QUINT automatically learns role-aligned utterance-query templates from user questions paired with their answers. When QUINT answers a question, it visualizes the complete derivation sequence from the natural language utterance to the final answer. The derivation provides an explanation of how the syntactic structure of the question was used to derive the structure of a SPARQL query, and how the phrases in the question were used to instantiate different parts of the query. When an answer seems unsatisfactory, the derivation provides valuable insights towards reformulating the question.
Quality assessment for Linked Data: A Survey

The development and standardization of Semantic Web technologies has resulted in an unprecedented volume of data being published on the Web as Linked Data (LD). However, we observe widely varying data quality ranging from extensively curated datasets to crowdsourced and extracted data of relatively low quality. In this article, we present the results of a systematic review of approaches for assessing the quality of LD. We gather existing approaches and analyze them qualitatively. In particular, we unify and formalize commonly used terminologies across papers related to data quality and provide a comprehensive list of 18 quality dimensions and 69 metrics. Additionally, we qualitatively analyze the 30 core approaches and 12 tools using a set of attributes. The aim of this article is to provide researchers and data curators a comprehensive understanding of existing work, thereby encouraging further experimentation and development of new approaches focused towards data quality, specifically for LD.
Natural Language Processing meets Journalism III

We collect a corpus of 1554 online news articles from 23 RSS feeds and analyze it in terms of controversy and sentiment. We use several existing sentiment lexicons and lists of controversial terms to perform a number of statistical analyses that explore how sentiment and controversy are related. We conclude that the negative sentiment and controversy are not necessarily positively correlated as has been claimed in the past. In addition, we apply an information theoretic approach and suggest that entropy might be a good predictor of controversy.
Executive Attention and Metacognitive Regulation

Metacognition refers to any knowledge or cognitive process that monitors or controls cognition. We highlight similarities between metacognitive and executive control functions, and ask how these processes might be implemented in the human brain. A review of brain imaging studies reveals a circuitry of attentional networks involved in these control processes, with its source located in midfrontal areas. These areas are active during conflict resolution, error correction, and emotional regulation. A developmental approach to the organization of the anatomy involved in executive control provides an added perspective on how these mechanisms are influenced by maturation and learning, and how they relate to metacognitive activity.
Hand gesture recognition using neural networks

Visual Interpretation of gestures can be useful in accomplishing natural Human Computer Interactions (HCI). In this paper we proposed a method for recognizing hand gestures. We have designed a system which can identify specific hand gestures and use them to convey information. At any time, a user can exhibit his/her hand doing a specific gesture in front of a web camera linked to a computer. Firstly, we captured the hand gesture of a user and stored it on disk. Then we read those videos captured one by one, converted them to binary images and created 3D Euclidian Space of binary values. We have used supervised feed-forward neural net based training and back propagation algorithm for classifying hand gestures into ten categories: hand pointing up, pointing down, pointing left, pointing right and pointing front and number of fingers user was showing. We could achieve up to 89% correct results on a typical test set.
Processing-in-Memory for Energy-Efficient Neural Network Training: A Heterogeneous Approach

Neural networks (NNs) have been adopted in a wide range of application domains, such as image classification, speech recognition, object detection, and computer vision. However, training NNs – especially deep neural networks (DNNs) – can be energy and time consuming, because of frequent data movement between processor and memory. Furthermore, training involves massive fine-grained operations with various computation and memory access characteristics. Exploiting high parallelism with such diverse operations is challenging. To address these challenges, we propose a software/hardware co-design of heterogeneous processing-in-memory (PIM) system. Our hardware design incorporates hundreds of fix-function arithmetic units and ARM-based programmable cores on the logic layer of a 3D die-stacked memory to form a heterogeneous PIM architecture attached to CPU. Our software design offers a programming model and a runtime system that program, offload, and schedule various NN training operations across compute resources provided by CPU and heterogeneous PIM. By extending the OpenCL programming model and employing a hardware heterogeneity-aware runtime system, we enable high program portability and easy program maintenance across various heterogeneous hardware, optimize system energy efficiency, and improve hardware utilization.