nils-herrmann/clean_scirepeval huggingface dataset:
(More) Efficient Reinforcement Learning via Posterior Sampling

References • We study efficient exploration in reinforcement learning. • Most provably-efficient learning algorithms introduce optimism about poorly understood states and actions. • Motivated by potential advantages relative to optimistic algorithms, we study an alternative approach: posterior sampling for reinforcement learning (PSRL). • This is the extension of the Thompson sampling algorithm for multi-armed bandit problems to reinforcement learning. • We establish the first regret bounds for this algorithm.  Conceptually simple, separates algorithm from analysis: • PSRL selects policies according to the probability they are optimal without need for explicit construction of confidence sets. • UCRL2 bounds error in each s, a separately, which allows for worst-case mis-estimation to occur simultaneously in every s, a . • We believe this will make PSRL more statistically efficient.
Understanding Behaviors that Lead to Purchasing: A Case Study of Pinterest

Online e-commerce applications are becoming a primary vehicle for people to find, compare, and ultimately purchase products. One of the fundamental questions that arises in e-commerce is to characterize, understand, and model user long-term purchasing intent, which is important as it allows for personalized and context relevant e-commerce services.
 In this paper we study user activity and purchasing behavior with the goal of building models of time-varying user purchasing intent. We analyze the purchasing behavior of nearly three million Pinterest users to determine short-term and long-term signals in user behavior that indicate higher purchase intent.
 We find that users with long-term purchasing intent tend to save and clickthrough on more content. However, as users approach the time of purchase their activity becomes more topically focused and actions shift from saves to searches. We further find that purchase signals in online behavior can exist weeks before a purchase is made and can also be traced across different purchase categories. Finally, we synthesize these insights in predictive models of user purchasing intent. Taken together, our work identifies a set of general principles and signals that can be used to model user purchasing intent across many content discovery applications.
Measuring E-Marketing Mix Elements for Online Business

E-marketing strategy is normally based and built up on the traditional 4 P's (Product, Price, Promotion and Place) that forms the classic marketi ng mix; e-marketing’s uniqueness is created using a series of specific and relational functions that are combined with the 4P’s to form the emarketing mix elements, each of which contain assoc iated e-marketing mix tools that are provided on business web sites to facilitate sales transactions. This research analyses the importance of each e-marketing tool related to its supporting e-marketing mix element. Furthermore, the composite score of each e-marketin g mix element is determined. This research concludes with a discussion of the relative weights of e-marketing tools.
A review of three-phase improved power quality AC-DC converters

Three-phase AC-DC converters have been developed to a matured level with improved power quality in terms of power-factor correction, reduced total harmonic distortion at input AC mains, and regulated DC output in buck, boost, buck-boost, multilevel, and multipulse modes with unidirectional and bidirectional power flow. This paper presents an exhaustive review of three-phase improved power quality AC-DC converters (IPQCs) configurations, control strategies, selection of components, comparative factors, recent trends, their suitability, and selection for specific applications. It is aimed at presenting a state of the art on the IPQC technology to researchers, designers, and application engineers dealing with three-phase AC-DC converters. A classified list of around 450 research articles on IPQCs is also appended for a quick reference.
A general regression neural network

A memory-based network that provides estimates of continuous variables and converges to the underlying (linear or nonlinear) regression surface is described. The general regression neural network (GRNN) is a one-pass learning algorithm with a highly parallel structure. It is shown that, even with sparse data in a multidimensional measurement space, the algorithm provides smooth transitions from one observed value to another. The algorithmic form can be used for any regression problem in which an assumption of linearity is not justified.
AnswerBus question answering system

Web information retrieval. It accepts users’ natural-language questions in English, German, French, Spanish, Italian and Portuguese and provides answers in English. Five search engines and directories are used to retrieve Web pages that are relevant to user questions. From the Web pages, AnswerBus extracts sentences that are determined to contain answers. Its current rate of correct answers to TREC-8’s 200 questions is 70.5% with the average response time to the questions being seven seconds. The performance of AnswerBus in terms of accuracy and response time is better than other similar systems.
Weakly-supervised DCNN for RGB-D Object Recognition in Real-World Applications Which Lack Large-scale Annotated Training Data

This paper addresses the problem of RGBD object recognition in real-world applications, where large amounts of annotated training data are typically unavailable. To overcome this problem, we propose a novel, weakly-supervised learning architecture (DCNN-GPC) which combines parametric models (a pair of Deep Convolutional Neural Networks (DCNN) for RGB and D modalities) with non-parametric models (Gaussian Process Classification). Our system is initially trained using a small amount of labeled data, and then automatically propagates labels to large-scale unlabeled data. We first run 3Dbased objectness detection on RGBD videos to acquire many unlabeled object proposals, and then employ DCNN-GPC to label them. As a result, our multi-modal DCNN can be trained end-to-end using only a small amount of human annotation. Finally, our 3D-based objectness detection and multi-modal DCNN are integrated into a real-time detection and recognition pipeline. In our approach, bounding-box annotations are not required and boundary-aware detection is achieved. We also propose a novel way to pretrain a DCNN for the depth modality, by training on virtual depth images projected from CAD models. We pretrain our multi-modal DCNN on public 3D datasets, achieving performance comparable to state-of-the-art methods on Washington RGBS Dataset. We then finetune the network by further training on a small amount of annotated data from our novel dataset of industrial objects (nuclear waste simulants). Our weakly supervised approach has demonstrated to be highly effective in solving a novel RGBD object recognition application which lacks of human annotations.
Link prediction in directed social networks

In today’s online social networks, it becomes essential to help newcomers as well as existing community members to find new social contacts. In scientific literature, this recommendation task is known as link prediction. Link prediction has important practical applications in social network platforms. It allows social network platform providers to recommend friends to their users. Another application is to infer missing links in partially observed networks. The shortcoming of many of the existing link prediction methods is that they mostly focus on undirected graphs only. This work closes this gap and introduces link prediction methods and metrics for directed graphs. Here, we compare well-known similarity metrics and their suitability for link prediction in directed social networks. We advance existing techniques and propose mining of subgraph patterns that are used to predict links in networks such as GitHub, GooglePlus, and Twitter. Our results show that the proposed metrics and techniques yield more accurate predictions when compared with metrics not accounting for the directed nature of the underlying networks.
Correlates of Qualtiy Sleep and Academic Performance

Sleep problems bave become epidemic aod traditional research has discovered many causes of poor sleep. The purpose of this study was to complement existiog research by using a salutogenic or health origins framework to investigate the correlates of good sleep. The aoalysis for this study used the National College Health Assessment data that included 54,111 participaots at 71 institutions. Participaots were raodomly selected or were in raodomly selected classrooms. Results of these aoalyses indicated that males aod females who reported "good sleep" were more likely to have engaged regularly in physical activity, felt less exhausted, were more likely to have a healthy Body Mass Index (BMI), aod also performed better academically. In addition, good male sleepers experienced less anxietY aod had less back pain. Good female sleepers also had fewer abusive relationships aod fewer broken bones, were more likely to have been nonsmokers aod were not binge drinkers. Despite the limitations of this exploratory study, these results are compelling, however they suggest the need for future research to clarify the identified relationships.
Compound TCP: A scalable and TCP-friendly congestion control for high-speed networks

Many applications require fast data transfer over high speed and long distance networks. However, standard TCP fails to fully utilize the network capacity in high-speed and long distance networks due to its conservative congestion control (CC) algorithm. Some works have been proposed to improve the connection’s throughput by adopting more aggressive loss-based CC algorithms, which may severely decrease the throughput of regular TCP flows sharing the network path. On the other hand, pure delay-based approaches may not work well if they compete with loss-based flows. In this paper, we propose a novel Compound TCP (CTCP) approach, which is a synergy of delay-based and loss-based approach. More specifically, we add a scalable delay-based component into the standard TCP Reno congestion avoidance algorithm (a.k.a., the loss-based component). The sending rate of CTCP is controlled by both components. This new delay-based component can rapidly increase sending rate when the network path is under utilized, but gracefully retreat in a busy network when a bottleneck queue is built. Augmented with this delay-based component, CTCP provides very good bandwidth scalability and at the same time achieves good TCP-fairness. We conduct extensive packet level simulations and test our CTCP implementation on the Windows platform over a production high-speed network link in the Microsoft intranet. Our simulation and experiments results verify the properties of CTCP.
Gamification. using game-design elements in non-gaming contexts

"Gamification" is an informal umbrella term for the use of video game elements in non-gaming systems to improve user experience (UX) and user engagement. The recent introduction of 'gamified' applications to large audiences promises new additions to the existing rich and diverse research on the heuristics, design patterns and dynamics of games and the positive UX they provide. However, what is lacking for a next step forward is the integration of this precise diversity of research endeavors. Therefore, this workshop brings together practitioners and researchers to develop a shared understanding of existing approaches and findings around the gamification of information systems, and identify key synergies, opportunities, and questions for future research.
Review Networks for Caption Generation

We propose a novel extension of the encoder-decoder framework, called a review network. The review network is generic and can enhance any existing encoderdecoder model: in this paper, we consider RNN decoders with both CNN and RNN encoders. The review network performs a number of review steps with attention mechanism on the encoder hidden states, and outputs a thought vector after each review step; the thought vectors are used as the input of the attention mechanism in the decoder. We show that conventional encoder-decoders are a special case of our framework. Empirically, we show that our framework improves over state-ofthe-art encoder-decoder systems on the tasks of image captioning and source code
Control-limited differential dynamic programming

Trajectory optimizers are a powerful class of methods for generating goal-directed robot motion. Differential Dynamic Programming (DDP) is an indirect method which optimizes only over the unconstrained control-space and is therefore fast enough to allow real-time control of a full humanoid robot on modern computers. Although indirect methods automatically take into account state constraints, control limits pose a difficulty. This is particularly problematic when an expensive robot is strong enough to break itself. In this paper, we demonstrate that simple heuristics used to enforce limits (clamping and penalizing) are not efficient in general. We then propose a generalization of DDP which accommodates box inequality constraints on the controls, without significantly sacrificing convergence quality or computational effort. We apply our algorithm to three simulated problems, including the 36-DoF HRP-2 robot. A movie of our results can be found here goo.gl/eeiMnn.
Making Learning and Web 2.0 Technologies Work for Higher Learning Institutions in Africa

This paper seeks to assess the extent to which the learning and web 2.0 technologies are utilized to support learning and teaching in Africa’s higher learning institutions, with a specific focus on Tanzania's public universities.  A combination of "content analysis" and "semistructured interviews" was used to collect data. The study conducted semi-structured interviews with ICT personnel from six out of the eight public universities in Tanzania in 2011.  The study found that the adoption of e-learning and web 2.0 technologies is still in its infancy stages in the Tanzania public universities. However, there was much enthusiasm amongst respondents for developing the potential of e-learning and web 2.0 tools in
Brain tumor segmentation with Deep Neural Networks

In this paper, we present a fully automatic brain tumor segmentation method based on Deep Neural Networks (DNNs). The proposed networks are tailored to glioblastomas (both low and high grade) pictured in MR images. By their very nature, these tumors can appear anywhere in the brain and have almost any kind of shape, size, and contrast. These reasons motivate our exploration of a machine learning solution that exploits a flexible, high capacity DNN while being extremely efficient. Here, we give a description of different model choices that we've found to be necessary for obtaining competitive performance. We explore in particular different architectures based on Convolutional Neural Networks (CNN), i.e. DNNs specifically adapted to image data. We present a novel CNN architecture which differs from those traditionally used in computer vision. Our CNN exploits both local features as well as more global contextual features simultaneously. Also, different from most traditional uses of CNNs, our networks use a final layer that is a convolutional implementation of a fully connected layer which allows a 40 fold speed up. We also describe a 2-phase training procedure that allows us to tackle difficulties related to the imbalance of tumor labels. Finally, we explore a cascade architecture in which the output of a basic CNN is treated as an additional source of information for a subsequent CNN. Results reported on the 2013 BRATS test data-set reveal that our architecture improves over the currently published state-of-the-art while being over 30 times faster.
Contributions to the Theory of Optimal Control

THIS is one of the two ground-breaking papers by Kalman that appeared in 1960—with the other one (discussed next) being the filtering and prediction paper. This first paper, which deals with linear-quadratic feedback control, set the stage for what came to be known as LQR (Linear-Quadratic-Regulator) control, while the combination of the two papers formed the basis for LQG (Linear-Quadratic-Gaussian) control. Both LQR and LQG control had major influence on researchers, teachers, and practitioners of control in the decades that followed. The idea of designing a feedback controller such that the integral of the square of tracking error is minimized was first proposed by Wiener  and Hall , and further developed in the influential book by Newton, Gould and Kaiser . However, the problem formulation in this book remained unsatisfactory from a mathematical point of view, but, more importantly, the algorithms obtained allowed application only to rather low order systems and were thus of limited value. This is not surprising since it basically took until theH2-interpretation in the 1980s of LQG control before a satisfactory formulation of least squares feedback control design was obtained. Kalman’s formulation in terms of finding the least squares control that evolves from an arbitrary initial state is a precise formulation of the optimal least squares transient control problem. The paper introduced the very important notion of c ntrollability, as the possibility of transfering any initial state to zero by a suitable control action. It includes the necessary and sufficient condition for controllability in terms of the positive definiteness of the Controllability Grammian, and the fact that the linear time-invariant system withn states,
Analysis of Thompson Sampling for the Multi-armed Bandit Problem

We show that the Thompson Sampling algorithm achieves logarithmic expected regret for the Bernoulli multi-armed bandit problem. More precisely, for the two-armed bandit problem, the expected regret in time T is O( lnT ∆ + 1 ∆3 ). And, for the N -armed bandit problem, the expected regret in time T is O( [ ( ∑N i=2 1 ∆i ) ] lnT ). Our bounds are optimal but for the dependence on ∆i and the constant factors in big-Oh.
Distributional Similarity vs. PU Learning for Entity Set Expansion

Distributional similarity is a classic technique for entity set expansion, where the system is given a set of seed entities of a particular class, and is asked to expand the set using a corpus to obtain more entities of the same class as represented by the seeds. This paper shows that a machine learning model called positive and unlabeled learning (PU learning) can model the set expansion problem better. Based on the test results of 10 corpora, we show that a PU learning technique outperformed distributional similarity significantly.
From QoS to QoE: A Tutorial on Video Quality Assessment

Quality of experience (QoE) is the perceptual quality of service (QoS) from the users' perspective. For video service, the relationship between QoE and QoS (such as coding parameters and network statistics) is complicated because users' perceptual video quality is subjective and diversified in different environments. Traditionally, QoE is obtained from subjective test, where human viewers evaluate the quality of tested videos under a laboratory environment. To avoid high cost and offline nature of such tests, objective quality models are developed to predict QoE based on objective QoS parameters, but it is still an indirect way to estimate QoE. With the rising popularity of video streaming over the Internet, data-driven QoE analysis models have newly emerged due to availability of large-scale data. In this paper, we give a comprehensive survey of the evolution of video quality assessment methods, analyzing their characteristics, advantages, and drawbacks. We also introduce QoE-based video applications and, finally, identify the future research directions of QoE.
An overview of statistical learning theory

Statistical learning theory was introduced in the late 1960's. Until the 1990's it was a purely theoretical analysis of the problem of function estimation from a given collection of data. In the middle of the 1990's new types of learning algorithms (called support vector machines) based on the developed theory were proposed. This made statistical learning theory not only a tool for the theoretical analysis but also a tool for creating practical algorithms for estimating multidimensional functions. This article presents a very general overview of statistical learning theory including both theoretical and algorithmic aspects of the theory. The goal of this overview is to demonstrate how the abstract learning theory established conditions for generalization which are more general than those discussed in classical statistical paradigms and how the understanding of these conditions inspired new algorithmic approaches to function estimation problems. A more detailed overview of the theory (without proofs) can be found in Vapnik (1995). In Vapnik (1998) one can find detailed description of the theory (including proofs).
Twenty-Five Years of Hydroinformatics

Hydroinformatics emerged in 1991 when numerical modelling of water expanded its range from one that was restricted to the modelling of flows to a much wider ranging sociotechnical discipline that supports stakeholders in addressing their water-related problems. However, despite numerous advances in hydroinformatics, the current practical and research effort is still very much technocratic (or techno-centric) which in turn may restrict the potential of hydroinformatics in its scope and its reach. This Special Issue, through the compilation of thirteen papers, illustrates some of the developments and applications in the field of hydroinformatics and marks the twenty-five years of its existence. We hope that this will help to further raise the awareness of the subject and its developments and applications. In the Editorial of this Special Issue, we briefly discuss the origin of hydroinformatics and we introduce the papers that are featuring in this Special Issue. We also give a way forward for future research and application.
Bug Prioritization to Facilitate Bug Report Triage

The large number of new bug reports received in bug repositories of software systems makes their management a challenging task. Handling these reports manually is time consuming, and often results in delaying the resolution of important bugs. To address this issue, a recommender may be developed which automatically prioritizes the new bug reports. In this paper, we propose and evaluate a classification based approach to build such a recommender. We use the Naïve Bayes and Support Vector Machine (SVM) classifiers, and present a comparison to evaluate which classifier performs better in terms of accuracy. Since a bug report contains both categorical and text features, another evaluation we perform is to determine the combination of features that better determines the priority of a bug. To evaluate the bug priority recommender, we use precision and recall measures and also propose two new measures, Nearest False Negatives (NFN) and Nearest False Positives (NFP), which provide insight into the results produced by precision and recall. Our findings are that the results of SVM are better than the Naïve Bayes algorithm for text features, whereas for categorical features, Naïve Bayes performance is better than SVM. The highest accuracy is achieved with SVM when categorical and text features are combined for training.