nils-herrmann/clean_scirepeval huggingface dataset:
Robust statistical modeling of monthly rainfall: The minimum density power divergence approach.

Statistical modeling of rainfall is an important challenge in meteorology, particularly from the perspective of rainfed agriculture where a proper assessment of the future availability of rainwater is necessary. The probability models mostly used for this purpose are exponential, gamma, Weibull and lognormal distributions, where the unknown model parameters are routinely estimated using the maximum likelihood estimator (MLE). However, presence of outliers or extreme observations is quite common in rainfall data and the MLEs being highly sensitive to them often leads to spurious inference. In this paper, we discuss a robust parameter estimation approach based on the minimum density power divergence estimators (MDPDEs) which provides a class of estimates through a tuning parameter including the MLE as a special case. The underlying tuning parameter controls the trade-offs between efficiency and robustness of the resulting inference; we also discuss a procedure for data-driven optimal selection of this tuning parameter as well as robust selection of an appropriate model that provides best fit to some specific rainfall data. We fit the above four parametric models to the areally-weighted monthly rainfall data from the 36 meteorological subdivisions of India for the years 1951-2014 and compare the fits based on the MLE and the proposed optimum MDPDE; the superior performances of the MDPDE based approach are illustrated for several cases. For all month-subdivision combinations, the best-fit models and the estimated median rainfall amounts are provided. Software (written in R) for calculating MDPDEs and their standard errors, optimal tuning parameter selection and model selection are also provided.
Revealing the Importance of Semantic Retrieval for Machine Reading at Scale

Machine Reading at Scale (MRS) is a challenging task in which a system is given an input query and is asked to produce a precise output by “reading” information from a large knowledge base. The task has gained popularity with its natural combination of information retrieval (IR) and machine comprehension (MC). Advancements in representation learning have led to separated progress in both IR and MC; however, very few studies have examined the relationship and combined design of retrieval and comprehension at different levels of granularity, for development of MRS systems. In this work, we give general guidelines on system design for MRS by proposing a simple yet effective pipeline system with special consideration on hierarchical semantic retrieval at both paragraph and sentence level, and their potential effects on the downstream task. The system is evaluated on both fact verification and open-domain multihop QA, achieving state-of-the-art results on the leaderboard test sets of both FEVER and HOTPOTQA. To further demonstrate the importance of semantic retrieval, we present ablation and analysis studies to quantify the contribution of neural retrieval modules at both paragraph-level and sentence-level, and illustrate that intermediate semantic retrieval modules are vital for not only effectively filtering upstream information and thus saving downstream computation, but also for shaping upstream data distribution and providing better data for downstream modeling.
Option pricing under normal dynamics with stochastic volatility

In this paper, we derive the price of a European call option of an asset following a normal process assuming stochastic volatility. The volatility is assumed to follow the Cox Ingersoll Ross (CIR) process. We then use the fast Fourier transform (FFT) to evaluate the option price given we know the characteristic function of the return analytically. We compare the results of fast Fourier transform with the Monte Carlo simulation results of our process. Further, we present a numerical example to understand the normal implied volatility of the model.
Masked-RPCA: Sparse and Low-rank Decomposition Under Overlaying Model and Application to Moving Object Detection

Foreground detection in a given video sequence is a pivotal step in many computer vision applications such as video surveillance system. Robust Principal Component Analysis (RPCA) performs low-rank and sparse decomposition and accomplishes such a task when the background is stationary and the foreground is dynamic and relatively small. A fundamental issue with RPCA is the assumption that the low-rank and sparse components are added at each element, whereas in reality, the moving foreground is overlaid on the background. We propose the representation via masked decomposition (i.e. an overlaying model) where each element either belongs to the low-rank or the sparse component, decided by a mask. We propose the Masked-RPCA algorithm to recover the mask and the low-rank components simultaneously, utilizing linearizing and alternating direction techniques. We further extend our formulation to be robust to dynamic changes in the background and enforce spatial connectivity in the foreground component. Our study shows significant improvement of the detected mask compared to post-processing on the sparse component obtained by other frameworks.
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

Recent work in language modeling demonstrates that training large transformer models advances the state of the art in Natural Language Processing applications. However, very large models can be quite difficult to train due to memory constraints. In this work, we present our techniques for training very large transformer models and implement a simple, efficient intra-layer model parallel approach that enables training transformer models with billions of parameters. Our approach does not require a new compiler or library changes, is orthogonal and complimentary to pipeline model parallelism, and can be fully implemented with the insertion of a few communication operations in native PyTorch. We illustrate this approach by converging transformer based models up to 8.3 billion parameters using 512 GPUs. We sustain 15.1 PetaFLOPs across the entire application with 76% scaling efficiency when compared to a strong single GPU baseline that sustains 39 TeraFLOPs, which is 30% of peak FLOPs. To demonstrate that large language models can further advance the state of the art (SOTA), we train an 8.3 billion parameter transformer language model similar to GPT-2 and a 3.9 billion parameter model similar to BERT. We show that careful attention to the placement of layer normalization in BERT-like models is critical to achieving increased performance as the model size grows. Using the GPT-2 model we achieve SOTA results on the WikiText103 (10.8 compared to SOTA perplexity of 15.8) and LAMBADA (66.5% compared to SOTA accuracy of 63.2%) datasets. Our BERT model achieves SOTA results on the RACE dataset (90.9% compared to SOTA accuracy of 89.4%).
Understanding truncated non-commutative geometries through computer simulations

When aiming to apply mathematical results of non-commutative geometry to physical problems the question arises how they translate to a context in which only a part of the spectrum is known. In this article we aim to detect when a finite-dimensional triple is the truncation of the Dirac spectral triple of a spin manifold. To that end, we numerically investigate the restriction that the higher Heisenberg equation places on a truncated Dirac operator. We find a bounded perturbation of the Dirac operator on the Riemann sphere that induces the same Chern class.
Deterministic algorithms for the Lovasz Local Lemma: simpler, more general, and more parallel

The Lovasz Local Lemma (LLL) is a keystone principle in probability theory, guaranteeing the existence of configurations which avoid a collection $\mathcal B$ of "bad" events which are mostly independent and have low probability. In its simplest "symmetric" form, it asserts that whenever a bad-event has probability $p$ and affects at most $d$ bad-events, and $e p d < 1$, then a configuration avoiding all $\mathcal B$ exists. 
A seminal algorithm of Moser & Tardos (2010) gives nearly-automatic randomized algorithms for most constructions based on the LLL. However, deterministic algorithms have lagged behind. We address three specific shortcomings of the prior deterministic algorithms. First, our algorithm applies to the LLL criterion of Shearer (1985); this is more powerful than alternate LLL criteria and also removes a number of nuisance parameters and leads to cleaner and more legible bounds. Second, we provide parallel algorithms with much greater flexibility in the functional form of of the bad-events. Third, we provide a derandomized version of the MT-distribution, that is, the distribution of the variables at the termination of the MT algorithm. 
We show applications to non-repetitive vertex coloring, independent transversals, strong coloring, and other problems. These give deterministic algorithms which essentially match the best previous randomized sequential and parallel algorithms.
From the Internet of Information to the Internet of Intelligence

In the era of the Internet of information, we have gone through layering, cross-layer, and cross-system design paradigms. Recently, the ``curse of modeling" and ``curse of dimensionality" of the cross-system design paradigm have resulted in the popularity of using artificial intelligence (AI) to optimize the Internet of information. However, many significant research challenges remain to be addressed for the AI approach, including the lack of high-quality training data due to privacy and resources constraints in this data-driven approach. To address these challenges, we need to take a look at humans' cooperation in a larger time scale. To facilitate cooperation in modern history, we have built three major technologies: ``grid of transportation", ``grid of energy", and ``the Internet of information". In this paper, we argue that the next cooperation paradigm could be the ``Internet of intelligence (Intelligence-Net)", where intelligence can be easily obtained like energy and information, enabled by the recent advances in blockchain technology. We present some recent advances in these areas, and discuss some open issues and challenges that need to be addressed in the future.
Big Data Analytics for Large-scale Wireless Networks

The wide proliferation of various wireless communication systems and wireless devices has led to the arrival of big data era in large-scale wireless networks. Big data of large-scale wireless networks has the key features of wide variety, high volume, real-time velocity, and huge value leading to the unique research challenges that are different from existing computing systems. In this article, we present a survey of the state-of-art big data analytics (BDA) approaches for large-scale wireless networks. In particular, we categorize the life cycle of BDA into four consecutive stages: Data Acquisition, Data Preprocessing, Data Storage, and Data Analytics. We then present a detailed survey of the technical solutions to the challenges in BDA for large-scale wireless networks according to each stage in the life cycle of BDA. Moreover, we discuss the open research issues and outline the future directions in this promising area.
MER-SDN: Machine Learning Framework for Traffic Aware Energy Efficient Routing in SDN

Software Defined Networking (SDN) achieves programmability of a network through separation of the control and data planes. It enables flexibility in network management and control. Energy efficiency is one of the challenging global problems which has both economic and environmental impact. A massive amount of information is generated in the controller of an SDN based networks. Machine learning gives the ability to computers to progressively learn from data without having to write specific instructions. In this work, we propose MER-SDN, a machine learning framework for traffic aware energy efficient routing in SDN. Feature extraction, training, and testing are the three main stages of the learning machine. Experiments are conducted on Mininet and POX controller using a real-world network topology and dynamic traffic trace from SNDlib. Results show that our approach achieves more than 65% feature size reduction, more than 70% accuracy in parameter prediction of an energy efficient heuristics algorithm, also our prediction refine heuristics converges the predicted value to the optimal parameters values with up to 25X speedup as compared to the brute force method.
Relaxed Softmax for learning from Positive and Unlabeled data

In recent years, the softmax model and its fast approximations have become the de-facto loss functions for deep neural networks when dealing with multi-class prediction. This loss has been extended to language modeling and recommendation, two fields that fall into the framework of learning from Positive and Unlabeled data. In this paper, we stress the different drawbacks of the current family of softmax losses and sampling schemes when applied in a Positive and Unlabeled learning setup. We propose both a Relaxed Softmax loss (RS) and a new negative sampling scheme based on Boltzmann formulation. We show that the new training objective is better suited for the tasks of density estimation, item similarity and next-event prediction by driving uplifts in performance on textual and recommendation datasets against classical softmax.
Extractive Summarization of Long Documents by Combining Global and Local Context

In this paper, we propose a novel neural single-document extractive summarization model for long documents, incorporating both the global context of the whole document and the local context within the current topic. We evaluate the model on two datasets of scientific papers , Pubmed and arXiv, where it outperforms previous work, both extractive and abstractive models, on ROUGE-1, ROUGE-2 and METEOR scores. We also show that, consistently with our goal, the benefits of our method become stronger as we apply it to longer documents. Rather surprisingly, an ablation study indicates that the benefits of our model seem to come exclusively from modeling the local context, even for the longest documents.
Baryon–anti-Baryon photoproduction

Baryon - anti-baryon photoproduction off a proton target is being studied in detail at Jefferson Lab with the GlueX Experiment. We observe $p\bar{p}$ and, for the first time, $\Lambda\bar{\Lambda}$ photoproduction (with $\Lambda \rightarrow \pi^- p$, $\bar{\Lambda} \rightarrow \pi^+ \bar{p}$) from thresholds up to $E_{\gamma} = 11.4$ GeV. Preliminary spectra from data accumulated during the GlueX Phase-I period are shown. Angular distributions of the photoproduced hyperons indicate more than one production mechanism in the reaction channel $\gamma p \rightarrow \Lambda\bar{\Lambda}p$. A Monte Carlo simulation with four mechanisms, tested through comparison between simulation and experimental data, is presented.
What matters, context or sentiment?: Analysing the influence of news in U.S. elections using Natural Language Processing

A key question in the analysis of collective social behaviour is related to know if and how mass media can influence public opinion. In this paper, we explore quantitatively the relation between a specific manifestation of public opinion and the intention to vote a given candidate, with the information related to the candidates in the Mass Media. We analyse the political news articles related to the US presidential campaign during the year 2016, using techniques of natural language processing. We applied recursive deep models for semantic composition over sentiment treebank to be able to detect the sentiment 1 ar X iv :1 90 9. 08 09 5v 1 [ cs .S I] 6 S ep 2 01 9 of sentences, and topic detection methods in order to characterise how media outlets get involved in the election coverage. The results of the analysis were compared to the outcomes of political polls in order to know which of these two aspects of the information have more influence and if there exists any causal relationship between them. Our results suggest that the sentiment content of the news by itself is not enough to understand the differences in the polls between both candidates (validating the popular quote “There is no such thing as bad publicity”) but the topic coverage distribution does and the sentiment content discriminated by topic is consistent with these results. In particular, we found that one topic (news related to Clinton’s email scandal) is the more relevant in this analysis because it correlates negatively and shows a significant causal relation with the difference between Clinton and Trump in the polls and also explain the negative values of sentiment analysis in Fox News coverage.
The Disruptions of 5G on Data-Driven Technologies and Applications

With 5G on the verge of being adopted as the next mobile network, there is a need to analyze its impact on the landscape of computing and data management. In this paper, we analyze the impact of 5G on both traditional and emerging technologies and project our view on future research challenges and opportunities. With a predicted increase of 10-100x in bandwidth and 5-10x decrease in latency, 5G is expected to be the main enabler for smart cities, smart IoT and efficient healthcare, where machine learning is conducted at the edge. In this context, we investigate how 5G can help the development of federated learning. Network slicing, another key feature of 5G, allows running multiple isolated networks on the same physical infrastructure. However, security remains the main concern in the context of virtualization, multi-tenancy and high device density. Formal verification of 5G networks can be applied to detect security issues in massive virtualized environments. In summary, 5G will make the world even more densely and closely connected. What we have experienced in 4G connectivity will pale in comparison to the vast amounts of possibilities engendered by 5G.
Ensemble Knowledge Distillation for Learning Improved and Efficient Networks

Ensemble models comprising of deep Convolutional Neural Networks (CNN) have shown significant improvements in model generalization but at the cost of large computation and memory requirements. In this paper, we present a framework for learning compact CNN models with improved classification performance and model generalization. For this, we propose a CNN architecture of a compact student model with parallel branches which are trained using ground truth labels and information from high capacity teacher networks in an ensemble learning fashion. Our framework provides two main benefits: i) Distilling knowledge from different teachers into the student network promotes heterogeneity in feature learning at different branches of the student network and enables the network to learn diverse solutions to the target problem. ii) Coupling the branches of the student network through ensembling encourages collaboration and improves the quality of the final predictions by reducing variance in the network outputs. Experiments on the well established CIFAR-10 and CIFAR-100 datasets show that our Ensemble Knowledge Distillation (EKD) improves classification accuracy and model generalization especially in situations with limited training data. Experiments also show that our EKD based compact networks outperform in terms of mean accuracy on the test datasets compared to state-of-the-art knowledge distillation based methods.
Split Deep Q-Learning for Robust Object Singulation*

Extracting a known target object from a pile of other objects in a cluttered environment is a challenging robotic manipulation task encountered in many robotic applications. In such conditions, the target object touches or is covered by adjacent obstacle objects, thus rendering traditional grasping techniques ineffective. In this paper, we propose a pushing policy aiming at singulating the target object from its surrounding clutter, by means of lateral pushing movements of both the neighboring objects and the target object until sufficient ’grasping room’ has been achieved. To achieve the above goal we employ reinforcement learning and particularly Deep Qlearning (DQN) to learn optimal push policies by trial and error. A novel Split DQN is proposed to improve the learning rate and increase the modularity of the algorithm. Experiments show that although learning is performed in a simulated environment the transfer of learned policies to a real environment is effective thanks to robust feature selection. Finally, we demonstrate that the modularity of the algorithm allows the addition of extra primitives without retraining the model from scratch.
Spherical View Synthesis for Self-Supervised 360° Depth Estimation

Learning based approaches for depth perception are limited by the availability of clean training data. This has led to the utilization of view synthesis as an indirect objective for learning depth estimation using efficient data acquisition procedures. Nonetheless, most research focuses on pinhole based monocular vision, with scarce works presenting results for omnidirectional input. In this work, we explore spherical view synthesis for learning monocular 360 depth in a self-supervised manner and demonstrate its feasibility. Under a purely geometrically derived formulation we present results for horizontal and vertical baselines, as well as for the trinocular case. Further, we show how to better exploit the expressiveness of traditional CNNs when applied to the equirectangular domain in an efficient manner. Finally, given the availability of ground truth depth data, our work is uniquely positioned to compare view synthesis against direct supervision in a consistent and fair manner. The results indicate that alternative research directions might be better suited to enable higher quality depth perception. Our data, models and code are publicly available at https://vcl3d.github.io/SphericalViewSynthesis/.
The Explanation Game: Explaining Machine Learning Models with Cooperative Game Theory

A number of techniques have been proposed to explain a machine learning (ML) model's prediction by attributing it to the corresponding input features. Popular among these are techniques that apply the Shapley value method from cooperative game theory. While existing papers focus on the axiomatic motivation of Shapley values, and efficient techniques for computing them, they neither justify the game formulations used nor address the uncertainty implicit in their methods' outputs. For instance, the SHAP algorithm's formulation may give substantial attributions to features that play no role in a model. Furthermore, without infinite data and computation, SHAP attributions are approximations subject to hitherto uncharacterized uncertainty. In this work, we illustrate how subtle differences in the underlying game formulations of existing methods can cause large differences in attribution for a prediction. We then present a general game formulation that unifies existing methods. Using the primitive of single-reference games, we decompose the Shapley values of the general game formulation into Shapley values of single-reference games. This decomposition enables us to introduce confidence intervals to quantify the uncertainty in estimated attributions. Additionally, this decomposition enables contrastive explanations of a prediction through comparisons with different groups of reference inputs. We tie this idea to classic work on Norm Theory in cognitive psychology, and propose a general framework for generating explanations for ML models, called formulate, approximate, and explain (FAE).