nils-herrmann/clean_scirepeval huggingface dataset:
$\texttt{DeepSqueeze}$: Decentralization Meets Error-Compensated Compression

Communication is a key bottleneck in distributed training. Recently, an \emph{error-compensated} compression technology was particularly designed for the \emph{centralized} learning and receives huge successes, by showing significant advantages over state-of-the-art compression based methods in saving the communication cost. Since the \emph{decentralized} training has been witnessed to be superior to the traditional \emph{centralized} training in the communication restricted scenario, therefore a natural question to ask is "how to apply the error-compensated technology to the decentralized learning to further reduce the communication cost." However, a trivial extension of compression based centralized training algorithms does not exist for the decentralized scenario. key difference between centralized and decentralized training makes this extension extremely non-trivial. In this paper, we propose an elegant algorithmic design to employ error-compensated stochastic gradient descent for the decentralized scenario, named $\texttt{DeepSqueeze}$. Both the theoretical analysis and the empirical study are provided to show the proposed $\texttt{DeepSqueeze}$ algorithm outperforms the existing compression based decentralized learning algorithms. To the best of our knowledge, this is the first time to apply the error-compensated compression to the decentralized learning.
Fake News Detection as Natural Language Inference

This report describes the entry by the Intelligent Knowledge Management (IKM) Lab in the WSDM 2019 Fake News Classification challenge. We treat the task as natural language inference (NLI). We individually train a number of the strongest NLI models as well as BERT. We ensemble these results and retrain with noisy labels in two stages. We analyze transitivity relations in the train and test sets and determine a set of test cases that can be reliably classified on this basis. The remainder of test cases are classified by our ensemble. Our entry achieves test set accuracy of 88.063% for 3rd place in the competition.
Probing Neural Network Comprehension of Natural Language Arguments

We are surprised to find that BERT’s peak performance of 77% on the Argument Reasoning Comprehension Task reaches just three points below the average untrained human baseline. However, we show that this result is entirely accounted for by exploitation of spurious statistical cues in the dataset. We analyze the nature of these cues and demonstrate that a range of models all exploit them. This analysis informs the construction of an adversarial dataset on which all models achieve random accuracy. Our adversarial dataset provides a more robust assessment of argument comprehension and should be adopted as the standard in future work.
A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI

Recently, artificial intelligence and machine learning in general have demonstrated remarkable performances in many tasks, from image processing to natural language processing, especially with the advent of deep learning (DL). Along with research progress, they have encroached upon many different fields and disciplines. Some of them require high level of accountability and thus transparency, for example, the medical sector. Explanations for machine decisions and predictions are thus needed to justify their reliability. This requires greater interpretability, which often means we need to understand the mechanism underlying the algorithms. Unfortunately, the blackbox nature of the DL is still unresolved, and many machine decisions are still poorly understood. We provide a review on interpretabilities suggested by different research works and categorize them. The different categories show different dimensions in interpretability research, from approaches that provide “obviously” interpretable information to the studies of complex patterns. By applying the same categorization to interpretability in medical research, it is hoped that: 1) clinicians and practitioners can subsequently approach these methods with caution; 2) insight into interpretability will be born with more considerations for medical practices; and 3) initiatives to push forward data-based, mathematically grounded, and technically grounded medical education are encouraged.
CLaRO: a Data-driven CNL for Specifying Competency Questions

Competency Questions (CQs) for an ontology and similar artefacts aim to provide insights into the contents of an ontology and to demarcate its scope. The absence of a controlled natural language, tooling and automation to support the authoring of CQs has hampered their effective use in ontology development and evaluation. The few question templates that exists are based on informal analyses of a small number of CQs and have limited coverage of question types and sentence constructions. We aim to fill this gap by proposing a template-based CNL to author CQs, called CLaRO. For its design, we exploited a new dataset of 234 CQs that had been processed automatically into 106 patterns, which we analysed and used to design a template-based CNL, with an additional CNL model and XML serialisation. The CNL was evaluated with a subset of questions from the original dataset and with two sets of newly sourced CQs. The coverage of CLaRO, with its 93 main templates and 41 linguistic variants, is about 90% for unseen questions. CLaRO has the potential to facilitate streamlining formalising ontology content requirements and, given that about one third of the competency questions in the test sets turned out to be invalid questions, assist in writing good questions.
Constructions and properties of a class of random scale-free networks

Complex networks have abundant and extensive applications in real life. Recently, researchers have proposed a large variety of complex networks, in which some are deterministic and others are random. The goal of this paper is to generate a class of random scale-free networks. To achieve this, we introduce three types of operations, i.e., rectangle operation, diamond operation, and triangle operation, and provide the concrete process for generating random scale-free networks N(p,q,r,t), where probability parameters p,q,r hold on p+q+r=1 with 0≤p,q,r≤1. We then discuss their topological properties, such as average degree, degree distribution, diameter, and clustering coefficient. First, we calculate the average degree of each member and discover that each member is a sparse graph. Second, by computing the degree distribution of our network N(p,q,r,t), we find that degree distribution obeys the power-law distribution, which implies that each member is scale-free. Next, according to our analysis of the diameter of our network N(p,q,r,t), we reveal the fact that the diameter may abruptly transform from small to large. Afterward, we give the calculation process of the clustering coefficient and discover that its value is mainly determined by r.
Block based Singular Value Decomposition approach to matrix factorization for recommender systems

With the abundance of data in recent years, interesting challenges are posed in the area of recommender systems. Producing high quality recommendations with scalability and performance is the need of the hour. Singular Value Decomposition(SVD) based recommendation algorithms have been leveraged to produce better results. In this paper, we extend the SVD technique further for scalability and performance in the context of 1) multi-threading 2) multiple computational units (with the use of Graphical Processing Units) and 3) distributed computation. We propose block based matrix factorization (BMF) paired with SVD. This enabled us to take advantage of SVD over basic matrix factorization(MF) while taking advantage of parallelism and scalability through BMF. We used Compute Unified Device Architecture (CUDA) platform and related hardware for leveraging Graphical Processing Unit (GPU) along with block based SVD to demonstrate the advantages in terms of performance and memory.
A Fourier approach to the inverse source problem in an absorbing and anisotropic scattering medium

We revisit the inverse source problem in a two dimensional absorbing and scattering medium and present a direct reconstruction method, which does not require iterative solvability of the forward problem, using measurements of the radiating flux at the boundary. The attenuation and scattering coefficients are known and the unknown source is isotropic. The approach is based on the Cauchy problem for a Beltrami-like equation for the sequence valued maps, and extends the original ideas of Bukhgeim from the non-scattering to the scattering media. We demonstrate the feasibility of the method in a numerical experiment in which the scattering is modeled by the two dimensional Henyey–Greenstein kernel with parameters meaningful in optical tomography.
Confidence Collapse in a Multi-Household, Self-Reflexive DSGE Model

We investigate a multi-household DSGE model in which past aggregate consumption impacts the confidence, and therefore consumption propensity, of individual households. We find that such a minimal setup is extremely rich, and leads to a variety of realistic output dynamics: high output with no crises; high output with increased volatility and deep, short lived recessions; alternation of high and low output states where relatively mild drop in economic conditions can lead to a temporary confidence collapse and steep decline in economic activity. The crisis probability depends exponentially on the parameters of the model, which means that markets cannot efficiently price the associated risk premium. We conclude by stressing that within our framework, narratives become an important monetary policy tool, that can help steering the economy back on track.
t-k-means: A k-means Variant with Robustness and Stability

Lloyd's $k$-means algorithm is one of the most classical clustering method, which is widely used in data mining or as a data pre-processing procedure. However, due to the thin-tailed property of the Gaussian distribution, $k$-means suffers from relatively poor performance on the heavy-tailed data or outliers. In addition, $k$-means have a relatively weak stability, $i.e.$ its result has a large variance, which reduces the credibility of the model. In this paper, we propose a robust and stable $k$-means variant, the $t$-$k$-means, as well as its fast version in solving the flat clustering problem. Theoretically, we detail the derivations of $t$-$k$-means and analyze its robustness and stability from the aspect of loss function, influence function and the expression of clustering center. A large number of experiments are conducted, which empirically demonstrates that our method has empirical soundness while preserving running efficiency.
Inhomogeneous domain walls in spintronic nanowires

In case of a spin-polarized current, the magnetization dynamics in nanowires are governed by the classical Landau–Lifschitz equation with Gilbert damping term, augmented by a typically non-variational Slonczewski term. Taking axial symmetry into account, we study the existence of domain wall type coherent structure solutions, with focus on one space dimension and spin-polarization, but our results also apply to vanishing spin-torque term. Using methods from bifurcation theory for arbitrary constant applied fields, we prove the existence of domain walls with non-trivial azimuthal profile, referred to as inhomogeneous. We present an apparently new type of domain wall, referred to as non-flat, whose approach of the axial magnetization has a certain oscillatory character. Additionally, we present the leading order mechanism for the parameter selection of flat and non-flat inhomogeneous domain walls for an applied field below a threshold, which depends on anisotropy, damping, and spin-transfer. Moreover, numerical continuation results of all these domain wall solutions are presented.
Computational Human Dynamics

This thesis summarises my scientific contributions in the domain of network science, human dynamics and computational social science. These contributions are associated to computer science, physics, statistics, and applied mathematics. The goal of this thesis is twofold, on one hand to write a concise summary of my most interesting scientific contributions, and on the other hand to provide an up-to-date view and perspective about my field. I start my dissertation with an introduction to position the reader on the landscape of my field and to put in perspective my contributions. In the second chapter I concentrate on my works on bursty human dynamics, addressing heterogeneous temporal characters of human actions and interactions. Next, I discuss my contributions to the field of temporal networks and give a synthesises of my works on various methods of the representation, characterisation, and modelling of time-varying structures. Finally, I discuss my works on the data-driven observations and modelling of collective social phenomena. There, I summarise studies on the static observations of emergent patterns of socioeconomic inequalities and their correlations with social-communication networks, and with linguistic patterns. I also discuss dynamic observations and modelling of social contagion processes.
Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming

The ability to detect objects regardless of image distortions or weather conditions is crucial for real-world applications of deep learning like autonomous driving. We here provide an easy-to-use benchmark to assess how object detection models perform when image quality degrades. The three resulting benchmark datasets, termed Pascal-C, Coco-C and Cityscapes-C, contain a large variety of image corruptions. We show that a range of standard object detection models suffer a severe performance loss on corrupted images (down to 30--60\% of the original performance). However, a simple data augmentation trick---stylizing the training images---leads to a substantial increase in robustness across corruption type, severity and dataset. We envision our comprehensive benchmark to track future progress towards building robust object detection models. Benchmark, code and data are publicly available.
The cyclicality of loan loss provisions under three different accounting models: the United Kingdom, Spain, and Brazil.

A controversy involving loan loss provisions in banks concerns their relationship with the business cycle. While international accounting standards for recognizing provisions (incurred loss model) would presumably be pro-cyclical, accentuating the effects of the current economic cycle, an alternative model, the expected loss model, has countercyclical characteristics, acting as a buffer against economic imbalances caused by expansionary or contractionary phases in the economy. In Brazil, a mixed accounting model exists, whose behavior is not known to be pro-cyclical or countercyclical. The aim of this research is to analyze the behavior of these accounting models in relation to the business cycle, using an econometric model consisting of financial and macroeconomic variables. The study allowed us to identify the impact of credit risk behavior, earnings management, capital management, Gross Domestic Product (GDP) behavior, and the behavior of the unemployment rate on provisions in countries that use different accounting models. Data from commercial banks in the United Kingdom (incurred loss), in Spain (expected loss), and in Brazil (mixed model) were used, covering the period from 2001 to 2012. Despite the accounting models of the three countries being formed by very different rules regarding possible effects on the business cycles, the results revealed a pro-cyclical behavior of provisions in each country, indicating that when GDP grows, provisions tend to fall and vice versa. The results also revealed other factors influencing the behavior of loan loss provisions, such as earning management.
Improving heart rate variability measurements from consumer smartwatches with machine learning

The reactions of the human body to physical exercise, psychophysiological stress and heart diseases are reflected in heart rate variability (HRV). Thus, continuous monitoring of HRV can contribute to determining and predicting issues in well-being and mental health. HRV can be measured in everyday life by consumer wearable devices such as smartwatches which are easily accessible and affordable. However, they are arguably accurate due to the stability of the sensor. We hypothesize a systematic error which is related to the wearer movement. Our evidence builds upon explanatory and predictive modeling: we find a statistically significant correlation between error in HRV measurements and the wearer movement. We show that this error can be minimized by bringing into context additional available sensor information, such as accelerometer data. This work demonstrates our research-in-progress on how neural learning can minimize the error of such smartwatch HRV measurements.
Learning Variable Impedance Control for Contact Sensitive Tasks

Reinforcement learning algorithms have shown great success in solving different problems ranging from playing video games to robotics. However, they struggle to solve delicate robotic problems, especially those involving contact interactions. Though in principle a policy directly outputting joint torques should be able to learn to perform these tasks, in practice we see that it has difficulty to robustly solve the problem without any given structure in the action space. In this letter, we investigate how the choice of action space can give robust performance in presence of contact uncertainties. We propose learning a policy giving as output impedance and desired position in joint space and compare the performance of that approach to torque and position control under different contact uncertainties. Furthermore, we propose an additional reward term designed to regularize these variable impedance control policies, giving them interpretability and facilitating their transfer to real systems. We present extensive experiments in simulation of both floating and fixed-base systems in tasks involving contact uncertainties, as well as results for running the learned policies on a real system (accompanying videos can be seen here: https://youtu.be/AQuuQ-h4dBM).
Low-Shot Classification: A Comparison of Classical and Deep Transfer Machine Learning Approaches

Despite the recent success of deep transfer learning approaches in NLP, there is a lack of quantitative studies demonstrating the gains these models offer in low-shot text classification tasks over existing paradigms. Deep transfer learning approaches such as BERT and ULMFiT demonstrate that they can beat state-of-the-art results on larger datasets, however when one has only 100-1000 labelled examples per class, the choice of approach is less clear, with classical machine learning and deep transfer learning representing valid options. This paper compares the current best transfer learning approach with top classical machine learning approaches on a trinary sentiment classification task to assess the best paradigm. We find that BERT, representing the best of deep transfer learning, is the best performing approach, outperforming top classical machine learning algorithms by 9.7% on average when trained with 100 examples per class, narrowing to 1.8% at 1000 labels per class. We also show the robustness of deep transfer learning in moving across domains, where the maximum loss in accuracy is only 0.7% in similar domain tasks and 3.2% cross domain, compared to classical machine learning which loses up to 20.6%.
Inductive analysis of the Internet protocol TLS

Internet browsers use security protocols to protect sensitive messages. An inductive analysis of TLS (a descendant of SSL 3.0) has been performed using the theorem prover Isabelle. Proofs are based on higher-order logic and make no assumptions concerning beliefs of finiteness. All the obvious security goals can be proved; session resumption appears to be secure even if old session keys are compromised. The proofs suggest minor changes to simplify the analysis.
 TLS, even at an abstract level, is much more complicated than most protocols verified by researchers. Session keys are negotiated rather than distributed, and the protocol has many optional parts. Netherless, the resources needed to verify TLS are modest: six man-weeks of effort and three minutes of processor time.
Conversational Help for Task Completion and Feature Discovery in Personal Assistants

Intelligent Personal Assistants (IPAs) have become widely popular in recent times. Most of the commercial IPAs today support a wide range of skills including Alarms, Reminders, Weather Updates, Music, News, Factual Questioning-Answering, etc. The list grows every day, making it difficult to remember the command structures needed to execute various tasks. An IPA must have the ability to communicate information about supported skills and direct users towards the right commands needed to execute them. Users interact with personal assistants in natural language. A query is defined to be a Help Query if it seeks information about a personal assistant's capabilities, or asks for instructions to execute a task. In this paper, we propose an interactive system which identifies help queries and retrieves appropriate responses. Our system comprises of a C-BiLSTM based classifier, which is a fusion of Convolutional Neural Networks (CNN) and Bidirectional LSTM (BiLSTM) architectures, to detect help queries and a semantic Approximate Nearest Neighbours (ANN) module to map the query to an appropriate predefined response. Evaluation of our system on real-world queries from a commercial IPA and a detailed comparison with popular traditional machine learning and deep learning based models reveal that our system outperforms other approaches and returns relevant responses for help queries.
Deep Metric Learning With Alternating Projections Onto Feasible Sets

Minimizers of the typical distance metric learning loss functions can be considered as “feasible points” satisfying a set of constraints imposed by the training data. We reformulate distance metric learning problem as finding a feasible point of a constraint set where the embedding vectors of the training data satisfy desired intra-class and inter-class proximity. The feasible set induced by the constraint set is expressed as the intersection of the relaxed feasible sets which enforce the proximity constraints only for particular samples (a sample from each class) of the training data. Then, the feasible point problem is to be approximately solved by performing alternating projections onto those feasible sets. Such an approach introduces a regularization term and results in minimizing a typical loss function with a systematic batch set construction where these batches are constrained to contain the same sample from each class for a certain number of iterations. The proposed technique is applied with the well-accepted losses and evaluated on three popular benchmark datasets for image retrieval and clustering. Outperforming state-of-the-art, the proposed approach consistently improves the performance of the integrated loss functions with no additional computational cost.
The box dimensions of exceptional self-affine sets in $\mathbb{R}^3$

We study the box dimensions of self-affine sets in $\mathbb{R}^3$ which are generated by a finite collection of generalised permutation matrices. We obtain bounds for the dimensions which hold with very minimal assumptions and give rise to sharp results in many cases. There are many issues in extending the well-established planar theory to $\mathbb{R}^3$ including that the principal planar projections are (affine distortions of) self-affine sets with overlaps (rather than self-similar sets) and that the natural modified singular value function fails to be sub-multiplicative in general. We introduce several new techniques to deal with these issues and hopefully provide some insight into the challenges in extending the theory further.