nils-herrmann/clean_scirepeval huggingface dataset:
Improving Unsupervised Word-by-Word Translation with Language Model and Denoising Autoencoder

Unsupervised learning of cross-lingual word embedding offers elegant matching of words across languages, but has fundamental limitations in translating sentences. In this paper, we propose simple yet effective methods to improve word-by-word translation of cross-lingual embeddings, using only monolingual corpora but without any back-translation. We integrate a language model for context-aware search, and use a novel denoising autoencoder to handle reordering. Our system surpasses state-of-the-art unsupervised translation systems without costly iterative training. We also analyze the effect of vocabulary size and denoising type on the translation performance, which provides better understanding of learning the cross-lingual word embedding and its usage in translation.
Fine-grained quantum computational supremacy

(pp1089-1115) Tomoyuki Morimae and Suguru  Output probability distributions of several sub-universal quantum computing models cannot be classically efficiently sampled unless some unlikely consequences occur in classical complexity theory, such as the collapse of the polynomial-time hierarchy. These results, so called quantum supremacy, however, do not rule out possibilities of super-polynomial-time classical simulations. In this paper, we study ``fine-grained" version of quantum supremacy that excludes some exponential-time classical simulations. First, we focus on two sub-universal models, namely, the one-clean-qubit model (or the DQC1 model) and the HC1Q model. Assuming certain conjectures in fine-grained complexity theory, we show that for any a>0 output probability distributions of these models cannot be classically sampled within a constant multiplicative error and in 2^{(1-a)N+o(N)} time, where N is the number of qubits. Next, we consider universal quantum computing. For example, we consider quantum computing over Clifford and T gates, and show that under another fine-grained complexity conjecture, output probability distributions of Clifford-T quantum computing cannot be classically sampled in 2^{o(t)} time within a constant multiplicative error, where t is the number of T gates.
Calabi–Yau structures and special Lagrangian submanifolds of complexified symmetric spaces

By Bielawski, it is shown that there exist Calabi-Yau structures on the complexifications of symmetric spaces of compact type. In this paper, we describe the Calabi-Yau structures of the complexified symmetric spaces in terms of the Schwarz's theorem in detail. We consider the case where the Calabi-Yau structure arises from the Riemannian metric corresponding to the Stenzel metric. In the complexified symmetric spaces equipped with such a Calabi-Yau structure, we give constructions of special Lagrangian submanifolds of any phase which are invariant under the actions of symmetric subgroups of the isometry group of the original symmetric space of compact type.
Generalization in Deep Networks: The Role of Distance from Initialization

Why does training deep neural networks using stochastic gradient descent (SGD) result in a generalization error that does not worsen with the number of parameters in the network? To answer this question, we advocate a notion of effective model capacity that is dependent on {\em a given random initialization of the network} and not just the training algorithm and the data distribution. We provide empirical evidences that demonstrate that the model capacity of SGD-trained deep networks is in fact restricted through implicit regularization of {\em the $\ell_2$ distance from the initialization}. We also provide theoretical arguments that further highlight the need for initialization-dependent notions of model capacity. We leave as open questions how and why distance from initialization is regularized, and whether it is sufficient to explain generalization.
Vector representations of text data in deep learning

In this dissertation we report results of our research on dense distributed representations of text data. We propose two novel neural models for learning such representations. The first model learns representations at the document level, while the second model learns word-level representations. 
For document-level representations we propose Binary Paragraph Vector: a neural network models for learning binary representations of text documents, which can be used for fast document retrieval. We provide a thorough evaluation of these models and demonstrate that they outperform the seminal method in the field in the information retrieval task. We also report strong results in transfer learning settings, where our models are trained on a generic text corpus and then used to infer codes for documents from a domain-specific dataset. In contrast to previously proposed approaches, Binary Paragraph Vector models learn embeddings directly from raw text data. 
For word-level representations we propose Disambiguated Skip-gram: a neural network model for learning multi-sense word embeddings. Representations learned by this model can be used in downstream tasks, like part-of-speech tagging or identification of semantic relations. In the word sense induction task Disambiguated Skip-gram outperforms state-of-the-art models on three out of four benchmarks datasets. Our model has an elegant probabilistic interpretation. Furthermore, unlike previous models of this kind, it is differentiable with respect to all its parameters and can be trained with backpropagation. In addition to quantitative results, we present qualitative evaluation of Disambiguated Skip-gram, including two-dimensional visualisations of selected word-sense embeddings.
Tencent ML-Images: A Large-Scale Multi-Label Image Database for Visual Representation Learning

In existing visual representation learning tasks, deep convolutional neural networks (CNNs) are often trained on images annotated with single tag, such as ImageNet. However, single tag annotation cannot describe all important contents of one image, and some useful visual information may be wasted during training. In this work, we propose to train CNNs from images annotated with multiple tags, to enhance the quality of visual representation of the trained CNN model. To this end, we build a large-scale multi-label image database with 18M images and 11K categories, dubbed Tencent ML-Images. We efficiently train the ResNet-101 model with multi-label outputs on Tencent ML-Images, taking 90 hours for 60 epochs, based on a large-scale distributed deep learning framework, i.e., TFplus. The good quality of the visual representation of the Tencent ML-Images checkpoint is verified through three transfer learning tasks, including single-label image classification on ImageNet and Caltech-256, object detection on PASCAL VOC 2007, and semantic segmentation on PASCAL VOC 2012. The Tencent ML-Images database, the checkpoints of ResNet-101, and all the training codes have been released at https://github.com/Tencent/tencent-ml-images. It is expected to promote other vision tasks in the research and industry community.
Universal Deep Beamformer for Variable Rate Ultrasound Imaging

Ultrasound (US) imaging is based on the time-reversal principle, in which individual channel RF measurements are back-propagated and accumulated to form an image after applying specific delays. While this time reversal is usually implemented as a delay-and-sum (DAS) beamformer, the image quality quickly degrades as the number of measurement channels decreases. To address this problem, various types of adaptive beamforming techniques have been proposed using predefined models of the signals. However, the performance of these adaptive beamforming approaches degrade when the underlying model is not sufficiently accurate. Here, we demonstrate for the first time that a single universal deep beamformer trained using a purely data-driven way can generate significantly improved images over widely varying aperture and channel subsampling patterns. In particular, we design an end-to-end deep learning framework that can directly process sub-sampled RF data acquired at different subsampling rate and detector configuration to generate high quality ultrasound images using a single beamformer. Experimental results using B-mode focused ultrasound confirm the efficacy of the proposed methods.
Generative adversarial networks for financial trading strategies fine-tuning and combination

Systematic trading strategies are algorithmic procedures that allocate assets aiming to optimize a certain performance criterion. To obtain an edge in a highly competitive environment, an analyst needs to appropriately fine-tune their strategy, or discover how to combine weak signals in novel alpha creating manners. Both aspects, namely fine-tuning and combination, have been extensively researched using several methods, but emerging techniques such as Generative Adversarial Networks can have an impact on such aspects. Therefore, our work proposes the use of Conditional Generative Adversarial Networks (cGANs) for trading strategy calibration and aggregation. To this end, we provide a full methodology on: (i) the training and selection of a cGAN for time series data; (ii) how each sample is used for strategy calibration; and (iii) how all generated samples can be used for ensemble modelling. To provide evidence that our approach is well grounded, we have designed an experiment with multiple trading strategies, encompassing 579 assets. We compared cGAN with an ensemble scheme and model validation methods, both suited for time series. Our results suggest that cGANs are a suitable alternative for strategy calibration and combination, providing outperformance when the traditional techniques fail to generate any alpha.
Estimating physical properties from liquid crystal textures via machine learning and complexity-entropy methods.

Imaging techniques are essential tools for inquiring a number of properties from different materials. Liquid crystals are often investigated via optical and image processing methods. In spite of that, considerably less attention has been paid to the problem of extracting physical properties of liquid crystals directly from textures images of these materials. Here we present an approach that combines two physics-inspired image quantifiers (permutation entropy and statistical complexity) with machine learning techniques for extracting physical properties of nematic and cholesteric liquid crystals directly from their textures images. We demonstrate the usefulness and accuracy of our approach in a series of applications involving simulated and experimental textures, in which physical properties of these materials (namely: average order parameter, sample temperature, and cholesteric pitch length) are predicted with significant precision. Finally, we believe our approach can be useful in more complex liquid crystal experiments as well as for probing physical properties of other materials that are investigated via imaging techniques.
Dynamics, Radiation, and Overall Energy Budget of Earthquake Rupture With Coseismic Off‐Fault Damage

Earthquake ruptures dynamically activate coseismic off‐fault damage around fault cores. Systematic field observation efforts have shown the distribution of off‐fault damage around main faults, while numerical modeling using elastic‐plastic off‐fault material models have demonstrated the evolution of coseismic off‐fault damage during earthquake ruptures. Laboratory‐scale microearthquake experiments have pointed out the enhanced high‐frequency radiation due to the coseismic off‐fault damage. However, the detailed off‐fault fracturing mechanisms, subsequent radiation, and its contribution to the overall energy budget remain to be fully understood because of limitations of current observational techniques and model formulations. Here, we constructed a new physics‐based dynamic earthquake rupture modeling framework, based on the combined finite‐discrete element method, to investigate the fundamental mechanisms of coseismic off‐fault damage, and its effect on the rupture dynamics, the radiation and the overall energy budget. We conducted a 2‐D systematic case study with depth and showed the mechanisms of dynamic activation of the coseismic off‐fault damage. We found the decrease in rupture velocity and the enhanced high‐frequency radiation in near field due to the coseismic off‐fault damage. We then evaluated the overall energy budget, which shows a significant contribution of the coseismic off‐fault damage to the overall energy budget even at depth, where the damage zone width becomes narrower. The present numerical framework for the dynamic earthquake rupture modeling thus provides new insights into earthquake rupture dynamics with the coseismic off‐fault damage.
Minimal Higgs inflation with an 
R2
 term in Palatini gravity

It has recently been suggested that the Standard Model Higgs boson could act as the inflaton while minimally coupled to gravity - given that the gravity sector is extended with an $\alpha R^2$ term and the underlying theory of gravity is of Palatini, rather than metric, type. In this paper, we revisit the idea and correct some shortcomings in earlier studies. We find that in this setup the Higgs can indeed act as the inflaton and that the tree-level predictions of the model for the spectral index and the tensor-to-scalar ratio are $n_s\simeq 0.941$, $r\simeq 0.3/(1+10^{-8}\alpha)$, respectively, for a typical number of e-folds, $N=50$, between horizon exit of the pivot scale $k=0.05\, {\rm Mpc}^{-1}$ and the end of inflation. Even though the tensor-to-scalar ratio is suppressed compared to the usual minimally coupled case and can be made compatible with data for large enough $\alpha$, the result for $n_s$ is in severe tension with the Planck results. We briefly discuss extensions of the model.
Fusing Body Posture With Facial Expressions for Joint Recognition of Affect in Child–Robot Interaction

In this letter, we address the problem of multi-cue affect recognition in challenging scenarios such as child–robot interaction. Toward this goal we propose a method for automatic recognition of affect that leverages body expressions alongside facial ones, as opposed to traditional methods that typically focus only on the latter. Our deep-learning based method uses hierarchical multi-label annotations and multi-stage losses, can be trained both jointly and separately, and offers us computational models for both individual modalities, as well as for the whole body emotion. We evaluate our method on a challenging child–robot interaction database of emotional expressions collected by us, as well as on the GEneva multimodal emotion portrayal public database of acted emotions by adults, and show that the proposed method achieves significantly better results than facial-only expression baselines.
Wayeb: a Tool for Complex Event Forecasting

Complex Event Processing (CEP) systems have appeared in abundance during the last two decades. Their purpose is to detect in real-time interesting patterns upon a stream of events and to inform an analyst for the occurrence of such patterns in a timely manner. However, there is a lack of methods for forecasting when a pattern might occur before such an occurrence is actually detected by a CEP engine. We present Wayeb, a tool that attempts to address the issue of Complex Event Forecasting. Wayeb employs symbolic automata as a computational model for pattern detection and Markov chains for deriving a probabilistic description of a symbolic automaton.
Beyond socket options: making the Linux TCP stack truly extensible

The Transmission Control Protocol (TCP) is one of the most important protocols in today's Internet. Its specification and implementations have been refined for almost forty years. The Linux TCP stack is one of the most widely used TCP stacks given its utilisation on servers and Android smartphones and tablets. However, TCP and its implementations evolve very slowly. In this paper, we demonstrate how to leverage the eBPF virtual machine that is part of the recent versions of the Linux kernel to make the TCP stack easier to extend. We demonstrate a variety of use cases where the eBPF code is injected inside a running kernel to update or tune the TCP implementation. We first implement the TCP User Timeout Option. Then we propose a new option that enables a client to request a server to use a specific congestion control scheme. Our third extension is a TCP option that sets the initial congestion window. We then demonstrate how eBPF code can be used to tune the acknowledgment strategy.
Koji: Automating pipelines with mixed-semantics data sources

We propose a new result-oriented semantic for defining data processing workflows that manipulate data in different semantic forms (files or services) in a unified manner. This approach enables users to define workflows for a vast variety of reproducible data-processing tasks in a simple declarative manner which focuses on application-level results, while automating all control-plane considerations (like failure recovery without loss of progress and computation reuse) behind the scenes. 
The uniform treatment of files and services as data enables easy integration with existing data sources (e.g. data acquisition APIs) and sinks of data (e.g. database services). Whereas the focus on containers as transformations enables reuse of existing data-processing systems. 
We describe a declarative configuration mechanism, which can be viewed as an intermediate representation (IR) of reproducible data processing pipelines in the same spirit as, for instance, TensorFlow\cite{tensorflow} and ONNX\cite{onnx} utilize IRs for defining tensor-processing pipelines.
Stance Classification for Rumour Analysis in Twitter: Exploiting Affective Information and Conversation Structure

Analysing how people react to rumours associated with news in social media is an important task to prevent the spreading of misinformation, which is nowadays widely recognized as a dangerous tendency. In social media conversations, users show different stances and attitudes towards rumourous stories. Some users take a definite stance, supporting or denying the rumour at issue, while others just comment it, or ask for additional evidence related to the veracity of the rumour. On this line, a new shared task has been proposed at SemEval-2017 (Task 8, SubTask A), which is focused on rumour stance classification in English tweets. The goal is predicting user stance towards emerging rumours in Twitter, in terms of supporting, denying, querying, or commenting the original rumour, looking at the conversation threads originated by the rumour. This paper describes a new approach to this task, where the use of conversation-based and affective-based features, covering different facets of affect, has been explored. Our classification model outperforms the best-performing systems for stance classification at SemEval-2017 Task 8, showing the effectiveness of the feature set proposed.
Star formation rates and stellar masses from machine learning

Star-formation activity is a key property to probe the structure formation and hence characterise the large-scale structures of the universe. This information can be deduced from the star formation rate (SFR) and the stellar mass (M⋆), both of which, but especially the SFR, are very complex to estimate. Determining these quantities from UV, optical, or IR luminosities relies on complex modeling and on priors on galaxy types. We propose a method based on the machine-learning algorithm Random Forest to estimate the SFR and the M⋆ of galaxies at redshifts in the range 0.01 <  z <  0.3, independent of their type. The machine-learning algorithm takes as inputs the redshift, WISE luminosities, and WISE colours in near-IR, and is trained on spectra-extracted SFR and M⋆ from the SDSS MPA-JHU DR8 catalogue as outputs. We show that our algorithm can accurately estimate SFR and M⋆ with scatters of σSFR = 0.38 dex and σM⋆ = 0.16 dex for SFR and stellar mass, respectively, and that it is unbiased with respect to redshift or galaxy type. The full-sky coverage of the WISE satellite allows us to characterise the star-formation activity of all galaxies outside the Galactic mask with spectroscopic redshifts in the range 0.01 <  z <  0.3. The method can also be applied to photometric-redshift catalogues, with best scatters of σSFR = 0.42 dex and σM⋆ = 0.24 dex obtained in the redshift range 0.1 <  z <  0.3.
GASL: Guided Attention for Sparsity Learning in Deep Neural Networks

The main goal of network pruning is imposing sparsity on the neural network by increasing the number of parameters with zero value in order to reduce the architecture size and the computational speedup. In most of the previous research works, sparsity is imposed stochastically without considering any prior knowledge of the weights distribution or other internal network characteristics. Enforcing too much sparsity may induce accuracy drop due to the fact that a lot of important elements might have been eliminated. In this paper, we propose Guided Attention for Sparsity Learning (GASL) to achieve (1) model compression by having less number of elements and speed-up; (2) prevent the accuracy drop by supervising the sparsity operation via a guided attention mechanism and (3) introduce a generic mechanism that can be adapted for any type of architecture; Our work is aimed at providing a framework based on interpretable attention mechanisms for imposing structured and non-structured sparsity in deep neural networks. For Cifar-100 experiments, we achieved the state-of-the-art sparsity level and 2.91x speedup with competitive accuracy compared to the best method. For MNIST and LeNet architecture we also achieved the highest sparsity and speedup level.
Looking back at Postgres

This is a recollection of the UC Berkeley Postgres project, which was led by Mike Stonebraker from the mid-1980's to the mid-1990's. The article was solicited for Stonebraker's Turing Award book, as one of many personal/historical recollections. As a result it focuses on Stonebraker's design ideas and leadership. But Stonebraker was never a coder, and he stayed out of the way of his development team. The Postgres codebase was the work of a team of brilliant students and the occasional university "staff programmers" who had little more experience (and only slightly more compensation) than the students. I was lucky to join that team as a student during the latter years of the project. I got helpful input on this writeup from some of the more senior students on the project, but any errors or omissions are mine. If you spot any such, please contact me and I will try to fix them.
Accelerating Goal-Directed Reinforcement Learning by Model Characterization

We propose a hybrid approach aimed at improving the sample efficiency in goal-directed reinforcement learning. We do this via a two-step mechanism where firstly, we approximate a model from Model-Free reinforcement learning. Then, we leverage this approximate model along with a notion of reachability using Mean First Passage Times to perform Model-Based reinforcement learning. Built on such a novel observation, we design two new algorithms - Mean First Passage Time based Q-Learning (MFPT-Q)and Mean First Passage Time based DYNA (MFPT-DYNA), that have been fundamentally modified from the state-of-the-art reinforcement learning techniques. Preliminary results have shown that our hybrid approaches converge with much fewer iterations than their corresponding state-of-the-art counterparts and therefore requiring much fewer samples and much fewer training trials to converge.