OpenRaiser/Intern-Atlas huggingface dataset:
Are All Languages Created Equal in Multilingual BERT?
Venue: arXiv cs.CL 2020
Year: 2020

Multilingual BERT (mBERT) trained on 104 languages has shown surprisingly good cross-lingual performance on several NLP tasks, even without explicit cross-lingual signals. However, these evaluations have focused on cross-lingual transfer with high-resource languages, covering only a third of the languages covered by mBERT. We explore how mBERT performs on a much wider set of languages, focusing on the quality of representation for low-resource languages, measured by within-language performance. We consider three tasks: Named Entity Recognition (99 languages), Part-of-speech Tagging, and Dependency Parsing (54 languages each). mBERT does better than or comparable to baselines on high resource languages but does much worse for low resource languages. Furthermore, monolingual BERT models for these languages do even worse. Paired with similar languages, the performance gap between monolingual BERT and mBERT can be narrowed. We find that better models for low resource languages require more efficient pretraining techniques or more data.
(Re)construing Meaning in NLP
Venue: arXiv cs.CL 2020
Year: 2020

Human speakers have an extensive toolkit of ways to express themselves. In this paper, we engage with an idea largely absent from discussions of meaning in natural language understanding--namely, that the way something is expressed reflects different ways of conceptualizing or construing the information being conveyed. We first define this phenomenon more precisely, drawing on considerable prior work in theoretical cognitive semantics and psycholinguistics. We then survey some dimensions of construed meaning and show how insights from construal could inform theoretical and practical work in NLP.
GPT-too: A language-model-first approach for AMR-to-text generation
Venue: arXiv cs.CL 2020
Year: 2020

Meaning Representations (AMRs) are broad-coverage sentence-level semantic graphs. Existing approaches to generating text from AMR have focused on training sequence-to-sequence or graph-to-sequence models on AMR annotated data only. In this paper, we propose an alternative approach that combines a strong pre-trained language model with cycle consistency-based re-scoring. Despite the simplicity of the approach, our experimental results show these models outperform all previous techniques on the English LDC2017T10dataset, including the recent use of transformer architectures. In addition to the standard evaluation metrics, we provide human evaluation experiments that further substantiate the strength of our approach.
Riemannian Proximal Policy Optimization
Venue: arXiv cs.LG 2020
Year: 2020

In this paper, We propose a general Riemannian proximal optimization algorithm with guaranteed convergence to solve Markov decision process (MDP) problems. To model policy functions in MDP, we employ Gaussian mixture model (GMM) and formulate it as a nonconvex optimization problem in the Riemannian space of positive semidefinite matrices. For two given policy functions, we also provide its lower bound on policy improvement by using bounds derived from the Wasserstein distance of GMMs. Preliminary experiments show the efficacy of our proposed Riemannian proximal policy optimization algorithm.
Two-Dimensional Semi-Nonnegative Matrix Factorization for Clustering
Venue: arXiv cs.LG 2020
Year: 2020

In this paper, we propose a new Semi-Nonnegative Matrix Factorization method for 2-dimensional (2D) data, named TS-NMF. It overcomes the drawback of existing methods that seriously damage the spatial information of the data by converting 2D data to vectors in a preprocessing step. In particular, projection matrices are sought under the guidance of building new data representations, such that the spatial information is retained and projections are enhanced by the goal of clustering, which helps construct optimal projection directions. Moreover, to exploit nonlinear structures of the data, manifold is constructed in the projected subspace, which is adaptively updated according to the projections and less afflicted with noise and outliers of the data and thus more representative in the projected space. Hence, seeking projections, building new data representations, and learning manifold are seamlessly integrated in a single model, which mutually enhance other and lead to a powerful data representation. Comprehensive experimental results verify the effectiveness of TS-NMF in comparison with several state-of-the-art algorithms, which suggests high potential of the proposed method for real world applications.
A Lite Microphone Array Beamforming Scheme with Maximum Signal-to-Noise Ratio Filter
Venue: arXiv cs.SD 2020
Year: 2020

Since space-domain information can be utilized, microphone array beamforming is often used to enhance the quality of the speech by suppressing directional disturbance. However, with the increasing number of microphone, the complexity would be increased. In this paper, a concise beamforming scheme using Maximum Signal-to-Noise Ratio (SNR) filter is proposed to reduce the beamforming complexity. The maximum SNR filter is implemented by using the estimated direction-of-arrival (DOA) of the speech source localization (SSL) and the solving method of independent vector analysis (IVA). Our experiments show that when compared with other widely-used algorithms, the proposed algorithm obtain higher gain of signal-to-interference and noise ratio (SINR).
Human-Perception-Oriented Pseudo Analog Video Transmissions with Deep Learning
Venue: arXiv cs.MM 2020
Year: 2020

Recently, pseudo analog transmission has gained increasing attentions due to its ability to alleviate the cliff effect in video multicast scenarios. The existing pseudo analog systems are sorely optimized under the minimum mean squared error criterion without taking the perceptual video quality into consideration. In this paper, we propose a human-perception-based pseudo analog video transmission system named ROIC-Cast, which aims to intelligently enhance the transmission quality of the region-of-interest (ROI) parts. Firstly, the classic deep learning based saliency detection algorithm is adopted to decompose the continuous video sequences into ROI and non-ROI blocks. Secondly, an effective compression method is used to reduce the data amount of side information generated by the ROI extraction module. Then, the power allocation scheme is formulated as a convex problem, and the optimal transmission power for both ROI and non-ROI blocks is derived in a closed form. Finally, the simulations are conducted to validate the proposed system by comparing with a few of existing systems, e.g., KMV-Cast, SoftCast, and DAC-RAN. The proposed ROIC-Cast can achieve over 4.1dB peak signal- to-noise ratio gains of ROI compared with other systems, given the channel signal-to-noise ratio as -5dB, 0dB, 5dB, and 10dB, respectively. This significant performance improvement is due to the automatic ROI extraction, high-efficiency data compression as well as adaptive power allocation.
Combining Dynamic Symbolic Execution, Machine Learning and Search-Based Testing to Automatically Generate Test Cases for Classes
Venue: arXiv cs.SE 2020
Year: 2020

This article discusses a new technique to automatically generate test cases for object oriented programs. At the state of the art, the problem of generating adequate sets of complete test cases has not been satisfactorily solved yet. There are various techniques to automatically generate test cases (random testing, search-based testing, etc.) but each one has its own weaknesses. This article proposes an approach that distinctively combines dynamic symbolic execution, search-based testing and machine learning, to efficiently generate thorough class-level test suites. The preliminary data obtained carrying out some experiments confirm that we are going in the right direction.
Staying True to Your Word: (How) Can Attention Become Explanation?
Venue: arXiv cs.CL 2020
Year: 2020

The attention mechanism has quickly become ubiquitous in NLP. In addition to improving performance of models, attention has been widely used as a glimpse into the inner workings of NLP models. The latter aspect has in the recent years become a common topic of discussion, most notably in work of Jain and Wallace, 2019; Wiegreffe and Pinter, 2019. With the shortcomings of using attention weights as a tool of transparency revealed, the attention mechanism has been stuck in a limbo without concrete proof when and whether it can be used as an explanation. In this paper, we provide an explanation as to why attention has seen rightful critique when used with recurrent networks in sequence classification tasks. We propose a remedy to these issues in the form of a word level objective and our findings give credibility for attention to provide faithful interpretations of recurrent models.
Open Access effect on uncitedness: A large-scale study controlling by discipline, source type and visibility
Venue: arXiv cs.DL 2020
Year: 2020

There are many factors that affect the probability of being uncited during the first years after publication. In this study, we analyze three of these factors for journals, conference proceedings and book series: the field (in 316 subject categories of the Scopus database), the access modality (open access vs. paywalled), and the visibility of the source (through the percentile of the average impact in the subject category). We quantify the effect of these factors on the probability of being uncited. This probability is measured through the percentage of uncited documents in the serial sources of the Scopus database at about two years after publication. As a main result, we do not find any strong correlation between open access and uncitedness. Within the group of most cited journals (Q1 and top 10%), open access journals generally have somewhat lower uncited rates. However, in the intermediate quartiles (Q2 and Q3) almost no differences are observed, while for Q4 the uncited rate is again somewhat lower in the case of the OA group. This is important because it provides new evidence in the debate about open access citation advantage.
On the Choice of Auxiliary Languages for Improved Sequence Tagging
Venue: arXiv cs.CL 2020
Year: 2020

Recent work showed that embeddings from related languages can improve the performance of sequence tagging, even for monolingual models. In this analysis paper, we investigate whether the best auxiliary language can be predicted based on language distances and show that the most related language is not always the best auxiliary language. Further, we show that attention-based meta-embeddings can effectively combine pre-trained embeddings from different languages for sequence tagging and set new state-of-the-art results for part-of-speech tagging in five languages.
Pass-Fail Criteria for Scenario-Based Testing of Automated Driving Systems
Venue: arXiv cs.RO 2020
Year: 2020

The MUSICC project has created a proof-of-concept scenario database to be used as part of a type approval process for the verification of automated driving systems (ADS). This process must include a highly automated means of evaluating test results, as manual review at the scale required is impractical. This paper sets out a framework for assessing an ADS's behavioural safety in normal operation (i.e. performance of the dynamic driving task without component failures or malicious actions). Five top-level evaluation criteria for ADS performance are identified. Implementing these requires two types of outcome scoring rule: prescriptive (measurable rules which must always be followed) and risk-based (undesirable outcomes which must not occur too often). Scoring rules are defined in a programming language and will be stored as part of the scenario description. Risk-based rules cannot give a pass/fail decision from a single test case. Instead, a framework is defined to reach a decision for each functional scenario (set of test cases with common features). This considers statistical performance across many individual tests. Implications of this framework for hypothesis testing and scenario selection are identified.
On the Sixth International Olympiad in Cryptography NSUCRYPTO
Venue: arXiv cs.CR 2020
Year: 2020

NSUCRYPTO is the unique cryptographic Olympiad containing scientific mathematical problems for professionals, school and university students from any country. Its aim is to involve young researchers in solving curious and tough scientific problems of modern cryptography. From the very beginning, the concept of the Olympiad was not to focus on solving olympic tasks but on including unsolved research problems at the intersection of mathematics and cryptography. The Olympiad history starts in 2014. In 2019, it was held for the sixth time. In this paper, problems and their solutions of the Sixth International Olympiad in cryptography NSUCRYPTO'2019 are presented. We consider problems related to attacks on ciphers and hash functions, protocols, Boolean functions, Dickson polynomials, prime numbers, rotor machines, etc. We discuss several open problems on mathematical countermeasures to side-channel attacks, APN involutions, S-boxes, etc. The problem of finding a collision for the hash function Curl27 was partially solved during the Olympiad.
Continuous LWE
Venue: arXiv cs.CC 2020
Year: 2020

We introduce a continuous analogue of the Learning with Errors (LWE) problem, which we name CLWE. We give a polynomial-time quantum reduction from worst-case lattice problems to CLWE, showing that CLWE enjoys similar hardness guarantees to those of LWE. Alternatively, our result can also be seen as opening new avenues of (quantum) attacks on lattice problems. Our work resolves an open problem regarding the computational complexity of learning mixtures of Gaussians without separability assumptions (Diakonikolas 2016, Moitra 2018). As an additional motivation, (a slight variant of) CLWE was considered in the context of robust machine learning (Diakonikolas et al.~FOCS 2017), where hardness in the statistical query (SQ) model was shown; our work addresses the open question regarding its computational hardness (Bubeck et al.~ICML 2019).
Asymmetric scale functions for $t$-digests
Venue: arXiv cs.DS 2020
Year: 2020

The $t$-digest is a data structure that can be queried for approximate quantiles, with greater accuracy near the minimum and maximum of the distribution. We develop a $t$-digest variant with accuracy asymmetric about the median, thereby making possible alternative tradeoffs between computational resources and accuracy which may be of particular interest for distributions with significant skew. After establishing some theoretical properties of scale functions for $t$-digests, we show that a tangent line construction on the familiar scale functions preserves the crucial properties that allow $t$-digests to operate online and be mergeable. We conclude with an empirical study demonstrating the asymmetric variant preserves accuracy on one side of the distribution with a much smaller memory footprint.
Automated Copper Alloy Grain Size Evaluation Using a Deep-learning CNN
Venue: arXiv cs.CV 2020
Year: 2020

Moog Inc. has automated the evaluation of copper (Cu) alloy grain size using a deep-learning convolutional neural network (CNN). The proof-of-concept automated image acquisition and batch-wise image processing offers the potential for significantly reduced labor, improved accuracy of grain evaluation, and decreased overall turnaround times for approving Cu alloy bar stock for use in flight critical aircraft hardware. A classification accuracy of 91.1% on individual sub-images of the Cu alloy coupons was achieved. Process development included minimizing the variation in acquired image color, brightness, and resolution to create a dataset with 12300 sub-images, and then optimizing the CNN hyperparameters on this dataset using statistical design of experiments (DoE). Over the development of the automated Cu alloy grain size evaluation, a degree of "explainability" in the artificial intelligence (XAI) output was realized, based on the decomposition of the large raw images into many smaller dataset sub-images, through the ability to explain the CNN ensemble image output via inspection of the classification results from the individual smaller sub-images.
Mmwave Beam Management in Urban Vehicular Networks
Venue: arXiv cs.NI 2020
Year: 2020

Millimeter-wave (mmwave) communication represents a potential solution to capacity shortage in vehicular networks. However, effective beam alignment between senders and receivers requires accurate knowledge of the vehicles' position for fast beam steering, which is often impractical to obtain in real time. We address this problem by leveraging the traffic signals regulating vehicular mobility: as an example, we may coordinate beams with red traffic lights, as they correspond to higher vehicle densities and lower speeds. To evaluate our intuition, we propose a tractable, yet accurate, mmwave communication model accounting for both the distance and the heading of vehicles being served. Using such a model, we optimize the beam design and define a low-complexity, heuristic strategy. For increased realism, we consider as reference scenario a large-scale, real-world mobility trace of vehicles in Luxembourg. The results show that our approach closely matches the optimum and always outperforms static beam design based on road topology alone. Remarkably, it also yields better performance than solutions based on real-time mobility information.
The Virtual Block Interface: A Flexible Alternative to the Conventional Virtual Memory Framework
Venue: arXiv cs.AR 2020
Year: 2020

Computers continue to diversify with respect to system designs, emerging memory technologies, and application memory demands. Unfortunately, continually adapting the conventional virtual memory framework to each possible system configuration is challenging, and often results in performance loss or requires non-trivial workarounds. To address these challenges, we propose a new virtual memory framework, the Virtual Block Interface (VBI). We design VBI based on the key idea that delegating memory management duties to hardware can reduce the overheads and software complexity associated with virtual memory. VBI introduces a set of variable-sized virtual blocks (VBs) to applications. Each VB is a contiguous region of the globally-visible VBI address space, and an application can allocate each semantically meaningful unit of information (e.g., a data structure) in a separate VB. VBI decouples access protection from memory allocation and address translation. While the OS controls which programs have access to which VBs, dedicated hardware in the memory controller manages the physical memory allocation and address translation of the VBs. This approach enables several architectural optimizations to (1) efficiently and flexibly cater to different and increasingly diverse system configurations, and (2) eliminate key inefficiencies of conventional virtual memory. We demonstrate the benefits of VBI with two important use cases: (1) reducing the overheads of address translation (for both native execution and virtual machine environments), as VBI reduces the number of translation requests and associated memory accesses; and (2) two heterogeneous main memory architectures, where VBI increases the effectiveness of managing fast memory regions. For both cases, VBI significanttly improves performance over conventional virtual memory.
SwarmControl: An Automated Distributed Control Framework for Self-Optimizing Drone Networks
Venue: arXiv cs.NI 2020
Year: 2020

Networks of Unmanned Aerial Vehicles (UAVs), composed of hundreds, possibly thousands of highly mobile and wirelessly connected flying drones will play a vital role in future Internet of Things (IoT) and 5G networks. However, how to control UAV networks in an automated and scalable fashion in distributed, interference-prone, and potentially adversarial environments is still an open research problem. This article introduces SwarmControl, a new software-defined control framework for UAV wireless networks based on distributed optimization principles. In essence, SwarmControl provides the Network Operator (NO) with a unified centralized abstraction of the networking and flight control functionalities. High-level control directives are then automatically decomposed and converted into distributed network control actions that are executed through programmable software-radio protocol stacks. SwarmControl (i) constructs a network control problem representation of the directives of the NO; (ii) decomposes it into a set of distributed sub-problems; and (iii) automatically generates numerical solution algorithms to be executed at individual UAVs. We present a prototype of an SDR-based, fully reconfigurable UAV network platform that implements the proposed control framework, based on which we assess the effectiveness and flexibility of SwarmControl with extensive flight experiments. Results indicate that the SwarmControl framework enables swift reconfiguration of the network control functionalities, and it can achieve an average throughput gain of 159% compared to the state-of-the-art solutions.
Coverage Analysis of Net Inscriptions in Coloured Petri Net Models
Venue: arXiv cs.SE 2020
Year: 2020

High-level Petri net such as Coloured Petri Nets (CPNs) are characterised by the combination of Petri nets and a high-level programming language. In the context of CPNs and CPN Tools, the inscriptions (e.g., arc expressions and guards) are specified using Standard ML (SML). The application of simulation and state space exploration (SSE) for validating CPN models traditionally focuses on behavioural properties related to net structure, i.e., places and transitions. This means that the net inscriptions are only implicitly validated, and the extent to which these have been covered is not made explicit. The contribution of this paper is an approach that establishes a link between coverage analysis known from programming languages and net inscriptions of CPN models. Specifically, we consider Modified Condition/Decision Coverage (MC/DC) which generalises branch coverage of SML decisions. We have implemented our approach in a library for CPN Tools comprised of an annotation and instrumentation mechanism that transparently intercept and collect evaluations of Boolean conditions, and a post-processing tool that determines whether each decision is MC/DC-covered by a set of models executions (runs). We evaluate our approach on four larger public-available CPN models.
Neural Ordinary Differential Equation based Recurrent Neural Network Model
Venue: arXiv cs.LG 2020
Year: 2020

Neural differential equations are a promising new member in the neural network family. They show the potential of differential equations for time series data analysis. In this paper, the strength of the ordinary differential equation (ODE) is explored with a new extension. The main goal of this work is to answer the following questions: (i)~can ODE be used to redefine the existing neural network model? (ii)~can Neural ODEs solve the irregular sampling rate challenge of existing neural network models for a continuous time series, i.e., length and dynamic nature, (iii)~how to reduce the training and evaluation time of existing Neural ODE systems? This work leverages the mathematical foundation of ODEs to redesign traditional RNNs such as Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU). The main contribution of this paper is to illustrate the design of two new ODE-based RNN models (GRU-ODE model and LSTM-ODE) which can compute the hidden state and cell state at any point of time using an ODE solver. These models reduce the computation overhead of hidden state and cell state by a vast amount. The performance evaluation of these two new models for learning continuous time series with irregular sampling rate is then demonstrated. Experiments show that these new ODE based RNN models require less training time than Latent ODEs and conventional Neural ODEs. They can achieve higher accuracy quickly, and the design of the neural network is simpler than, previous neural ODE systems.