OpenRaiser/Intern-Atlas huggingface dataset:
Do all Roads Lead to Rome? Understanding the Role of Initialization in Iterative Back-Translation
Venue: arXiv cs.CL 2020
Year: 2020

Back-translation provides a simple yet effective approach to exploit monolingual corpora in Neural Machine Translation (NMT). Its iterative variant, where two opposite NMT models are jointly trained by alternately using a synthetic parallel corpus generated by the reverse model, plays a central role in unsupervised machine translation. In order to start producing sound translations and provide a meaningful training signal to each other, existing approaches rely on either a separate machine translation system to warm up the iterative procedure, or some form of pre-training to initialize the weights of the model. In this paper, we analyze the role that such initialization plays in iterative back-translation. Is the behavior of the final system heavily dependent on it? Or does iterative back-translation converge to a similar solution given any reasonable initialization? Through a series of empirical experiments over a diverse set of warmup systems, we show that, although the quality of the initial system does affect final performance, its effect is relatively small, as iterative back-translation has a strong tendency to convergence to a similar solution. As such, the margin of improvement left for the initialization method is narrow, suggesting that future research should focus more on improving the iterative mechanism itself.
Galois hulls of MDS codes and their quantum error correction
Venue: arXiv cs.IT 2020
Year: 2020

The hull of linear codes plays an important role in quantum information and coding theory. In the present paper, by investigating the Galois hulls of generalized Reed-Solomon (GRS) codes and extended GRS codes over the finite field Fq, we give several new families of MDS codes with Galois hulls of arbitrary dimensions that are not obtained before. Some of them generalize the ones in the literature [13]. As a consequence, using these MDS codes with Galois hulls of arbitrary dimensions, we construct nine new families of MDS entanglement-assisted quantum error-correcting codes (EAQECCs) with flexible parameters.
Social Welfare and Price of Anarchy in Preemptive Priority Queues
Venue: arXiv cs.GT 2020
Year: 2020

Consider an unobservable $M|G|1$ queue with preemptive-resume scheduling and two priority classes. Customers are strategic and may join the premium class for a fee. We analyze the resulting equilibrium outcomes, equilibrium stability, and social welfare. We find that for service distributions with coefficient of variation greater than 1, there exists a unique and stable mixed equilibrium at low loads. We also establish a tight bound on the price of anarchy, which is $4/3$.
Cities as they could be: Artificial Life and Urban Systems
Venue: arXiv cs.CY 2020
Year: 2020

The metaphor of cities as organisms has a long history in urban planning, and a few urban modeling approaches have explicitly been linked to Artificial Life. We propose in that paper to explore the extent of Artificial Life and Artificial Intelligence application to urban issues, by constructing and exploring a citation network of around 225,000 papers. It shows that most of the literature is indeed application of methodologies and a rather strong modularity of approaches. We finally develop ALife concepts which have a strong potential for the development of new urban theories.
Distributed Momentum for Byzantine-resilient Learning
Venue: arXiv cs.LG 2020
Year: 2020

Momentum is a variant of gradient descent that has been proposed for its benefits on convergence. In a distributed setting, momentum can be implemented either at the server or the worker side. When the aggregation rule used by the server is linear, commutativity with addition makes both deployments equivalent. Robustness and privacy are however among motivations to abandon linear aggregation rules. In this work, we demonstrate the benefits on robustness of using momentum at the worker side. We first prove that computing momentum at the workers reduces the variance-norm ratio of the gradient estimation at the server, strengthening Byzantine resilient aggregation rules. We then provide an extensive experimental demonstration of the robustness effect of worker-side momentum on distributed SGD.
Learning Nonparametric Human Mesh Reconstruction from a Single Image without Ground Truth Meshes
Venue: arXiv cs.CV 2020
Year: 2020

Nonparametric approaches have shown promising results on reconstructing 3D human mesh from a single monocular image. Unlike previous approaches that use a parametric human model like skinned multi-person linear model (SMPL), and attempt to regress the model parameters, nonparametric approaches relax the heavy reliance on the parametric space. However, existing nonparametric methods require ground truth meshes as their regression target for each vertex, and obtaining ground truth mesh labels is very expensive. In this paper, we propose a novel approach to learn human mesh reconstruction without any ground truth meshes. This is made possible by introducing two new terms into the loss function of a graph convolutional neural network (Graph CNN). The first term is the Laplacian prior that acts as a regularizer on the reconstructed mesh. The second term is the part segmentation loss that forces the projected region of the reconstructed mesh to match the part segmentation. Experimental results on multiple public datasets show that without using 3D ground truth meshes, the proposed approach outperforms the previous state-of-the-art approaches that require ground truth meshes for training.
VAT: Asymptotic Cost Analysis for Multi-Level Key-Value Stores
Venue: arXiv cs.DC 2020
Year: 2020

Over the past years, there has been an increasing number of key-value (KV) store designs, each optimizing for a different set of requirements. Furthermore, with the advancements of storage technology the design space of KV stores has become even more complex. More recent KV-store designs target fast storage devices, such as SSDs and NVM. Most of these designs aim to reduce amplification during data reorganization by taking advantage of device characteristics. However, until today most analysis of KV-store designs is experimental and limited to specific design points. This makes it difficult to compare tradeoffs across different designs, find optimal configurations and guide future KV-store design. In this paper, we introduce the Variable Amplification- Throughput analysis (VAT) to calculate insert-path amplification and its impact on multi-level KV-store performance.We use VAT to express the behavior of several existing design points and to explore tradeoffs that are not possible or easy to measure experimentally. VAT indicates that by inserting randomness in the insert-path, KV stores can reduce amplification by more than 10x for fast storage devices. Techniques, such as key-value separation and tiering compaction, reduce amplification by 10x and 5x, respectively. Additionally, VAT predicts that the advancements in device technology towards NVM, reduces the benefits from both using key-value separation and tiering.
Self-supervised Representation Learning for Ultrasound Video
Venue: arXiv cs.CV 2020
Year: 2020

Recent advances in deep learning have achieved promising performance for medical image analysis, while in most cases ground-truth annotations from human experts are necessary to train the deep model. In practice, such annotations are expensive to collect and can be scarce for medical imaging applications. Therefore, there is significant interest in learning representations from unlabelled raw data. In this paper, we propose a self-supervised learning approach to learn meaningful and transferable representations from medical imaging video without any type of human annotation. We assume that in order to learn such a representation, the model should identify anatomical structures from the unlabelled data. Therefore we force the model to address anatomy-aware tasks with free supervision from the data itself. Specifically, the model is designed to correct the order of a reshuffled video clip and at the same time predict the geometric transformation applied to the video clip. Experiments on fetal ultrasound video show that the proposed approach can effectively learn meaningful and strong representations, which transfer well to downstream tasks like standard plane detection and saliency prediction.
Communication-Optimal Tilings for Projective Nested Loops with Arbitrary Bounds
Venue: arXiv cs.DS 2020
Year: 2020

Reducing communication - either between levels of a memory hierarchy or between processors over a network - is a key component of performance optimization (in both time and energy) for many problems, including dense linear algebra, particle interactions, and machine learning. For these problems, which can be represented as nested-loop computations, previous tiling based approaches have been used to find both lower bounds on the communication required to execute them and optimal rearrangements, or blockings, to attain such lower bounds. However, such general approaches have typically assumed the problem sizes are large, an assumption that is often not met in practice. For instance, the classical $(\text{# arithmetic operations})/(\text{cache size})^{1/2}$ lower bound for matrix multiplication is not tight for matrix-vector multiplications, which must read in at least $O(\text{# arithmetic operations})$ words of memory; similar issues occur for almost all convolutions in machine learning applications, which use extremely small filter sizes (and therefore, loop bounds). In this paper, we provide an efficient way to both find and obtain, via an appropriate, efficiently constructible blocking, communication lower bounds and matching tilings which attain these lower bounds for nested loop programs with arbitrary loop bounds that operate on multidimensional arrays in the projective case, where the array indices are subsets of the loop indices. Our approach works on all such problems, regardless of dimensionality, size, memory access patterns, or number of arrays, and directly applies to (among other examples) matrix multiplication and similar dense linear algebra operations, tensor contractions, n-body pairwise interactions, pointwise convolutions, and fully connected layers.
A Note on Latency Variability of Deep Neural Networks for Mobile Inference
Venue: arXiv cs.DC 2020
Year: 2020

Running deep neural network (DNN) inference on mobile devices, i.e., mobile inference, has become a growing trend, making inference less dependent on network connections and keeping private data locally. The prior studies on optimizing DNNs for mobile inference typically focus on the metric of average inference latency, thus implicitly assuming that mobile inference exhibits little latency variability. In this note, we conduct a preliminary measurement study on the latency variability of DNNs for mobile inference. We show that the inference latency variability can become quite significant in the presence of CPU resource contention. More interestingly, unlike the common belief that the relative performance superiority of DNNs on one device can carry over to another device and/or another level of resource contention, we highlight that a DNN model with a better latency performance than another model can become outperformed by the other model when resource contention be more severe or running on another device. Thus, when optimizing DNN models for mobile inference, only measuring the average latency may not be adequate; instead, latency variability under various conditions should be accounted for, including but not limited to different devices and different levels of CPU resource contention considered in this note.
Comparison of Distal Teacher Learning with Numerical and Analytical Methods to Solve Inverse Kinematics for Rigid-Body Mechanisms
Venue: arXiv cs.RO 2020
Year: 2020

Several publications are concerned with learning inverse kinematics, however, their evaluation is often limited and none of the proposed methods is of practical relevance for rigid-body kinematics with a known forward model. We argue that for rigid-body kinematics one of the first proposed machine learning (ML) solutions to inverse kinematics -- distal teaching (DT) -- is actually good enough when combined with differentiable programming libraries and we provide an extensive evaluation and comparison to analytical and numerical solutions. In particular, we analyze solve rate, accuracy, sample efficiency and scalability. Further, we study how DT handles joint limits, singularities, unreachable poses, trajectories and provide a comparison of execution times. The three approaches are evaluated on three different rigid body mechanisms with varying complexity. With enough training data and relaxed precision requirements, DT has a better solve rate and is faster than state-of-the-art numerical solvers for a 15-DoF mechanism. DT is not affected by singularities while numerical solutions are vulnerable to them. In all other cases numerical solutions are usually better. Analytical solutions outperform the other approaches by far if they are available.
Secure Polar Coding for Adversarial Wiretap Channel
Venue: arXiv cs.IT 2020
Year: 2020

The adversarial wiretap channel (AWTC) model is a secure communication model in which adversary can directly read and write the transmitted bits in legitimate communication with fixed fractions. In this paper we propose a secure polar coding scheme to provide secure and reliable communication over the AWTC model. For the adversarial reading and writing action, we present a $ρ$ equivalent channel block and study its transformation under the channel polarization operation. We find that the generated channels are polarized in the sense that part of them are full-noise channels with probability almost $1$ and the rest part of them are noiseless channels with probability almost $1$. Based on this result, we polarize both equivalent channel blocks of adversarial reading and writing, and then construct a secure polar coding scheme by applying the multi-block chaining structure on the polarized equivalent blocks. Theoretically we prove that when block length $N$ goes infinity, the proposed scheme achieves the secrecy capacity of the AWTC model under both reliability and strong security criterions. Then by simulations, we prove that the proposed scheme can provide secure and reliable communication over AWTC model.
An Evaluation of Knowledge Graph Embeddings for Autonomous Driving Data: Experience and Practice
Venue: arXiv cs.AI 2020
Year: 2020

The autonomous driving (AD) industry is exploring the use of knowledge graphs (KGs) to manage the vast amount of heterogeneous data generated from vehicular sensors. The various types of equipped sensors include video, LIDAR and RADAR. Scene understanding is an important topic in AD which requires consideration of various aspects of a scene, such as detected objects, events, time and location. Recent work on knowledge graph embeddings (KGEs) - an approach that facilitates neuro-symbolic fusion - has shown to improve the predictive performance of machine learning models. With the expectation that neuro-symbolic fusion through KGEs will improve scene understanding, this research explores the generation and evaluation of KGEs for autonomous driving data. We also present an investigation of the relationship between the level of informational detail in a KG and the quality of its derivative embeddings. By systematically evaluating KGEs along four dimensions -- i.e. quality metrics, KG informational detail, algorithms, and datasets -- we show that (1) higher levels of informational detail in KGs lead to higher quality embeddings, (2) type and relation semantics are better captured by the semantic transitional distance-based TransE algorithm, and (3) some metrics, such as coherence measure, may not be suitable for intrinsically evaluating KGEs in this domain. Additionally, we also present an (early) investigation of the usefulness of KGEs for two use-cases in the AD domain.
Budget-Constrained Bandits over General Cost and Reward Distributions
Venue: arXiv cs.LG 2020
Year: 2020

We consider a budget-constrained bandit problem where each arm pull incurs a random cost, and yields a random reward in return. The objective is to maximize the total expected reward under a budget constraint on the total cost. The model is general in the sense that it allows correlated and potentially heavy-tailed cost-reward pairs that can take on negative values as required by many applications. We show that if moments of order $(2+γ)$ for some $γ> 0$ exist for all cost-reward pairs, $O(\log B)$ regret is achievable for a budget $B>0$. In order to achieve tight regret bounds, we propose algorithms that exploit the correlation between the cost and reward of each arm by extracting the common information via linear minimum mean-square error estimation. We prove a regret lower bound for this problem, and show that the proposed algorithms achieve tight problem-dependent regret bounds, which are optimal up to a universal constant factor in the case of jointly Gaussian cost and reward pairs.
MIndGrasp: A New Training and Testing Framework for Motor Imagery Based 3-Dimensional Assistive Robotic Control
Venue: arXiv cs.HC 2020
Year: 2020

With increasing global age and disability assistive robots are becoming more necessary, and brain computer interfaces (BCI) are often proposed as a solution to understanding the intent of a disabled person that needs assistance. Most frameworks for electroencephalography (EEG)-based motor imagery (MI) BCI control rely on the direct control of the robot in Cartesian space. However, for 3-dimensional movement, this requires 6 motor imagery classes, which is a difficult distinction even for more experienced BCI users. In this paper, we present a simulated training and testing framework which reduces the number of motor imagery classes to 4 while still grasping objects in three-dimensional space. This is achieved through semi-autonomous eye-in-hand vision-based control of the robotic arm, while the user-controlled BCI achieves movement to the left and right, as well as movement toward and away from the object of interest. Additionally, the framework includes a method of training a BCI directly on the assistive robotic system, which should be more easily transferrable to a real-world assistive robot than using a standard training protocol such as Graz-BCI. Presented results do not consider real human EEG data, but are rather shown as a baseline for comparison with future human data and other improvements on the system.
Average Age of Changed Information in the Internet of Things
Venue: arXiv cs.IT 2020
Year: 2020

The freshness of status updates is imperative in mission-critical Internet of things (IoT) applications. Recently, Age of Information (AoI) has been proposed to measure the freshness of updates at the receiver. However, AoI only characterizes the freshness over time, but ignores the freshness in the content. In this paper, we introduce a new performance metric, Age of Changed Information (AoCI), which captures both the passage of time and the change of information content. Also, we examine the AoCI in a time-slotted status update system, where a sensor samples the physical process and transmits the update packets with a cost. We formulate a Markov Decision Process (MDP) to find the optimal updating policy that minimizes the weighted sum of the AoCI and the update cost. Particularly, in a special case that the physical process is modeled by a two-state discrete time Markov chain with equal transition probability, we show that the optimal policy is of threshold type with respect to the AoCI and derive the closed-form of the threshold. Finally, simulations are conducted to exhibit the performance of the threshold policy and its superiority over the zero-wait baseline policy.
Authentication, Access Control, Privacy, Threats and Trust Management Towards Securing Fog Computing Environments: A Review
Venue: arXiv cs.CR 2020
Year: 2020

Fog computing is an emerging computing paradigm that has come into consideration for the deployment of IoT applications amongst researchers and technology industries over the last few years. Fog is highly distributed and consists of a wide number of autonomous end devices, which contribute to the processing. However, the variety of devices offered across different users are not audited. Hence, the security of Fog devices is a major concern in the Fog computing environment. Furthermore, mitigating and preventing those security measures is a research issue. Therefore, to provide the necessary security for Fog devices, we need to understand what the security concerns are with regards to Fog. All aspects of Fog security, which have not been covered by other literature works needs to be identified and need to be aggregate all issues in Fog security. It needs to be noted that computation devices consist of many ordinary users, and are not managed by any central entity or managing body. Therefore, trust and privacy is also a key challenge to gain market adoption for Fog. To provide the required trust and privacy, we need to also focus on authentication, threats and access control mechanisms as well as techniques in Fog computing. In this paper, we perform a survey and propose a taxonomy, which presents an overview of existing security concerns in the context of the Fog computing paradigm. We discuss the Blockchain-based solutions towards a secure Fog computing environment and presented various research challenges and directions for future research.
Advanced kNN: A Mature Machine Learning Series
Venue: arXiv cs.LG 2020
Year: 2020

k-nearest neighbour (kNN) is one of the most prominent, simple and basic algorithm used in machine learning and data mining. However, kNN has limited prediction ability, i.e., kNN cannot predict any instance correctly if it does not belong to any of the predefined classes in the training data set. The purpose of this paper is to suggest an Advanced kNN (A-kNN) algorithm that will be able to classify an instance as unknown, after verifying that it does not belong to any of the predefined classes. Performance of kNN and A-kNN is compared on three different data sets namely iris plant data set, BUPA liver disorder data set, and Alpha Beta detection data set. Results of A-kNN are significantly accurate for detecting unknown instances.
Non-iterative One-step Solution for Point Set Registration Problem on Pose Estimation without Correspondence
Venue: arXiv cs.RO 2020
Year: 2020

In this work, we propose to directly find the one-step solution for the point set registration problem without correspondences. Inspired by the Kernel Correlation method, we consider the fully connected objective function between two point sets, thus avoiding the computation of correspondences. By utilizing least square minimization, the transformed objective function is directly solved with existing well-known closed-form solutions, e.g., singular value decomposition, that is usually used for given correspondences. However, using equal weights of costs for each connection will degenerate the solution due to the large influence of distant pairs. Thus, we additionally set a scale on each term to avoid high costs on non-important pairs. As in feature-based registration methods, the similarity between descriptors of points determines the scaling weight. Given the weights, we get a one step solution. As the runtime is in $\mathcal O (n^2)$, we also propose a variant with keypoints that strongly reduces the cost. The experiments show that the proposed method gives a one-step solution without an initial guess. Our method exhibits competitive outlier robustness and accuracy, compared to various other methods, and it is more stable in case of large rotations. Additionally, our one-step solution achieves a performance on-par with the state-of-the-art feature based method TEASER.