OpenRaiser/Intern-Atlas huggingface dataset:
Scalable Intelligence-Enabled Networking with Traffic Engineering in 5G Scenarios for Future Audio-Visual-Tactile Internet
Venue: arXiv cs.NI 2019
Year: 2019

In order to improve future network performance, this paper proposes scalable intelligence-enabled networking (SIEN) with eliminating traffic redundancy for audio-visual-tactile Internet in 5G scenarios such as enhanced mobile broadband, ultra-reliable and low latency communication, and massive machine-type communication. The SIEN consists of an intelligent management plane (ImP), an intelligence-enabled plane (IeP), a control plane and a user plane. For the ImP, the containers with decision execution are constructed by a novel graph algorithm to organize objects such as network elements and resource partitions. For the IeP, a novel learning system is designed with decision making using a congruity function for generalization and personalization in the presence of imbalanced, conflicting and partial data. For the control plane, a scheme of identifier-locator mapping is designed by referring to information-centric networking and software-defined networking. For the user plane, the registrations, requests and data are forwarded to implement the SIEN and test its performance. The evaluation shows the SIEN outperforms four state-of-the-art techniques for redundant traffic reduction by up to 46.04% based on a mix of assumption, simulation and proof-of-concept implementation for audio-visual-tactile Internet multimedia service. To confirm the validity, the best case and the worst case for traffic offloading are tested with the data rate, the latency and the density. The evaluation only focused on the scalability issue, while the SIEN would be beneficial to improve more issues such as inter-domain security, ultra-low latency, on-demand mobility, multi-homing routing, and cross-layer feature incongruity.
An Overview of Capacity Results for Synchronization Channels
Venue: arXiv cs.IT 2019
Year: 2019

Synchronization channels, such as the well-known deletion channel, are surprisingly harder to analyze than memoryless channels, and they are a source of many fundamental problems in information theory and theoretical computer science. One of the most basic open problems regarding synchronization channels is the derivation of an exact expression for their capacity. Unfortunately, most of the classic information-theoretic techniques at our disposal fail spectacularly when applied to synchronization channels. Therefore, new approaches must be considered to tackle this problem. This survey gives an account of the great effort made over the past few decades to better understand the (broadly defined) capacity of synchronization channels, including both the main results and the novel techniques underlying them. Besides the usual notion of channel capacity, we also discuss the zero-error capacity of synchronization channels.
A new CP-approach for a parallel machine scheduling problem with time constraints on machine qualifications
Venue: arXiv cs.DC 2019
Year: 2019

This paper considers the scheduling of job families on parallel machines with time constraints on machine qualifications. In this problem, each job belongs to a family and a family can only be executed on a subset of qualified machines. In addition, machines can lose their qualifications during the schedule. Indeed, if no job of a family is scheduled on a machine during a given amount of time, the machine loses its qualification for this family. The goal is to minimize the sum of job completion times, i.e. the flow time, while maximizing the number of qualifications at the end of the schedule. The paper presents a new Constraint Programming (CP) model taking more advantages of the CP feature to model machine disqualifications. This model is compared with two existing models: an Integer Linear Programming (ILP) model and a Constraint Programming model. The experiments show that the new CP model outperforms the other model when the priority is given to the number of disqualifications objective. Furthermore, it is competitive with the other model when the flow time objective is prioritized.
Transform the Set: Memory Attentive Generation of Guided and Unguided Image Collages
Venue: arXiv cs.CV 2019
Year: 2019

Cutting and pasting image segments feels intuitive: the choice of source templates gives artists flexibility in recombining existing source material. Formally, this process takes an image set as input and outputs a collage of the set elements. Such selection from sets of source templates does not fit easily in classical convolutional neural models requiring inputs of fixed size. Inspired by advances in attention and set-input machine learning, we present a novel architecture that can generate in one forward pass image collages of source templates using set-structured representations. This paper has the following contributions: (i) a novel framework for image generation called Memory Attentive Generation of Image Collages (MAGIC) which gives artists new ways to create digital collages; (ii) from the machine-learning perspective, we show a novel Generative Adversarial Networks (GAN) architecture that uses Set-Transformer layers and set-pooling to blend sets of random image samples - a hybrid non-parametric approach.
FISHDBC: Flexible, Incremental, Scalable, Hierarchical Density-Based Clustering for Arbitrary Data and Distance
Venue: arXiv cs.LG 2019
Year: 2019

FISHDBC is a flexible, incremental, scalable, and hierarchical density-based clustering algorithm. It is flexible because it empowers users to work on arbitrary data, skipping the feature extraction step that usually transforms raw data in numeric arrays letting users define an arbitrary distance function instead. It is incremental and scalable: it avoids the $\mathcal O(n^2)$ performance of other approaches in non-metric spaces and requires only lightweight computation to update the clustering when few items are added. It is hierarchical: it produces a "flat" clustering which can be expanded to a tree structure, so that users can group and/or divide clusters in sub- or super-clusters when data exploration requires so. It is density-based and approximates HDBSCAN*, an evolution of DBSCAN.
Frequency and temporal convolutional attention for text-independent speaker recognition
Venue: arXiv cs.SD 2019
Year: 2019

Majority of the recent approaches for text-independent speaker recognition apply attention or similar techniques for aggregation of frame-level feature descriptors generated by a deep neural network (DNN) front-end. In this paper, we propose methods of convolutional attention for independently modelling temporal and frequency information in a convolutional neural network (CNN) based front-end. Our system utilizes convolutional block attention modules (CBAMs) [1] appropriately modified to accommodate spectrogram inputs. The proposed CNN front-end fitted with the proposed convolutional attention modules outperform the no-attention and spatial-CBAM baselines by a significant margin on the VoxCeleb [2, 3] speaker verification benchmark, and our best model achieves an equal error rate of 2:031% on the VoxCeleb1 test set, improving the existing state of the art result by a significant margin. For a more thorough assessment of the effects of frequency and temporal attention in real-world conditions, we conduct ablation experiments by randomly dropping frequency bins and temporal frames from the input spectrograms, concluding that instead of modelling either of the entities, simultaneously modelling temporal and frequency attention translates to better real-world performance.
Model-Agnostic Meta-Learning using Runge-Kutta Methods
Venue: arXiv cs.LG 2019
Year: 2019

Meta-learning has emerged as an important framework for learning new tasks from just a few examples. The success of any meta-learning model depends on (i) its fast adaptation to new tasks, as well as (ii) having a shared representation across similar tasks. Here we extend the model-agnostic meta-learning (MAML) framework introduced by Finn et al. (2017) to achieve improved performance by analyzing the temporal dynamics of the optimization procedure via the Runge-Kutta method. This method enables us to gain fine-grained control over the optimization and helps us achieve both the adaptation and representation goals across tasks. By leveraging this refined control, we demonstrate that there are multiple principled ways to update MAML and show that the classic MAML optimization is simply a special case of second-order Runge-Kutta method that mainly focuses on fast-adaptation. Experiments on benchmark classification, regression and reinforcement learning tasks show that this refined control helps attain improved results.
Evolution of transfer learning in natural language processing
Venue: arXiv cs.CL 2019
Year: 2019

In this paper, we present a study of the recent advancements which have helped bring Transfer Learning to NLP through the use of semi-supervised training. We discuss cutting-edge methods and architectures such as BERT, GPT, ELMo, ULMFit among others. Classically, tasks in natural language processing have been performed through rule-based and statistical methodologies. However, owing to the vast nature of natural languages these methods do not generalise well and failed to learn the nuances of language. Thus machine learning algorithms such as Naive Bayes and decision trees coupled with traditional models such as Bag-of-Words and N-grams were used to usurp this problem. Eventually, with the advent of advanced recurrent neural network architectures such as the LSTM, we were able to achieve state-of-the-art performance in several natural language processing tasks such as text classification and machine translation. We talk about how Transfer Learning has brought about the well-known ImageNet moment for NLP. Several advanced architectures such as the Transformer and its variants have allowed practitioners to leverage knowledge gained from unrelated task to drastically fasten convergence and provide better performance on the target task. This survey represents an effort at providing a succinct yet complete understanding of the recent advances in natural language processing using deep learning in with a special focus on detailing transfer learning and its potential advantages.
Kriptosare.gen, a dockerized Bitcoin testbed: analysis of server performance
Venue: arXiv cs.PF 2019
Year: 2019

Bitcoin is a peer-to-peer distributed cryptocurrency system, that keeps all transaction history in a public ledger known as blockchain. The Bitcoin network is implicitly pseudoanonymous and its nodes are controlled by independent entities making network analysis difficult. This calls for the development of a fully controlled testing environment. This paper presents Kriptosare.gen, a dockerized automatized Bitcoin testbed, for deploying full-scale custom Bitcoin networks. The testbed is deployed in a single machine executing four different experiments, each one with different network configuration. We perform a cost analysis to investigate how the resources are related with network parameters and provide experimental data quantifying the amount of computational resources needed to run the different types of simulations. Obtained results demonstrate that it is possible to run the testbed with a configuration similar to a real Bitcoin system.
JSDoop and TensorFlow.js: Volunteer Distributed Web Browser-Based Neural Network Training
Venue: arXiv cs.DC 2019
Year: 2019

In 2019, around 57\% of the population of the world has broadband access to the Internet. Moreover, there are 5.9 billion mobile broadband subscriptions, i.e., 1.3 subscriptions per user. So there is an enormous interconnected computational power held by users all around the world. Also, it is estimated that Internet users spend more than six and a half hours online every day. But in spite of being a great amount of time, those resources are idle most of the day. Therefore, taking advantage of them presents an interesting opportunity. In this study, we introduce JSDoop, a prototype implementation to profit from this opportunity. In particular, we propose a volunteer web browser-based high-performance computing library. JSdoop divides a problem into tasks and uses different queues to distribute the computation. Then, volunteers access the web page of the problem and start processing the tasks in their web browsers. We conducted a proof-of-concept using our proposal and TensorFlow.js to train a recurrent neural network that predicts text. We tested it in a computer cluster and with up to 32 volunteers. The experimental results show that training a neural network in distributed web browsers is feasible and accurate, has a high scalability, and it is an interesting area for research.
Comprehend Medical: a Named Entity Recognition and Relationship Extraction Web Service
Venue: arXiv cs.CL 2019
Year: 2019

Comprehend Medical is a stateless and Health Insurance Portability and Accountability Act (HIPAA) eligible Named Entity Recognition (NER) and Relationship Extraction (RE) service launched under Amazon Web Services (AWS) trained using state-of-the-art deep learning models. Contrary to many existing open source tools, Comprehend Medical is scalable and does not require steep learning curve, dependencies, pipeline configurations, or installations. Currently, Comprehend Medical performs NER in five medical categories: Anatomy, Medical Condition, Medications, Protected Health Information (PHI) and Treatment, Test and Procedure (TTP). Additionally, the service provides relationship extraction for the detected entities as well as contextual information such as negation and temporality in the form of traits. Comprehend Medical provides two Application Programming Interfaces (API): 1) the NERe API which returns all the extracted named entities, their traits and the relationships between them and 2) the PHId API which returns just the protected health information contained in the text. Furthermore, Comprehend Medical is accessible through AWS Console, Java and Python Software Development Kit (SDK), making it easier for non-developers and developers to use.
Linguistic evaluation of German-English Machine Translation using a Test Suite
Venue: arXiv cs.CL 2019
Year: 2019

We present the results of the application of a grammatical test suite for German$\rightarrow$English MT on the systems submitted at WMT19, with a detailed analysis for 107 phenomena organized in 14 categories. The systems still translate wrong one out of four test items in average. Low performance is indicated for idioms, modals, pseudo-clefts, multi-word expressions and verb valency. When compared to last year, there has been a improvement of function words, non-verbal agreement and punctuation. More detailed conclusions about particular systems and phenomena are also presented.
Sapphire: A Configurable Crypto-Processor for Post-Quantum Lattice-based Protocols
Venue: arXiv cs.CR 2019
Year: 2019

Public key cryptography protocols, such as RSA and elliptic curve cryptography, will be rendered insecure by Shor's algorithm when large-scale quantum computers are built. Cryptographers are working on quantum-resistant algorithms, and lattice-based cryptography has emerged as a prime candidate. However, high computational complexity of these algorithms makes it challenging to implement lattice-based protocols on low-power embedded devices. To address this challenge, we present Sapphire - a lattice cryptography processor with configurable parameters. Efficient sampling, with a SHA-3-based PRNG, provides two orders of magnitude energy savings; a single-port RAM-based number theoretic transform memory architecture is proposed, which provides 124k-gate area savings; while a low-power modular arithmetic unit accelerates polynomial computations. Our test chip was fabricated in TSMC 40nm low-power CMOS process, with the Sapphire cryptographic core occupying 0.28 mm2 area consisting of 106k logic gates and 40.25 KB SRAM. Sapphire can be programmed with custom instructions for polynomial arithmetic and sampling, and it is coupled with a low-power RISC-V micro-processor to demonstrate NIST Round 2 lattice-based CCA-secure key encapsulation and signature protocols Frodo, NewHope, qTESLA, CRYSTALS-Kyber and CRYSTALS-Dilithium, achieving up to an order of magnitude improvement in performance and energy-efficiency compared to state-of-the-art hardware implementations. All key building blocks of Sapphire are constant-time and secure against timing and simple power analysis side-channel attacks. We also discuss how masking-based DPA countermeasures can be implemented on the Sapphire core without any changes to the hardware.
Blockchain Tree as Solution for Distributed Storage of Personal ID Data and Document Access Control
Venue: arXiv cs.CR 2019
Year: 2019

This paper introduces a new method of Blockchain formation for reliable storage of personal data of ID-card holders. In particular, the model of the information system is presented, the new structure of smart ID-cards and information on these cards are proposed. The new structure of Blockchain - "Blockchain Tree" allows not only to store information from ID-cards but also to increase the level of security and access control to this information. The proposed Subchains system allows to integrate Blockchain of the lower level to Blockchain of the higher level, allowing to create a multilevel protected system.
Learning chordal extensions
Venue: arXiv cs.LG 2019
Year: 2019

A highly influential ingredient of many techniques designed to exploit sparsity in numerical optimization is the so-called chordal extension of a graph representation of the optimization problem. The definitive relation between chordal extension and the performance of the optimization algorithm that uses the extension is not a mathematically understood task. For this reason, we follow the current research trend of looking at Combinatorial Optimization tasks by using a Machine Learning lens, and we devise a framework for learning elimination rules yielding high-quality chordal extensions. As a first building block of the learning framework, we propose an on-policy imitation learning scheme that mimics the elimination ordering provided by the (classical) minimum degree rule. The results show that our on-policy imitation learning approach is effective in learning the minimum degree policy and, consequently, produces graphs with desirable fill-in characteristics.
Towards Annotating and Creating Sub-Sentence Summary Highlights
Venue: arXiv cs.CL 2019
Year: 2019

Highlighting is a powerful tool to pick out important content and emphasize. Creating summary highlights at the sub-sentence level is particularly desirable, because sub-sentences are more concise than whole sentences. They are also better suited than individual words and phrases that can potentially lead to disfluent, fragmented summaries. In this paper we seek to generate summary highlights by annotating summary-worthy sub-sentences and teaching classifiers to do the same. We frame the task as jointly selecting important sentences and identifying a single most informative textual unit from each sentence. This formulation dramatically reduces the task complexity involved in sentence compression. Our study provides new benchmarks and baselines for generating highlights at the sub-sentence level.
SNF: Serverless Network Functions
Venue: arXiv cs.DC 2019
Year: 2019

It is increasingly common to outsource network functions (NFs) to the cloud. However, no cloud providers offer NFs-as-a-Service (NFaaS) that allows users to run custom NFs. Our work addresses how a cloud provider can offer NFaaS. We use the emerging serverless computing paradigm as it has the right building blocks - usage-based billing, convenient event-driven programming model and automatic compute elasticity. Towards this end, we identify two core limitations of existing serverless platforms to support demanding stateful NFs - coupling of the billing and work assignment granularities, and state sharing via an external store. We develop a novel NFaaS framework, SNF, that overcomes these issues using two ideas. SNF allocates work at the granularity of flowlets observed in network traffic, whereas billing and programming occur on the basis of packets. SNF embellishes serverless platforms with ephemeral state that lasts for the duration of the flowlet and supports high performance state operations between compute units in a peer-to-peer manner. We present algorithms for work allocation and state maintenance, and demonstrate that our SNF prototype dynamically adapts compute resources for various stateful NFs based on traffic demand at very fine time scales, with minimal overheads.
Perceiving QUIC: Do Users Notice or Even Care?
Venue: arXiv cs.NI 2019
Year: 2019

QUIC, as the foundation for HTTP/3, is becoming an Internet reality. A plethora of studies already show that QUIC excels beyond TCP+TLS+HTTP/2. Yet, these studies compare a highly optimized QUIC Web stack against an unoptimized TCP-based stack. In this paper, we bring TCP up to speed to perform an eye-level comparison. Instead of relying on technical metrics, we perform two extensive user studies to investigate QUIC's impact on the quality of experience. First, we investigate if users can distinguish two protocol versions in a direct comparison, and we find that QUIC is indeed rated faster than TCP and even a tuned TCP. Yet, our second study shows that this perceived performance increase does mostly not matter to the users, and they rate QUIC and TCP indistinguishable.
Autoregressive Models: What Are They Good For?
Venue: arXiv cs.LG 2019
Year: 2019

Autoregressive (AR) models have become a popular tool for unsupervised learning, achieving state-of-the-art log likelihood estimates. We investigate the use of AR models as density estimators in two settings -- as a learning signal for image translation, and as an outlier detector -- and find that these density estimates are much less reliable than previously thought. We examine the underlying optimization issues from both an empirical and theoretical perspective, and provide a toy example that illustrates the problem. Overwhelmingly, we find that density estimates do not correlate with perceptual quality and are unhelpful for downstream tasks.
A Tool for Automatically Suggesting Source-Code Optimizations for Complex GPU Kernels
Venue: arXiv cs.DC 2019
Year: 2019

Future computing systems, from handhelds to supercomputers, will undoubtedly be more parallel and heterogeneous than todays systems to provide more performance and energy efficiency. Thus, GPUs are increasingly being used to accelerate general purpose applications, including applications with data dependent, irregular control flow and memory access patterns. However, the growing complexity, exposed memory hierarchy, incoherence, heterogeneity, and parallelism will make accelerator based systems progressively more difficult to program. In the foreseeable future, the vast majority of programmers will no longer be able to extract additional performance or energy savings from next generation systems be-cause the programming will be too difficult. Automatic performance analysis and optimization recommendation tools have the potential to avert this situation. They embody expert knowledge and make it available to software developers when needed. In this paper, we describe and evaluate such a tool.