OpenRaiser/Intern-Atlas huggingface dataset:
The Angel is in the Priors: Improving GAN based Image and Sequence Inpainting with Better Noise and Structural Priors
Venue: arXiv cs.CV 2019
Year: 2019

Contemporary deep learning based inpainting algorithms are mainly based on a hybrid dual stage training policy of supervised reconstruction loss followed by an unsupervised adversarial critic loss. However, there is a dearth of literature for a fully unsupervised GAN based inpainting framework. The primary aversion towards the latter genre is due to its prohibitively slow iterative optimization requirement during inference to find a matching noise prior for a masked image. In this paper, we show that priors matter in GAN: we learn a data driven parametric network to predict a matching prior for a given image. This converts an iterative paradigm to a single feed forward inference pipeline with a massive 1500X speedup and simultaneous improvement in reconstruction quality. We show that an additional structural prior imposed on GAN model results in higher fidelity outputs. To extend our model for sequence inpainting, we propose a recurrent net based grouped noise prior learning. To our knowledge, this is the first demonstration of an unsupervised GAN based sequence inpainting. A further improvement in sequence inpainting is achieved with an additional subsequence consistency loss. These contributions improve the spatio-temporal characteristics of reconstructed sequences. Extensive experiments conducted on SVHN, Standford Cars, CelebA and CelebA-HQ image datasets, synthetic sequences and ViDTIMIT video datasets reveal that we consistently improve upon previous unsupervised baseline and also achieve comparable performances(sometimes also better) to hybrid benchmarks.
On-Demand Routing for Urban VANETs using Cooperating UAVs
Venue: arXiv cs.NI 2019
Year: 2019

Vehicular ad hoc networks (VANETs) are characterized by frequent routing path failures due to the high mobility caused by the sudden changes of the direction of vehicles. The routing paths between two different vehicles should be established with this challenge in mind. Stability and connectedness are a mandatory condition to ensure a robust and reliable data delivery. The idea behind this work is to exploit a new reactive routing technique to provide regulated and well-connected routing paths. Unmanned Aerial Vehicles (UAVs) or what are referred to as drones can be both involved in the discovery process and be full members in these discovered paths in order to avoid possible disconnections on the ground when the network is sparsely connected. The different tests of this technique are performed based on NS-2 simulator and the outcomes are compared with those of related on-demand routing protocols dedicated for VANETs. Interesting results are distinguished showing a reduced end-to-end delay and a high delivery ratio, which proving that this heterogeneous communication between vehicles and UAVs is able to extend the network connectivity.
Incorporating Word and Subword Units in Unsupervised Machine Translation Using Language Model Rescoring
Venue: arXiv cs.CL 2019
Year: 2019

This paper describes CAiRE's submission to the unsupervised machine translation track of the WMT'19 news shared task from German to Czech. We leverage a phrase-based statistical machine translation (PBSMT) model and a pre-trained language model to combine word-level neural machine translation (NMT) and subword-level NMT models without using any parallel data. We propose to solve the morphological richness problem of languages by training byte-pair encoding (BPE) embeddings for German and Czech separately, and they are aligned using MUSE (Conneau et al., 2018). To ensure the fluency and consistency of translations, a rescoring mechanism is proposed that reuses the pre-trained language model to select the translation candidates generated through beam search. Moreover, a series of pre-processing and post-processing approaches are applied to improve the quality of final translations.
How to Win First-Order Safety Games
Venue: arXiv cs.LO 2019
Year: 2019

First-order (FO) transition systems have recently attracted attention for the verification of parametric systems such as network protocols, software-defined networks or multi-agent workflows like conference management systems. Desirable properties of these systems such as functional correctness or noninterference have conveniently been formulated as safety properties. In order to automatically synthesize strategies that enforce safety or noninterference, we generalize FO transition systems to FO safety games. We prove that the existence of a winning strategy of safety player in finite games is in fact, equivalent to second-order quantifier elimination. For the important case of FO games with monadic predicates only, we provide a complete classification into decidable and undecidable cases. For games with non-monadic predicates, we concentrate on universal first-order invariants, since these are sufficient to express a large class of noninterference properties. Based on general techniques for second-order quantifier elimination, we provide abstraction and refinement techniques in order to synthesize FO strategies that enforce safety. We demonstrate the usefulness of our approach by inferring nontrivial FO specifications in a leader election protocol as well as for paper assignment in a conference mangagement system to exclude unappreciated disclosure of reports.
The Next 700 Policy Miners: A Universal Method for Building Policy Miners
Venue: arXiv cs.CR 2019
Year: 2019

A myriad of access control policy languages have been and continue to be proposed. The design of policy miners for each such language is a challenging task that has required specialized machine learning and combinatorial algorithms. We present an alternative method, universal access control policy mining (Unicorn). We show how this method streamlines the design of policy miners for a wide variety of policy languages including ABAC, RBAC, RBAC with user-attribute constraints, RBAC with spatio-temporal constraints, and an expressive fragment of XACML. For the latter two, there were no known policy miners until now. To design a policy miner using Unicorn, one needs a policy language and a metric quantifying how well a policy fits an assignment of permissions to users. From these, one builds the policy miner as a search algorithm that computes a policy that best fits the given permission assignment. We experimentally evaluate the policy miners built with Unicorn on logs from Amazon and access control matrices from other companies. Despite the genericity of our method, our policy miners are competitive with and sometimes even better than specialized state-of-the-art policy miners. The true positive rates of policies we mined differ by only 5% from the policies mined by the state of the art and the false positive rates are always below 5%. In the case of ABAC, it even outperforms the state of the art.
Bidirectional Context-Aware Hierarchical Attention Network for Document Understanding
Venue: arXiv cs.CL 2019
Year: 2019

The Hierarchical Attention Network (HAN) has made great strides, but it suffers a major limitation: at level 1, each sentence is encoded in complete isolation. In this work, we propose and compare several modifications of HAN in which the sentence encoder is able to make context-aware attentional decisions (CAHAN). Furthermore, we propose a bidirectional document encoder that processes the document forwards and backwards, using the preceding and following sentences as context. Experiments on three large-scale sentiment and topic classification datasets show that the bidirectional version of CAHAN outperforms HAN everywhere, with only a modest increase in computation time. While results are promising, we expect the superiority of CAHAN to be even more evident on tasks requiring a deeper understanding of the input documents, such as abstractive summarization. Code is publicly available.
Model-based Lookahead Reinforcement Learning
Venue: arXiv cs.LG 2019
Year: 2019

Model-based Reinforcement Learning (MBRL) allows data-efficient learning which is required in real world applications such as robotics. However, despite the impressive data-efficiency, MBRL does not achieve the final performance of state-of-the-art Model-free Reinforcement Learning (MFRL) methods. We leverage the strengths of both realms and propose an approach that obtains high performance with a small amount of data. In particular, we combine MFRL and Model Predictive Control (MPC). While MFRL's strength in exploration allows us to train a better forward dynamics model for MPC, MPC improves the performance of the MFRL policy by sampling-based planning. The experimental results in standard continuous control benchmarks show that our approach can achieve MFRL`s level of performance while being as data-efficient as MBRL.
PHY Research Is Sick but Curable: An Empirical Analysis
Venue: arXiv cs.IT 2019
Year: 2019

The controversy and argument on the usefulness of the physical layer (PHY) academic research for wireless communications are long-standing since the cellular communication paradigm gets to its maturity. In particular, researchers suspect that the performance improvement in cellular communications is primarily attributable to the increases in telecommunication infrastructure and radio spectrum instead of the PHY academic research, whereas concrete evidence is lacking. To respond to this controversy from an objective perspective, we employ econometric approaches to quantify the contributions of the PHY academic research and other performance determinants. Through empirical analysis and the quantitative evidence obtained, albeit preliminary, we shed light on the following issues: 1) what determines the cross-national differences in cellular network performance; 2) to what extent the PHY academic research and other factors affect cellular network performance; 3) what suggestions we can obtain from the data analysis for the stakeholders of the PHY research. To the best of our knowledge, this article is the first `empirical telecommunication research,' and the first effort to involve econometric methodologies to evaluate the usefulness of the PHY academic research.
Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-training
Venue: arXiv cs.CV 2019
Year: 2019

We propose Unicoder-VL, a universal encoder that aims to learn joint representations of vision and language in a pre-training manner. Borrow ideas from cross-lingual pre-trained models, such as XLM and Unicoder, both visual and linguistic contents are fed into a multi-layer Transformer for the cross-modal pre-training, where three pre-trained tasks are employed, including Masked Language Modeling (MLM), Masked Object Classification (MOC) and Visual-linguistic Matching (VLM). The first two tasks learn context-aware representations for input tokens based on linguistic and visual contents jointly. The last task tries to predict whether an image and a text describe each other. After pretraining on large-scale image-caption pairs, we transfer Unicoder-VL to caption-based image-text retrieval and visual commonsense reasoning, with just one additional output layer. We achieve state-of-the-art or comparable results on both two tasks and show the powerful ability of the cross-modal pre-training.
Recommendations and specifications for data scope analysis tools
Venue: arXiv cs.DC 2019
Year: 2019

This document is one of the deliverable reports created for the ESCAPE project. ESCAPE stands for Energy-efficient Scalable Algorithms for Weather Prediction at Exascale. The project develops world-class, extreme-scale computing capabilities for European operational numerical weather prediction and future climate models. This is done by identifying Weather & Climate dwarfs which are key patterns in terms of computation and communication (in the spirit of the Berkeley dwarfs). These dwarfs are then optimised for different hardware architectures (single and multi-node) and alternative algorithms are explored. Performance portability is addressed through the use of domain specific languages. In today's computer architectures, moving data is considerably more time- and energy consuming than computing on this data. One of the key performance optimizations for any application is therefore to minimize data motion and maximize data reuse. Especially on modern supercomputers with very complex and deep memory hierarchies, it is mandatory to take data locality into account. Especially when targeting accelerators with directive systems like OpenACC or OpenMP, identifying data scope, access type and data reuse are critical to minimize the data transfers from and to the accelerator. Unfortunately, manually identifying data locality information in complex code bases can be a time consuming task and tool support is therefore desirable. In this report we summarize the results of a survey of currently available tools that support software developers and performance engineers with data locality information in complex code bases like numerical weather prediction (NWP) or climate simulation applications. Based on the survey results we then recommend a tool and specify some extensions for a tool to solve the problems encountered in an NWP application.
The Adoption of Robotics by Government Agencies: Evidence from Crime Labs
Venue: arXiv cs.CY 2019
Year: 2019

While firms and factories often adopt technologies like robotics and advanced manufacturing techniques at a fast rate, government agencies are often seen as lagging in their adoption of such tools. We offer evidence about the adoption of robotics from the case of American crime laboratories.
A Mulching Proposal
Venue: arXiv cs.CY 2019
Year: 2019

he ethical implications of algorithmic systems have been much discussed in both HCI and the broader community of those interested in technology design, development and policy. In this paper, we explore the application of one prominent ethical framework - Fairness, Accountability, and Transparency - to a proposed algorithm that resolves various societal issues around food security and population ageing. Using various standardised forms of algorithmic audit and evaluation, we drastically increase the algorithm's adherence to the FAT framework, resulting in a more ethical and beneficent system. We discuss how this might serve as a guide to other researchers or practitioners looking to ensure better ethical outcomes from algorithmic systems in their line of work.
JVS corpus: free Japanese multi-speaker voice corpus
Venue: arXiv cs.SD 2019
Year: 2019

Thanks to improvements in machine learning techniques, including deep learning, speech synthesis is becoming a machine learning task. To accelerate speech synthesis research, we are developing Japanese voice corpora reasonably accessible from not only academic institutions but also commercial companies. In 2017, we released the JSUT corpus, which contains 10 hours of reading-style speech uttered by a single speaker, for end-to-end text-to-speech synthesis. For more general use in speech synthesis research, e.g., voice conversion and multi-speaker modeling, in this paper, we construct the JVS corpus, which contains voice data of 100 speakers in three styles (normal, whisper, and falsetto). The corpus contains 30 hours of voice data including 22 hours of parallel normal voices. This paper describes how we designed the corpus and summarizes the specifications. The corpus is available at our project page.
Message Passing Attention Networks for Document Understanding
Venue: arXiv cs.CL 2019
Year: 2019

Graph neural networks have recently emerged as a very effective framework for processing graph-structured data. These models have achieved state-of-the-art performance in many tasks. Most graph neural networks can be described in terms of message passing, vertex update, and readout functions. In this paper, we represent documents as word co-occurrence networks and propose an application of the message passing framework to NLP, the Message Passing Attention network for Document understanding (MPAD). We also propose several hierarchical variants of MPAD. Experiments conducted on 10 standard text classification datasets show that our architectures are competitive with the state-of-the-art. Ablation studies reveal further insights about the impact of the different components on performance. Code is publicly available at: https://github.com/giannisnik/mpad .
Knowledge Compilation for Boolean Functional Synthesis
Venue: arXiv cs.LO 2019
Year: 2019

Given a Boolean formula F(X,Y), where X is a vector of outputs and Y is a vector of inputs, the Boolean functional synthesis problem requires us to compute a Skolem function vector G(Y)for X such that F(G(Y),Y) holds whenever \exists X F(X,Y) holds. In this paper, we investigate the relation between the representation of the specification F(X,Y) and the complexity of synthesis. We introduce a new normal form for Boolean formulas, called SynNNF, that guarantees polynomial-time synthesis and also polynomial-time existential quantification for some order of quantification of variables. We show that several normal forms studied in the knowledge compilation literature are subsumed by SynNNF, although SynNNFcan be super-polynomially more succinct than them. Motivated by these results, we propose an algorithm to convert a specification in CNF to SynNNF, with the intent of solving the Boolean functional synthesis problem. Experiments with a prototype implementation show that this approach solves several benchmarks beyond the reach of state-of-the-art tools.
Sensors and Game Synchronization for Data Analysis in eSports
Venue: arXiv cs.HC 2019
Year: 2019

eSports industry has greatly progressed within the last decade in terms of audience and fund rising, broadcasting, networking and hardware. Since the number and quality of professional team has evolved too, there is a reasonable need in improving skills and training process of professional eSports athletes. In this work, we demonstrate a system able to collect heterogeneous data (physiological, environmental, video, telemetry) and guarantying synchronization with 10 ms accuracy. In particular, we demonstrate how to synchronize various sensors and ensure post synchronization, i.e. logged video, a so-called demo file, with the sensors data. Our experimental results achieved on the CS:GO game discipline show up to 3 ms accuracy of the time synchronization of the gaming computer.
Understanding Cyber Athletes Behaviour Through a Smart Chair: CS:GO and Monolith Team Scenario
Venue: arXiv cs.HC 2019
Year: 2019

eSports is the rapidly developing multidisciplinary domain. However, research and experimentation in eSports are in the infancy. In this work, we propose a smart chair platform - an unobtrusive approach to the collection of data on the eSports athletes and data further processing with machine learning methods. The use case scenario involves three groups of players: `cyber athletes' (Monolith team), semi-professional players and newbies all playing CS:GO discipline. In particular, we collect data from the accelerometer and gyroscope integrated in the chair and apply machine learning algorithms for the data analysis. Our results demonstrate that the professional athletes can be identified by their behaviour on the chair while playing the game.
Feedback-based, Automated Failure Testing of Microservice-based Applications
Venue: arXiv cs.SE 2019
Year: 2019

Modern distributed applications are moving toward a microservice architecture, in which each service is developed and managed independently, and new features and updates are delivered continuously. A guiding principle of microservice architecture is that it must be built to anticipate and mitigate a variety of hardware and software failures. In order to test the fault handling capabilities of microservces, this paper presents IntelliFT, a feedback-based, automated failure testing technique for microservice based applications, which aims to expose the defects in the fault-handling logic quickly. The initial experimental result on a medium-size microservice benchmark system shows that the proposed approach is effective.
Workload-Aware Opportunistic Energy Efficiency in Multi-FPGA Platforms
Venue: arXiv cs.AR 2019
Year: 2019

The continuous growth of big data applications with high computational and scalability demands has resulted in increasing popularity of cloud computing. Optimizing the performance and power consumption of cloud resources is therefore crucial to relieve the costs of data centers. In recent years, multi-FPGA platforms have gained traction in data centers as low-cost yet high-performance solutions particularly as acceleration engines, thanks to the high degree of parallelism they provide. Nonetheless, the size of data centers workloads varies during service time, leading to significant underutilization of computing resources while consuming a large amount of power, which turns out as a key factor of data center inefficiency, regardless of the underlying hardware structure. In this paper, we propose an efficient framework to throttle the power consumption of multi-FPGA platforms by dynamically scaling the voltage and hereby frequency during runtime according to prediction of, and adjustment to the workload level, while maintaining the desired Quality of Service (QoS). This is in contrast to, and more efficient than, conventional approaches that merely scale (i.e., power-gate) the computing nodes or frequency. The proposed framework carefully exploits a pre-characterized library of delay-voltage, and power-voltage information of FPGA resources, which we show is indispensable to obtain the efficient operating point due to the different sensitivity of resources w.r.t. voltage scaling, particularly considering multiple power rails residing in these devices. Our evaluations by implementing state-of-the-art deep neural network accelerators revealed that, providing an average power reduction of 4.0X, the proposed framework surpasses the previous works by 33.6% (up to 83%).
A clock-less ultra-low power bit-serial LVDS link for Address-Event multi-chip systems
Venue: arXiv cs.ET 2019
Year: 2019

We present a power efficient clock-less fully asynchronous bit-serial Low Voltage Differential Signaling (LVDS) link with event-driven instant wake-up and self-sleep features, optimized for high speed inter-chip communication of asynchronous address-events between neuromorphic chips. The proposed LVDS link makes use of the Level-Encoded Dual-Rail (LEDR) representation and a token-ring architecture to encode and transmit data, avoiding the use of conventional large ClockData Recovery (CDR) modules with power-hungry DLL or PLL circuits. We implemented the LVDS circuits in a device fabricated with a standard 0.18 um CMOS process. The total silicon area used for such block is of 0.14 mm^2. We present experimental measurement results to demonstrate that, with a bit rate of 1.5 Gbps and an event width of 32-bit, the proposed LVDS link can achieve transmission event rates of 35.7 M Events/second with current consumption of 19.3 mA and 3.57 mA for receiver and transmitter blocks, respectively. Given the clock-less and instant on/off design choices made, the power consumption of the whole link depends linearly on the data transmission rate. We show that the current consumption can go down to sub-uA for low event rates (e.g., <1k Events/second), with a floor of 80 nA for transmitter and 42 nA for receiver, determined mainly by static off-leakage currents.