OpenRaiser/Intern-Atlas huggingface dataset:
FlexSA: Flexible Systolic Array Architecture for Efficient Pruned DNN Model Training
Venue: arXiv cs.LG 2020
Year: 2020

Modern deep learning models have high memory and computation cost. To make them fast and memory-cost efficient, structured model pruning is commonly used. We find that pruning a model using a common training accelerator with large systolic arrays is extremely performance-inefficient. To make a systolic array efficient for pruning and training, we propose FlexSA, a flexible systolic array architecture. FlexSA dynamically reconfigures the systolic array structure and offers multiple sub-systolic operating modes, which are designed for energy- and memory bandwidth-efficient processing of tensors with different sizes and shapes. We also present a compilation heuristic for tiling matrix-multiplication-and-accumulation operations in a training workload to best utilize the resources of FlexSA. Based on our evaluation, FlexSA with the proposed compilation heuristic improves compute resource utilization of pruning and training modern CNN models by 37% compared to a conventional training accelerator with a large systolic array. FlexSA also improves on-chip data reuse by 1.7X saving 28% energy compared to naive systolic array splitting.
A Novel Attention-based Aggregation Function to Combine Vision and Language
Venue: arXiv cs.CV 2020
Year: 2020

The joint understanding of vision and language has been recently gaining a lot of attention in both the Computer Vision and Natural Language Processing communities, with the emergence of tasks such as image captioning, image-text matching, and visual question answering. As both images and text can be encoded as sets or sequences of elements -- like regions and words -- proper reduction functions are needed to transform a set of encoded elements into a single response, like a classification or similarity score. In this paper, we propose a novel fully-attentive reduction method for vision and language. Specifically, our approach computes a set of scores for each element of each modality employing a novel variant of cross-attention, and performs a learnable and cross-modal reduction, which can be used for both classification and ranking. We test our approach on image-text matching and visual question answering, building fair comparisons with other reduction choices, on both COCO and VQA 2.0 datasets. Experimentally, we demonstrate that our approach leads to a performance increase on both tasks. Further, we conduct ablation studies to validate the role of each component of the approach.
Secure Non-public Health Enterprise Networks
Venue: arXiv cs.NI 2020
Year: 2020

Increasing demand for secure remote operation in industry and technology advancements to support delivering efficient services and tele-mentoring have opened a new market in healthcare sector and emergency services based on 5G and Tactile internet capabilities. In a connected world, hospitals would benefit from providing the on-time availability either for continuous health monitoring or critical services to the citizens in need. In this paper, we propose a secure non-public health enterprise network concept to enable an end-to-end secure and location-agnostic communication between a patient and a healthcare service provider, and other contacts with patients consent either in case of an emergency or to be stored in the medical records. We present how applying non-public enterprise networks can address market demand in health care sector for improved end-to-end security and privacy when dealing with personal and critical information. We present the three tier architecture model describing continuous authentication mechanisms based on biometric collection as well as the dynamic network solutions in the healthcare domain. The biometric collection can be done using ambient or IoT sensors as well as wearable or implantable devices to monitor the patient unobtrusively. Furthermore, end-to-end security solutions should adapt dynamically based on the user profile and situation awareness to address the required level of security at the network side. We discuss the related research challenges for developing the presented non-public health enterprise platform and provide suggestions for future work based on the healthcare sector requirements and opportunities.
Compact retail shelf segmentation for mobile deployment
Venue: arXiv cs.CV 2020
Year: 2020

The recent surge of automation in the retail industries has rapidly increased demand for applying deep learning models on mobile devices. To make the deep learning models real-time on-device, a compact efficient network becomes inevitable. In this paper, we work on one such common problem in the retail industries - Shelf segmentation. Shelf segmentation can be interpreted as a pixel-wise classification problem, i.e., each pixel is classified as to whether they belong to visible shelf edges or not. The aim is not just to segment shelf edges, but also to deploy the model on mobile devices. As there is no standard solution for such dense classification problem on mobile devices, we look at semantic segmentation architectures which can be deployed on edge. We modify low-footprint semantic segmentation architectures to perform shelf segmentation. In addressing this issue, we modified the famous U-net architecture in certain aspects to make it fit for on-devices without impacting significant drop in accuracy and also with 15X fewer parameters. In this paper, we proposed Light Weight Segmentation Network (LWSNet), a small compact model able to run fast on devices with limited memory and can train with less amount (~ 100 images) of labeled data.
Nested Tailbiting Convolutional Codes for Secrecy, Privacy, and Storage
Venue: arXiv cs.IT 2020
Year: 2020

A key agreement problem is considered that has a biometric or physical identifier, a terminal for key enrollment, and a terminal for reconstruction. A nested convolutional code design is proposed that performs vector quantization during enrollment and error control during reconstruction. Physical identifiers with small bit error probability illustrate the gains of the design. One variant of the nested convolutional codes improves on the best known key vs. storage rate ratio but it has high complexity. A second variant with lower complexity performs similar to nested polar codes. The results suggest that the choice of code for key agreement with identifiers depends primarily on the complexity constraint.
Learning to Rank in the Position Based Model with Bandit Feedback
Venue: arXiv cs.LG 2020
Year: 2020

Personalization is a crucial aspect of many online experiences. In particular, content ranking is often a key component in delivering sophisticated personalization results. Commonly, supervised learning-to-rank methods are applied, which suffer from bias introduced during data collection by production systems in charge of producing the ranking. To compensate for this problem, we leverage contextual multi-armed bandits. We propose novel extensions of two well-known algorithms viz. LinUCB and Linear Thompson Sampling to the ranking use-case. To account for the biases in a production environment, we employ the position-based click model. Finally, we show the validity of the proposed algorithms by conducting extensive offline experiments on synthetic datasets as well as customer facing online A/B experiments.
Coverage Improvement of Wireless Sensor Networks via Spatial Profile Information
Venue: arXiv cs.IT 2020
Year: 2020

This paper considers a wireless sensor network deployed to sense an environment variable with a known spatial statistical profile. We propose to use the additional information of the spatial profile to improve the sensing range of sensors while allowing some tolerance in their sensing accuracy. We show that the use of this information improves the sensing performance of the total WSN. For this, we first derive analytical expressions for various performance metrics to measure the improvement in the sensing performance of WSN. We then discuss the sensing gains quantitatively using numerical results.
Word Interdependence Exposes How LSTMs Compose Representations
Venue: arXiv cs.CL 2020
Year: 2020

Recent work in NLP shows that LSTM language models capture compositional structure in language data. For a closer look at how these representations are composed hierarchically, we present a novel measure of interdependence between word meanings in an LSTM, based on their interactions at the internal gates. To explore how compositional representations arise over training, we conduct simple experiments on synthetic data, which illustrate our measure by showing how high interdependence can hurt generalization. These synthetic experiments also illustrate a specific hypothesis about how hierarchical structures are discovered over the course of training: that parent constituents rely on effective representations of their children, rather than on learning long-range relations independently. We further support this measure with experiments on English language data, where interdependence is higher for more closely syntactically linked word pairs.
SCELMo: Source Code Embeddings from Language Models
Venue: arXiv cs.SE 2020
Year: 2020

Continuous embeddings of tokens in computer programs have been used to support a variety of software development tools, including readability, code search, and program repair. Contextual embeddings are common in natural language processing but have not been previously applied in software engineering. We introduce a new set of deep contextualized word representations for computer programs based on language models. We train a set of embeddings using the ELMo (embeddings from language models) framework of Peters et al (2018). We investigate whether these embeddings are effective when fine-tuned for the downstream task of bug detection. We show that even a low-dimensional embedding trained on a relatively small corpus of programs can improve a state-of-the-art machine learning system for bug detection.
User Concerns & Tradeoffs in Technology-Facilitated Contact Tracing
Venue: arXiv cs.CR 2020
Year: 2020

The COVID19 pandemic spread across the world in late 2019 and early 2020. As the pandemic spread, technologists joined forces with public health officials to develop apps to support COVID19 response. Yet, for these technological solutions to benefit public health, users must be willing to adopt these apps.This paper details the potential inputs to a user's decision to adopt a COVID19 contact-tracing app or other technology and empirically validates the relevance of these inputs via both the literature and a demographically-representative survey of 1,000 Americans.
KoParadigm: A Korean Conjugation Paradigm Generator
Venue: arXiv cs.CL 2020
Year: 2020

Korean is a morphologically rich language. Korean verbs change their forms in a fickle manner depending on tense, mood, speech level, meaning, etc. Therefore, it is challenging to construct comprehensive conjugation paradigms of Korean verbs. In this paper we introduce a Korean (verb) conjugation paradigm generator, dubbed KoParadigm. To the best of our knowledge, it is the first Korean conjugation module that covers all contemporary Korean verbs and endings. KoParadigm is not only linguistically well established, but also computationally simple and efficient. We share it via PyPi.
An ASP-Based Approach to Counterfactual Explanations for Classification
Venue: arXiv cs.LG 2020
Year: 2020

We propose answer-set programs that specify and compute counterfactual interventions as a basis for causality-based explanations to decisions produced by classification models. They can be applied with black-box models and models that can be specified as logic programs, such as rule-based classifiers. The main focus in on the specification and computation of maximum responsibility causal explanations. The use of additional semantic knowledge is investigated.
A Retrospective Analysis of User Exposure to (Illicit) Cryptocurrency Mining on the Web
Venue: arXiv cs.CR 2020
Year: 2020

In late 2017, a sudden proliferation of malicious JavaScript was reported on the Web: browser-based mining exploited the CPU time of website visitors to mine the cryptocurrency Monero. Several studies measured the deployment of such code and developed defenses. However, previous work did not establish how many users were really exposed to the identified mining sites and whether there was a real risk given common user browsing behavior. In this paper, we present a retroactive analysis to close this research gap. We pool large-scale, longitudinal data from several vantage points, gathered during the prime time of illicit cryptomining, to measure the impact on web users. We leverage data from passive traffic monitoring of university networks and a large European ISP, with suspected mining sites identified in previous active scans. We corroborate our results with data from a browser extension with a large user base that tracks site visits. We also monitor open HTTP proxies and the Tor network for malicious injection of code. We find that the risk for most Web users was always very low, much lower than what deployment scans suggested. Any exposure period was also very brief. However, we also identify a previously unknown and exploited attack vector on mobile devices.
Deep Conversational Recommender Systems: A New Frontier for Goal-Oriented Dialogue Systems
Venue: arXiv cs.LG 2020
Year: 2020

In recent years, the emerging topics of recommender systems that take advantage of natural language processing techniques have attracted much attention, and one of their applications is the Conversational Recommender System (CRS). Unlike traditional recommender systems with content-based and collaborative filtering approaches, CRS learns and models user's preferences through interactive dialogue conversations. In this work, we provide a summarization of the recent evolution of CRS, where deep learning approaches are applied to CRS and have produced fruitful results. We first analyze the research problems and present key challenges in the development of Deep Conversational Recommender Systems (DCRS), then present the current state of the field taken from the most recent researches, including the most common deep learning models that benefit DCRS. Finally, we discuss future directions for this vibrant area.
Conversational Word Embedding for Retrieval-Based Dialog System
Venue: arXiv cs.CL 2020
Year: 2020

Human conversations contain many types of information, e.g., knowledge, common sense, and language habits. In this paper, we propose a conversational word embedding method named PR-Embedding, which utilizes the conversation pairs $ \left\langle{post, reply} \right\rangle$ to learn word embedding. Different from previous works, PR-Embedding uses the vectors from two different semantic spaces to represent the words in post and reply. To catch the information among the pair, we first introduce the word alignment model from statistical machine translation to generate the cross-sentence window, then train the embedding on word-level and sentence-level. We evaluate the method on single-turn and multi-turn response selection tasks for retrieval-based dialog systems. The experiment results show that PR-Embedding can improve the quality of the selected response. PR-Embedding source code is available at https://github.com/wtma/PR-Embedding
Learning Interpretable and Discrete Representations with Adversarial Training for Unsupervised Text Classification
Venue: arXiv cs.CL 2020
Year: 2020

Learning continuous representations from unlabeled textual data has been increasingly studied for benefiting semi-supervised learning. Although it is relatively easier to interpret discrete representations, due to the difficulty of training, learning discrete representations for unlabeled textual data has not been widely explored. This work proposes TIGAN that learns to encode texts into two disentangled representations, including a discrete code and a continuous noise, where the discrete code represents interpretable topics, and the noise controls the variance within the topics. The discrete code learned by TIGAN can be used for unsupervised text classification. Compared to other unsupervised baselines, the proposed TIGAN achieves superior performance on six different corpora. Also, the performance is on par with a recently proposed weakly-supervised text classification method. The extracted topical words for representing latent topics show that TIGAN learns coherent and highly interpretable topics.
Parameter Estimation in a Noisy 1D Environment via Two Absorbing Receivers
Venue: arXiv cs.ET 2020
Year: 2020

This paper investigates the estimation of different parameters, e.g., propagation distance and flow velocity, by utilizing two fully-absorbing receivers (RXs) in a one-dimensional (1D) environment. The time-varying number of absorbed molecules at each RX and the number of absorbed molecules in a time interval as time approaches infinity are derived. Noisy molecules in this environment, that are released by sources in addition to the transmitter, are also considered. A novel estimation method, namely difference estimation (DE), is proposed to eliminate the effect of noise by using the difference of received signals at the two RXs. For DE, the Cramer-Rao lower bound (CRLB) on the variance of estimation is derived. Independent maximum likelihood estimation is also considered at each RX as a benchmark to show the performance advantage of DE. Aided by particle-based simulation, the derived analytical results are verified. Furthermore, numerical results show that DE attains the CRLB and is less sensitive to the change of noise than independent estimation at each RX.
Assessing the Bilingual Knowledge Learned by Neural Machine Translation Models
Venue: arXiv cs.CL 2020
Year: 2020

Machine translation (MT) systems translate text between different languages by automatically learning in-depth knowledge of bilingual lexicons, grammar and semantics from the training examples. Although neural machine translation (NMT) has led the field of MT, we have a poor understanding on how and why it works. In this paper, we bridge the gap by assessing the bilingual knowledge learned by NMT models with phrase table -- an interpretable table of bilingual lexicons. We extract the phrase table from the training examples that an NMT model correctly predicts. Extensive experiments on widely-used datasets show that the phrase table is reasonable and consistent against language pairs and random seeds. Equipped with the interpretable phrase table, we find that NMT models learn patterns from simple to complex and distill essential bilingual knowledge from the training examples. We also revisit some advances that potentially affect the learning of bilingual knowledge (e.g., back-translation), and report some interesting findings. We believe this work opens a new angle to interpret NMT with statistic models, and provides empirical supports for recent advances in improving NMT models.
VD-BERT: A Unified Vision and Dialog Transformer with BERT
Venue: arXiv cs.CV 2020
Year: 2020

Visual dialog is a challenging vision-language task, where a dialog agent needs to answer a series of questions through reasoning on the image content and dialog history. Prior work has mostly focused on various attention mechanisms to model such intricate interactions. By contrast, in this work, we propose VD-BERT, a simple yet effective framework of unified vision-dialog Transformer that leverages the pretrained BERT language models for Visual Dialog tasks. The model is unified in that (1) it captures all the interactions between the image and the multi-turn dialog using a single-stream Transformer encoder, and (2) it supports both answer ranking and answer generation seamlessly through the same architecture. More crucially, we adapt BERT for the effective fusion of vision and dialog contents via visually grounded training. Without the need of pretraining on external vision-language data, our model yields new state of the art, achieving the top position in both single-model and ensemble settings (74.54 and 75.35 NDCG scores) on the visual dialog leaderboard. Our code and pretrained models are released at https://github.com/salesforce/VD-BERT.
Genetic programming approaches to learning fair classifiers
Venue: arXiv cs.NE 2020
Year: 2020

Society has come to rely on algorithms like classifiers for important decision making, giving rise to the need for ethical guarantees such as fairness. Fairness is typically defined by asking that some statistic of a classifier be approximately equal over protected groups within a population. In this paper, current approaches to fairness are discussed and used to motivate algorithmic proposals that incorporate fairness into genetic programming for classification. We propose two ideas. The first is to incorporate a fairness objective into multi-objective optimization. The second is to adapt lexicase selection to define cases dynamically over intersections of protected groups. We describe why lexicase selection is well suited to pressure models to perform well across the potentially infinitely many subgroups over which fairness is desired. We use a recent genetic programming approach to construct models on four datasets for which fairness constraints are necessary, and empirically compare performance to prior methods utilizing game-theoretic solutions. Methods are assessed based on their ability to generate trade-offs of subgroup fairness and accuracy that are Pareto optimal. The result show that genetic programming methods in general, and random search in particular, are well suited to this task.
Formalising the Optimised Link State Routing Protocol
Venue: arXiv cs.NI 2020
Year: 2020

Routing protocol specifications are traditionally written in plain English. Often this yields ambiguities, inaccuracies or even contradictions. Formal methods techniques, such as process algebras, avoid these problems, thus leading to more precise and verifiable descriptions of protocols. In this paper we use the timed process algebra T-AWN for modelling the Optimised Link State Routing protocol (OLSR) version 2.
Neural Hair Rendering
Venue: arXiv cs.CV 2020
Year: 2020

In this paper, we propose a generic neural-based hair rendering pipeline that can synthesize photo-realistic images from virtual 3D hair models. Unlike existing supervised translation methods that require model-level similarity to preserve consistent structure representation for both real images and fake renderings, our method adopts an unsupervised solution to work on arbitrary hair models. The key component of our method is a shared latent space to encode appearance-invariant structure information of both domains, which generates realistic renderings conditioned by extra appearance inputs. This is achieved by domain-specific pre-disentangled structure representation, partially shared domain encoder layers and a structure discriminator. We also propose a simple yet effective temporal conditioning method to enforce consistency for video sequence generation. We demonstrate the superiority of our method by testing it on a large number of portraits and comparing it with alternative baselines and state-of-the-art unsupervised image translation methods.