OpenRaiser/Intern-Atlas huggingface dataset:
Low-dimensional Semantic Space: from Text to Word Embedding
Venue: arXiv cs.CL 2019
Year: 2019

This article focuses on the study of Word Embedding, a feature-learning technique in Natural Language Processing that maps words or phrases to low-dimensional vectors. Beginning with the linguistic theories concerning contextual similarities - "Distributional Hypothesis" and "Context of Situation", this article introduces two ways of numerical representation of text: One-hot and Distributed Representation. In addition, this article presents statistical-based Language Models(such as Co-occurrence Matrix and Singular Value Decomposition) as well as Neural Network Language Models (NNLM, such as Continuous Bag-of-Words and Skip-Gram). This article also analyzes how Word Embedding can be applied to the study of word-sense disambiguation and diachronic linguistics.
Cryptanalysis of a new version of the MOR scheme
Venue: arXiv cs.CR 2019
Year: 2019

We show that an attack based on the linear decomposition method introduced by the author can be efficiently applied to the new version of the MOR scheme proposed in \cite{BMSS}. We draw attention to some inaccuracies in the description of this version. We show how the action of an exponent of a given automorphism (for example, the action of its inverse) can be calculated, and we also show how the unknown exponent of automorphism can be calculated if we go over to the corresponding linear transformation. This method can be applied to different matrix groups over an arbitrary constructive field. It does not depend on the specific properties of the underlined matrix group. The considered problem is reduced in probabilistic polynomial time to the similar problem in small extensions of the underlined field.
Potential Applications of Machine Learning at Multidisciplinary Medical Team Meetings
Venue: arXiv cs.AI 2019
Year: 2019

While machine learning (ML) systems have produced great advances in several domains, their use in support of complex cooperative work remains a research challenge. A particularly challenging setting, and one that may benefit from ML support is the work of multidisciplinary medical teams (MDTs). This paper focuses on the activities performed during the multidisciplinary medical team meeting (MDTM), reviewing their main characteristics in light of a longitudinal analysis of several MDTs in a large teaching hospital over a period of ten years and of our development of ML methods to support MDTMs, and identifying opportunities and possible pitfalls for the use of ML to support MDTMs.
The Bow-Tie Centrality: A Novel Measure for Directed and Weighted Networks with an Intrinsic Node Property
Venue: arXiv cs.SI 2019
Year: 2019

Today, there exist many centrality measures for assessing the importance of nodes in a network as a function of their position and the underlying topology. One class of such measures builds on eigenvector centrality, where the importance of a node is derived from the importance of its neighboring nodes. For directed and weighted complex networks, where the nodes can carry some intrinsic property value, there have been centrality measures proposed that are variants of eigenvector centrality. However, these expressions all suffer from shortcomings. Here, an extension of such centrality measures is presented that remedies all previously encountered issues. While similar improved centrality measures have been proposed as algorithmic recipes, the novel quantity that is presented here is a purely analytical expression, only utilizing the adjacency matrix and the vector of node values. The derivation of the new centrality measure is motivated in detail. Specifically, the centrality itself is ideal for the analysis of directed and weighted networks (with node properties) displaying a bow-tie topology. The novel bow-tie centrality is then computed for a unique and extensive real-world data set, coming from economics. It is shown how the bow-tie centrality assesses the relevance of nodes similarly to other eigenvector centrality measures, while not being plagued by their drawbacks in the presence of cycles in the network.
Spot Evasion Attacks: Adversarial Examples for License Plate Recognition Systems with Convolutional Neural Networks
Venue: arXiv cs.CV 2019
Year: 2019

Recent studies have shown convolution neural networks (CNNs) for image recognition are vulnerable to evasion attacks with carefully manipulated adversarial examples. Previous work primarily focused on how to generate adversarial examples closed to source images, by introducing pixel-level perturbations into the whole or specific part of images. In this paper, we propose an evasion attack on CNN classifiers in the context of License Plate Recognition (LPR), which adds predetermined perturbations to specific regions of license plate images, simulating some sort of naturally formed spots (such as sludge, etc.). Therefore, the problem is modeled as an optimization process searching for optimal perturbation positions, which is different from previous work that consider pixel values as decision variables. Notice that this is a complex nonlinear optimization problem, and we use a genetic-algorithm based approach to obtain optimal perturbation positions. In experiments, we use the proposed algorithm to generate various adversarial examples in the form of rectangle, circle, ellipse and spots cluster. Experimental results show that these adversarial examples are almost ignored by human eyes, but can fool HyperLPR with high attack success rate over 93%. Therefore, we believe that this kind of spot evasion attacks would pose a great threat to current LPR systems, and needs to be investigated further by the security community.
Privacy for Free: Communication-Efficient Learning with Differential Privacy Using Sketches
Venue: arXiv cs.LG 2019
Year: 2019

Communication and privacy are two critical concerns in distributed learning. Many existing works treat these concerns separately. In this work, we argue that a natural connection exists between methods for communication reduction and privacy preservation in the context of distributed machine learning. In particular, we prove that Count Sketch, a simple method for data stream summarization, has inherent differential privacy properties. Using these derived privacy guarantees, we propose a novel sketch-based framework (DiffSketch) for distributed learning, where we compress the transmitted messages via sketches to simultaneously achieve communication efficiency and provable privacy benefits. Our evaluation demonstrates that DiffSketch can provide strong differential privacy guarantees (e.g., $\varepsilon$= 1) and reduce communication by 20-50x with only marginal decreases in accuracy. Compared to baselines that treat privacy and communication separately, DiffSketch improves absolute test accuracy by 5%-50% while offering the same privacy guarantees and communication compression.
On Batch Bayesian Optimization
Venue: arXiv cs.LG 2019
Year: 2019

We present two algorithms for Bayesian optimization in the batch feedback setting, based on Gaussian process upper confidence bound and Thompson sampling approaches, along with frequentist regret guarantees and numerical results.
Eye Semantic Segmentation with a Lightweight Model
Venue: arXiv cs.CV 2019
Year: 2019

In this paper, we present a multi-class eye segmentation method that can run the hardware limitations for real-time inference. Our approach includes three major stages: get a grayscale image from the input, segment three distinct eye region with a deep network, and remove incorrect areas with heuristic filters. Our model based on the encoder decoder structure with the key is the depthwise convolution operation to reduce the computation cost. We experiment on OpenEDS, a large scale dataset of eye images captured by a head-mounted display with two synchronized eye facing cameras. We achieved the mean intersection over union (mIoU) of 94.85% with a model of size 0.4 megabytes. The source code are available https://github.com/th2l/Eye_VR_Segmentation
On the Effectiveness of the Pooling Methods for Biomedical Relation Extraction with Deep Learning
Venue: arXiv cs.CL 2019
Year: 2019

Deep learning models have achieved state-of-the-art performances on many relation extraction datasets. A common element in these deep learning models involves the pooling mechanisms where a sequence of hidden vectors is aggregated to generate a single representation vector, serving as the features to perform prediction for RE. Unfortunately, the models in the literature tend to employ different strategies to perform pooling for RE, leading to the challenge to determine the best pooling mechanism for this problem, especially in the biomedical domain. In order to answer this question, in this work, we conduct a comprehensive study to evaluate the effectiveness of different pooling mechanisms for the deep learning models in biomedical RE. The experimental results suggest that dependency-based pooling is the best pooling strategy for RE in the biomedical domain, yielding the state-of-the-art performance on two benchmark datasets for this problem.
Temporal Action Localization using Long Short-Term Dependency
Venue: arXiv cs.CV 2019
Year: 2019

Temporal action localization in untrimmed videos is an important but difficult task. Difficulties are encountered in the application of existing methods when modeling temporal structures of videos. In the present study, we developed a novel method, referred to as Gemini Network, for effective modeling of temporal structures and achieving high-performance temporal action localization. The significant improvements afforded by the proposed method are attributable to three major factors. First, the developed network utilizes two subnets for effective modeling of temporal structures. Second, three parallel feature extraction pipelines are used to prevent interference between the extractions of different stage features. Third, the proposed method utilizes auxiliary supervision, with the auxiliary classifier losses affording additional constraints for improving the modeling capability of the network. As a demonstration of its effectiveness, the Gemini Network was used to achieve state-of-the-art temporal action localization performance on two challenging datasets, namely, THUMOS14 and ActivityNet.
Supervised level-wise pretraining for recurrent neural network initialization in multi-class classification
Venue: arXiv cs.LG 2019
Year: 2019

Recurrent Neural Networks (RNNs) can be seriously impacted by the initial parameters assignment, which may result in poor generalization performances on new unseen data. With the objective to tackle this crucial issue, in the context of RNN based classification, we propose a new supervised layer-wise pretraining strategy to initialize network parameters. The proposed approach leverages a data-aware strategy that sets up a taxonomy of classification problems automatically derived by the model behavior. To the best of our knowledge, despite the great interest in RNN-based classification, this is the first data-aware strategy dealing with the initialization of such models. The proposed strategy has been tested on four benchmarks coming from two different domains, i.e., Speech Recognition and Remote Sensing. Results underline the significance of our approach and point out that data-aware strategies positively support the initialization of Recurrent Neural Network based classification models.
Wearable Affective Life-Log System for Understanding Emotion Dynamics in Daily Life
Venue: arXiv cs.AI 2019
Year: 2019

Past research on recognizing human affect has made use of a variety of physiological sensors in many ways. Nonetheless, how affective dynamics are influenced in the context of human daily life has not yet been explored. In this work, we present a wearable affective life-log system (ALIS), that is robust as well as easy to use in daily life to detect emotional changes and determine their cause-and-effect relationship on users' lives. The proposed system records how a user feels in certain situations during long-term activities with physiological sensors. Based on the long-term monitoring, the system analyzes how the contexts of the user's life affect his/her emotion changes. Furthermore, real-world experimental results demonstrate that the proposed wearable life-log system enables us to build causal structures to find effective stress relievers suited to every stressful situation in school life.
What does a network layer hear? Analyzing hidden representations of end-to-end ASR through speech synthesis
Venue: arXiv cs.CL 2019
Year: 2019

End-to-end speech recognition systems have achieved competitive results compared to traditional systems. However, the complex transformations involved between layers given highly variable acoustic signals are hard to analyze. In this paper, we present our ASR probing model, which synthesizes speech from hidden representations of end-to-end ASR to examine the information maintain after each layer calculation. Listening to the synthesized speech, we observe gradual removal of speaker variability and noise as the layer goes deeper, which aligns with the previous studies on how deep network functions in speech recognition. This paper is the first study analyzing the end-to-end speech recognition model by demonstrating what each layer hears. Speaker verification and speech enhancement measurements on synthesized speech are also conducted to confirm our observation further.
Rate-Power Region of SWIPT Systems Employing Nonlinear Energy Harvester Circuits with Memory
Venue: arXiv cs.IT 2019
Year: 2019

In this paper, we study the rate-power region of a simultaneous wireless information and power transfer (SWIPT) system where a transmitter (TX) broadcasts a common signal to an information receiver (IR) and an energy harvester (EH). Since practical EH circuits include a reactive element as part of their signal rectifier and the voltage on this element cannot drop or rise instantaneously, the EH circuit has memory. We model the memory effect of the EH by a Markov reward chain. Furthermore, since an analytical model that includes all non-linear and memory effects of the EH circuit is not available, we employ a deep neural network (DNN) to model the Markov chain evolution. We formulate an optimization problem to determine the rate-power region of the considered SWIPT system and propose an iterative algorithm based on sequential quadratic programming (SQP) to solve it. Our numerical results show that the optimal input distribution and the rate-power region depend on both the input power level at the EH and the symbol duration.
Modeling and Performance Analysis in Cache-enabled Millimeter Wave HetNets with Access and Backhaul Integration
Venue: arXiv cs.NI 2019
Year: 2019

Recently, a mmWave-based access and backhaul integration HetNetsarchitecture (mABHetNets) has been envisioned to provide the high link capacity in both access and backhaul links. In mABHetNets, the access link and the backhaul link will share the same mm-wave spectral resources to serve cellular users as well as to maintain the backhaul capacity of the SBSs. However, a large spectrum bandwidth are occupied by the backhaul link to maintain the backhaul link capacity. Up to 50\% mmWave spectrum resource will be used in backhaul link to satisfy the high speed data traffic. Such "\emph{spectrum occupation}" of backhaul link has restricted network performance to achieve a better possible improvement. To overcome spectrum occupation problem in the existing mABHetNets, involving caches in the mABHetNets brings a opportunity. When the backhaul traffic is offloaded by caching popular fils in the SBSs and the corresponding mmWave spectrum can be transfered to the access link by the bandwidth partition. Then with more spectrum for access link, both the network throughout and the coverage ability are increased significantly. However, the caching power consumption will be increased and the EE will be decreased. Therefore, we analyze the througput and EE in mABHetNets by stochastic geometry tool. Extensive numerical and simulations results are carried out to validate our work.
A Comprehensive Study on Pedestrians' Evacuation
Venue: arXiv cs.MA 2019
Year: 2019

Human beings face threats because of unexpected happenings, which can be avoided through an adequate crisis evacuation plan, which is vital to stop wound and demise as its negative results. Consequently, different typical evacuation pedestrians have been created. Moreover, through applied research, these models for various applications, reproductions, and conditions have been examined to present an operational model. Furthermore, new models have been developed to cooperate with system evacuation in residential places in case of unexpected events. This research has taken into account an inclusive and a 'systematic survey of pedestrian evacuation' to demonstrate models methods by focusing on the applications' features, techniques, implications, and after that gather them under various types, for example, classical models, hybridized models, and generic model. The current analysis assists scholars in this field of study to write their forthcoming papers about it, which can suggest a novel structure to recent typical intelligent reproduction with novel features.
Optimal Adaptive Detection of Monotone Patterns
Venue: arXiv cs.DS 2019
Year: 2019

We investigate adaptive sublinear algorithms for detecting monotone patterns in an array. Given fixed $2 \leq k \in \mathbb{N}$ and $\varepsilon > 0$, consider the problem of finding a length-$k$ increasing subsequence in an array $f \colon [n] \to \mathbb{R}$, provided that $f$ is $\varepsilon$-far from free of such subsequences. Recently, it was shown that the non-adaptive query complexity of the above task is $Θ((\log n)^{\lfloor \log_2 k \rfloor})$. In this work, we break the non-adaptive lower bound, presenting an adaptive algorithm for this problem which makes $O(\log n)$ queries. This is optimal, matching the classical $Ω(\log n)$ adaptive lower bound by Fischer [2004] for monotonicity testing (which corresponds to the case $k=2$), and implying in particular that the query complexity of testing whether the longest increasing subsequence (LIS) has constant length is $Θ(\log n)$.
Analysing Coreference in Transformer Outputs
Venue: arXiv cs.CL 2019
Year: 2019

We analyse coreference phenomena in three neural machine translation systems trained with different data settings with or without access to explicit intra- and cross-sentential anaphoric information. We compare system performance on two different genres: news and TED talks. To do this, we manually annotate (the possibly incorrect) coreference chains in the MT outputs and evaluate the coreference chain translations. We define an error typology that aims to go further than pronoun translation adequacy and includes types such as incorrect word selection or missing words. The features of coreference chains in automatic translations are also compared to those of the source texts and human translations. The analysis shows stronger potential translationese effects in machine translated outputs than in human translations.
Spherical Text Embedding
Venue: arXiv cs.CL 2019
Year: 2019

Unsupervised text embedding has shown great power in a wide range of NLP tasks. While text embeddings are typically learned in the Euclidean space, directional similarity is often more effective in tasks such as word similarity and document clustering, which creates a gap between the training stage and usage stage of text embedding. To close this gap, we propose a spherical generative model based on which unsupervised word and paragraph embeddings are jointly learned. To learn text embeddings in the spherical space, we develop an efficient optimization algorithm with convergence guarantee based on Riemannian optimization. Our model enjoys high efficiency and achieves state-of-the-art performances on various text embedding tasks including word similarity and document clustering.
Understand customer reviews with less data and in short time: pretrained language representation and active learning
Venue: arXiv cs.CL 2019
Year: 2019

In this paper, we address customer review understanding problems by using supervised machine learning approaches, in order to achieve a fully automatic review aspects categorisation and sentiment analysis. In general, such supervised learning algorithms require domain-specific expert knowledge for generating high quality labeled training data, and the cost of labeling can be very high. To achieve an in-production customer review machine learning enabled analysis tool with only a limited amount of data and within a reasonable training data collection time, we propose to use pre-trained language representation to boost model performance and active learning framework for accelerating the iterative training process. The results show that with integration of both components, the fully automatic review analysis can be achieved at a much faster pace.
ElecSim: Monte-Carlo Open-Source Agent-Based Model to Inform Policy for Long-Term Electricity Planning
Venue: arXiv cs.MA 2019
Year: 2019

Due to the threat of climate change, a transition from a fossil-fuel based system to one based on zero-carbon is required. However, this is not as simple as instantaneously closing down all fossil fuel energy generation and replacing them with renewable sources -- careful decisions need to be taken to ensure rapid but stable progress. To aid decision makers, we present a new tool, ElecSim, which is an open-sourced agent-based modelling framework used to examine the effect of policy on long-term investment decisions in electricity generation. ElecSim allows non-experts to rapidly prototype new ideas. Different techniques to model long-term electricity decisions are reviewed and used to motivate why agent-based models will become an important strategic tool for policy. We motivate why an open-source toolkit is required for long-term electricity planning. Actual electricity prices are compared with our model and we demonstrate that the use of a Monte-Carlo simulation in the system improves performance by $52.5\%$. Further, using ElecSim we demonstrate the effect of a carbon tax to encourage a low-carbon electricity supply. We show how a £40 ($\$50$) per tonne of CO2 emitted would lead to 70% renewable electricity by 2050.
Autonomous Vehicles Meet the Physical World: RSS, Variability, Uncertainty, and Proving Safety (Expanded Version)
Venue: arXiv cs.RO 2019
Year: 2019

The Responsibility-Sensitive Safety (RSS) model offers provable safety for vehicle behaviors such as minimum safe following distance. However, handling worst-case variability and uncertainty may significantly lower vehicle permissiveness, and in some situations safety cannot be guaranteed. Digging deeper into Newtonian mechanics, we identify complications that result from considering vehicle status, road geometry and environmental parameters. An especially challenging situation occurs if these parameters change during the course of a collision avoidance maneuver such as hard braking. As part of our analysis, we expand the original RSS following distance equation to account for edge cases involving potential collisions mid-way through a braking process. We additionally propose a Micro-Operational Design Domain (μODD) approach to subdividing the operational space as a way of improving permissiveness. Confining probabilistic aspects of safety to μODD transitions permits proving safety (when possible) under the assumption that the system has transitioned to the correct μODD for the situation. Each μODD can additionally be used to encode system fault responses, take credit for advisory information (e.g., from vehicle-to-vehicle communication), and anticipate likely emergent situations.