OpenRaiser/Intern-Atlas huggingface dataset:
Retrieve and Refine: Exemplar-based Neural Comment Generation
Venue: arXiv cs.SE 2019
Year: 2019

Code comment generation is a crucial task in the field of automatic software development. Most previous neural comment generation systems used an encoder-decoder neural network and encoded only information from source code as input. Software reuse is common in software development. However, this feature has not been introduced to existing systems. Inspired by the traditional IR-based approaches, we propose to use the existing comments of similar source code as exemplars to guide the comment generation process. Based on an open source search engine, we first retrieve a similar code and treat its comment as an exemplar. Then we applied a seq2seq neural network to conduct an exemplar-based comment generation. We evaluate our approach on a large-scale Java corpus, and experimental results demonstrate that our model significantly outperforms the state-of-the-art methods.
Low Complexity Channel Model for Mobility Investigations in 5G Networks
Venue: arXiv cs.NI 2019
Year: 2019

Millimeter-wave has become an integral part of 5G networks to meet the ever-increasing demand for user data throughput. Employing higher carrier frequencies introduces new challenges for the propagation channel such as higher path loss and rapid signal degradations. On the other hand, higher frequencies allow deployment of small-sized antenna elements that enable beamforming. To investigate user mobility under these new propagation conditions, a proper model is needed that captures spatial and temporal characteristics of the channel in beamformed networks. Current channel models that have been developed for 5G networks are computationally inefficient and lead to infeasible simulation time for most user mobility simulations. In this paper, we present a simplified channel model that captures the spatial and temporal characteristics of the 5G propagation channel and runs in feasible simulation time. To this end, coherence time and path diversity originating from fully fledged Geometry based Stochastic Channel Model (GSCM) are analyzed and adopted in Jakes channel model. Furthermore, the deviation of multipath beamforming gain from single ray beamforming gain is analyzed and a regression curve is obtained to be used in the system-level simulations. We show through simulations that the proposed simplified channel model leads to mobility results comparable to Jakes model for high path diversity. Moreover, the multi-path beamforming gain increases the interference in the system and in turn number of mobility failures.
Unmanned Aerial Vehicles in Logistics: Efficiency Gains and Communication Performance of Hybrid Combinations of Ground and Aerial Vehicles
Venue: arXiv cs.NI 2019
Year: 2019

Unmanned Aerial Vehicles (UAVs) have drastically gained popularity in various Intelligent Transportation System (ITS) applications to improve the safety and efficiency of transportation systems. In this context, the combination of ground vehicles, such as delivery trucks, with drones to assist in the last mile pick-up and delivery of the parcels has been recently proposed. While aerial vehicles promise increased efficiency based on flexible routes and parallelized operation, highly reliable wireless communication is also required for the control and coordination of potentially many drones acting in a self-organized way. In this paper, we analyze the improvements procured by drone usage in parcel delivery compared to traditional delivery and propose a simulation framework to further quantify the efficiency gains of the parcel delivery logistics and to analyze the performance of different wireless communications options. To this end, we consider a heterogeneous vehicle routing problem with various constraints. We consider two approaches regarding the dispatching and recovery of drones and evaluate their benefits as opposed to parcel delivery with a classic truck only. Furthermore, we compare two networking technologies for enabling coordination of the self-organizing teams of drones with a realistically modeled environment: one approach relying on base station oriented Long Term Evolution (LTE) vs. a more decentralized Cellular Vehicle-to-Everything (C-V2X) solution. The results show time savings of nearly 40% can be achieved through drone usage and that the negative impact of urban shadowing on network communications in the base station oriented LTE approach can be compensated by leveraging decentralized C-V2X communications
Estimator Vectors: OOV Word Embeddings based on Subword and Context Clue Estimates
Venue: arXiv cs.CL 2019
Year: 2019

Semantic representations of words have been successfully extracted from unlabeled corpuses using neural network models like word2vec. These representations are generally high quality and are computationally inexpensive to train, making them popular. However, these approaches generally fail to approximate out of vocabulary (OOV) words, a task humans can do quite easily, using word roots and context clues. This paper proposes a neural network model that learns high quality word representations, subword representations, and context clue representations jointly. Learning all three types of representations together enhances the learning of each, leading to enriched word vectors, along with strong estimates for OOV words, via the combination of the corresponding context clue and subword embeddings. Our model, called Estimator Vectors (EV), learns strong word embeddings and is competitive with state of the art methods for OOV estimation.
Opinion aspect extraction in Dutch childrens diary entries
Venue: arXiv cs.CL 2019
Year: 2019

Aspect extraction can be used in dialogue systems to understand the topic of opinionated text. Expressing an empathetic reaction to an opinion can strengthen the bond between a human and, for example, a robot. The aim of this study is three-fold: 1. create a new annotated dataset for both aspect extraction and opinion words for Dutch childrens language, 2. acquire aspect extraction results for this task and 3. improve current results for aspect extraction in Dutch reviews. This was done by training a deep learning Gated Recurrent Unit (GRU) model, originally developed for an English review dataset, on Dutch restaurant review data to classify both opinion words and their respective aspects. We obtained state-of-the-art performance on the Dutch restaurant review dataset. Additionally, we acquired aspect extraction results for the Dutch childrens dataset. Since the model was trained on standardised language, these results are quite promising.
Application of integral equations to simulate local fields in carbon nanotube reinforced composites
Venue: arXiv cs.CE 2019
Year: 2019

We consider the steady heat conduction problem within a thermal isotropic and homogeneous square ring composite reinforced by non-overlapping and randomly distributed carbon nanotubes (CNTs). We treat the CNTs as rigid line inclusions and assume their temperature distribution to be fixed to an undetermined constant value along each line. We suppose also that the temperature distribution is known on the outer boundary and that there is no heat flux through the inner square. The equations for the temperature distribution are governed by the two-dimensional Laplace equation with mixed Dirichlet- Neumann boundary conditions. This boundary value problem is solved using a boundary integral equation method. We demonstrate the performance of our approach through four numerical examples with small and large numbers of CNTs and different side length of the inner square.
Learning Humanoid Robot Running Skills through Proximal Policy Optimization
Venue: arXiv cs.RO 2019
Year: 2019

In the current level of evolution of Soccer 3D, motion control is a key factor in team's performance. Recent works takes advantages of model-free approaches based on Machine Learning to exploit robot dynamics in order to obtain faster locomotion skills, achieving running policies and, therefore, opening a new research direction in the Soccer 3D environment. In this work, we present a methodology based on Deep Reinforcement Learning that learns running skills without any prior knowledge, using a neural network whose inputs are related to robot's dynamics. Our results outperformed the previous state-of-the-art sprint velocity reported in Soccer 3D literature by a significant margin. It also demonstrated improvement in sample efficiency, being able to learn how to run in just few hours. We reported our results analyzing the training procedure and also evaluating the policies in terms of speed, reliability and human similarity. Finally, we presented key factors that lead us to improve previous results and shared some ideas for future work.
Development and evaluation of an open-source, machine learning-based average annual daily traffic estimation software
Venue: arXiv cs.SE 2019
Year: 2019

Traditionally, Departments of Transportation (DOTs) use the factor-based model to estimate Annual Average Daily Traffic (AADT) from short-term traffic counts. The expansion factors, derived from the permanent traffic count stations, are applied to the short-term counts for AADT estimation. The inherent challenges of the factor-based method (i.e., grouping the count stations, applying proper expansion factors) make the estimated AADT values erroneous. Based on a survey conducted by the authors, 97% of the 39 public transportation agencies use the factor-based AADT estimation model, and these agencies face the aforementioned challenges while using factor-based models to estimate AADT. To derive a more accurate AADT, this paper presents the "estimAADTion" software, which is an open-source software developed based on a machine learning method called support vector regression (SVR) for estimating AADT using 24-hour short-term count data. DOTs conduct short-term counts at different locations periodically. This software has been designed to estimate AADT at a particular location from the short-term counts collected at those locations. In order to estimate AADT from short-term counts, the software uses data from permanent count stations to train the SVR model. The performance of the "estimAADTion" software is validated using the short-term count data from South Carolina. The Mean Absolute Percentage Error (MAPE) of the AADT estimated from the software is 3%, while the factor-based method produces a MAPE value of 6%.
Breast Anatomy Enriched Tumor Saliency Estimation
Venue: arXiv cs.CV 2019
Year: 2019

Breast cancer investigation is of great significance, and developing tumor detection methodologies is a critical need. However, it is a challenging task for breast ultrasound due to the complicated breast structure and poor quality of the images. In this paper, we propose a novel tumor saliency estimation model guided by enriched breast anatomy knowledge to localize the tumor. Firstly, the breast anatomy layers are generated by a deep neural network. Then we refine the layers by integrating a non-semantic breast anatomy model to solve the problems of incomplete mammary layers. Meanwhile, a new background map generation method weighted by the semantic probability and spatial distance is proposed to improve the performance. The experiment demonstrates that the proposed method with the new background map outperforms four state-of-the-art TSE models with increasing 10% of F_meansure on the BUS public dataset.
Mimicking Networks Parameterized by Connectivity
Venue: arXiv cs.DS 2019
Year: 2019

Given a graph $G=(V,E)$, capacities $w(e)$ on edges, and a subset of terminals $\mathcal{T} \subseteq V: |\mathcal{T}| = k$, a mimicking network for $(G,\mathcal{T})$ is a graph $(H,w')$ that contains copies of $\mathcal{T}$ and preserves the value of minimum cuts separating any subset $A, B \subseteq \mathcal{T}$ of terminals. Mimicking networks of size $2^{2^k}$ are known to exist and can be constructed algorithmically, while the best known lower bound is $2^{Ω(k)}$; therefore, an exponential size is required if one aims at preserving cuts exactly. In this paper, we study mimicking networks that preserve connectivity of the graph exactly up to the value of $c$, where $c$ is a parameter. This notion of mimicking network is sufficient for some applications, as we will elaborate. We first show that a mimicking of size $3^c \cdot k$ exists, that is, we can preserve cuts with small capacity using a network of size linear in $k$. Next, we show an algorithm that finds such a mimicking network in time $2^{O(c^2)} \operatorname{poly}(m)$.
Trojan Attacks on Wireless Signal Classification with Adversarial Machine Learning
Venue: arXiv cs.NI 2019
Year: 2019

We present a Trojan (backdoor or trapdoor) attack that targets deep learning applications in wireless communications. A deep learning classifier is considered to classify wireless signals using raw (I/Q) samples as features and modulation types as labels. An adversary slightly manipulates training data by inserting Trojans (i.e., triggers) to only few training data samples by modifying their phases and changing the labels of these samples to a target label. This poisoned training data is used to train the deep learning classifier. In test (inference) time, an adversary transmits signals with the same phase shift that was added as a trigger during training. While the receiver can accurately classify clean (unpoisoned) signals without triggers, it cannot reliably classify signals poisoned with triggers. This stealth attack remains hidden until activated by poisoned inputs (Trojans) to bypass a signal classifier (e.g., for authentication). We show that this attack is successful over different channel conditions and cannot be mitigated by simply preprocessing the training and test data with random phase variations. To detect this attack, activation based outlier detection is considered with statistical as well as clustering techniques. We show that the latter one can detect Trojan attacks even if few samples are poisoned.
Sidebar: Scratchpad Based Communication Between CPUs and Accelerators
Venue: arXiv cs.AR 2019
Year: 2019

Hardware accelerators for neural networks have shown great promise for both performance and power. These accelerators are at their most efficient when optimized for a fixed functionality. But this inflexibility limits the longevity of the hardware itself as the underlying neural network algorithms and structures undergo improvements and changes. We propose and evaluate a flexible design paradigm for accelerators with a close coordination with host processors. The relatively static matrix operations are implemented in specialized accelerators while fast-evolving functions, such as activations, are computed on the host processor. This architecture is enabled by a low latency shared buffer we call Sidebar. Sidebar memory is shared between the accelerator and host, exists outside of program address space and holds intermediate data only. We show that a generalised DMA dependent flexible accelerator design performs poorly in both perf and energy as compared to an equivalent fixed function accelerator. Sidebar based accelerator design achieves near identical performance and energy to equivalent fixed function accelerator while still providing all the flexibility of computing activations on the host processor.
Relation Module for Non-answerable Prediction on Question Answering
Venue: arXiv cs.CL 2019
Year: 2019

Machine reading comprehension(MRC) has attracted significant amounts of research attention recently, due to an increase of challenging reading comprehension datasets. In this paper, we aim to improve a MRC model's ability to determine whether a question has an answer in a given context (e.g. the recently proposed SQuAD 2.0 task). Our solution is a relation module that is adaptable to any MRC model. The relation module consists of both semantic extraction and relational information. We first extract high level semantics as objects from both question and context with multi-head self-attentive pooling. These semantic objects are then passed to a relation network, which generates relationship scores for each object pair in a sentence. These scores are used to determine whether a question is non-answerable. We test the relation module on the SQuAD 2.0 dataset using both BiDAF and BERT models as baseline readers. We obtain 1.8% gain of F1 on top of the BiDAF reader, and 1.0% on top of the BERT base model. These results show the effectiveness of our relation module on MRC
GF + MMT = GLF -- From Language to Semantics through LF
Venue: arXiv cs.CL 2019
Year: 2019

These days, vast amounts of knowledge are available online, most of it in written form. Search engines help us access this knowledge, but aggregating, relating and reasoning with it is still a predominantly human effort. One of the key challenges for automated reasoning based on natural-language texts is the need to extract meaning (semantics) from texts. Natural language understanding (NLU) systems describe the conversion from a set of natural language utterances to terms in a particular logic. Tools for the co-development of grammar and target logic are currently largely missing. We will describe the Grammatical Logical Framework (GLF), a combination of two existing frameworks, in which large parts of a symbolic, rule-based NLU system can be developed and implemented: the Grammatical Framework (GF) and MMT. GF is a tool for syntactic analysis, generation, and translation with complex natural language grammars and MMT can be used to specify logical systems and to represent knowledge in them. Combining these tools is possible, because they are based on compatible logical frameworks: Martin-Löf type theory and LF. The flexibility of logical frameworks is needed, as NLU research has not settled on a particular target logic for meaning representation. Instead, new logics are developed all the time to handle various language phenomena. GLF allows users to develop the logic and the language parsing components in parallel, and to connect them for experimentation with the entire pipeline.
Aggregation Signature for Small Object Tracking
Venue: arXiv cs.CV 2019
Year: 2019

Small object tracking becomes an increasingly important task, which however has been largely unexplored in computer vision. The great challenges stem from the facts that: 1) small objects show extreme vague and variable appearances, and 2) they tend to be lost easier as compared to normal-sized ones due to the shaking of lens. In this paper, we propose a novel aggregation signature suitable for small object tracking, especially aiming for the challenge of sudden and large drift. We make three-fold contributions in this work. First, technically, we propose a new descriptor, named aggregation signature, based on saliency, able to represent highly distinctive features for small objects. Second, theoretically, we prove that the proposed signature matches the foreground object more accurately with a high probability. Third, experimentally, the aggregation signature achieves a high performance on multiple datasets, outperforming the state-of-the-art methods by large margins. Moreover, we contribute with two newly collected benchmark datasets, i.e., small90 and small112, for visually small object tracking. The datasets will be available in https://github.com/bczhangbczhang/.
Man is to Person as Woman is to Location: Measuring Gender Bias in Named Entity Recognition
Venue: arXiv cs.IR 2019
Year: 2019

We study the bias in several state-of-the-art named entity recognition (NER) models---specifically, a difference in the ability to recognize male and female names as PERSON entity types. We evaluate NER models on a dataset containing 139 years of U.S. census baby names and find that relatively more female names, as opposed to male names, are not recognized as PERSON entities. We study the extent of this bias in several NER systems that are used prominently in industry and academia. In addition, we also report a bias in the datasets on which these models were trained. The result of this analysis yields a new benchmark for gender bias evaluation in named entity recognition systems. The data and code for the application of this benchmark will be publicly available for researchers to use.
Superposition as Data Augmentation using LSTM and HMM in Small Training Sets
Venue: arXiv cs.LG 2019
Year: 2019

Considering audio and image data as having quantum nature (data are represented by density matrices), we achieved better results on training architectures such as 3-layer stacked LSTM and HMM by mixing training samples using superposition augmentation and compared with plain default training and mix-up augmentation. This augmentation technique originates from the mix-up approach but provides more solid theoretical reasoning based on quantum properties. We achieved 3% improvement (from 68% to 71%) by using 38% lesser number of training samples in Russian audio-digits recognition task and 7,16% better accuracy than mix-up augmentation by training only 500 samples using HMM on the same task. Also, we achieved 1.1% better accuracy than mix-up on first 900 samples in MNIST using 3-layer stacked LSTM.
Fixed-Confidence Guarantees for Bayesian Best-Arm Identification
Venue: arXiv cs.LG 2019
Year: 2019

We investigate and provide new insights on the sampling rule called Top-Two Thompson Sampling (TTTS). In particular, we justify its use for fixed-confidence best-arm identification. We further propose a variant of TTTS called Top-Two Transportation Cost (T3C), which disposes of the computational burden of TTTS. As our main contribution, we provide the first sample complexity analysis of TTTS and T3C when coupled with a very natural Bayesian stopping rule, for bandits with Gaussian rewards, solving one of the open questions raised by Russo (2016). We also provide new posterior convergence results for TTTS under two models that are commonly used in practice: bandits with Gaussian and Bernoulli rewards and conjugate priors.
Toward Co-existing Database Schemas based on Bidirectional Transformation
Venue: arXiv cs.DB 2019
Year: 2019

According to strong demands for rapid and reliable software delivery, co-existing database schema versions with multiple application versions are reality to contribute them. Current database management systems do not support co-existing schema versions in one database. Although a design of co-existing schema based on updatable view tables was previously proposed, its flexibility is limited due to pre-defined several restrictions to achieve data synchronization among schemas and handling independent unsynchronized data in each schema. In this preliminary report, we present a new approach for co-existing schemas based on bidirectional transformation. We explain the required properties to realize co-existing schemas, bidirectionality and totality. We show that the co-existing schemas can be implemented systematically by applying putback-based bidirectional transformation to satisfy both the bidirectionality and the totality. While the bidirectionality can be satisfied by applying bidirectional transformation, to satisfy the totality, extra functions need to be introduced. How to derive these extra functions is presented.
On the complexity of colouring antiprismatic graphs
Venue: arXiv cs.DM 2019
Year: 2019

A graph G is prismatic if for every triangle T of G, every vertex of G not in T has a unique neighbour in T. The complement of a prismatic graph is called \emph{antiprismatic}. The complexity of colouring antiprismatic graphs is still unknown. Equivalently, the complexity of the clique cover problem in prismatic graphs is not known. Chudnovsky and Seymour gave a full structural description of prismatic graphs. They showed that the class can be divided into two subclasses: the orientable prismatic graphs, and the non-orientable prismatic graphs. We give a polynomial time algorithm that solves the clique cover problem in every non-orientable prismatic graph. It relies on the the structural description and on later work of Javadi and Hajebi. We give a polynomial time algorithm which solves the vertex-disjoint triangles problem for every prismatic graph. It does not rely on the structural description.