OpenRaiser/Intern-Atlas huggingface dataset:
$\mathbb{Z}_2\mathbb{Z}_4$-Additive Cyclic Codes Are Asymptotically Good
Venue: arXiv cs.IT 2019
Year: 2019

We construct a class of $\mathbb{Z}_2\mathbb{Z}_4$-additive cyclic codes generated by pairs of polynomials, study their algebraic structures, and obtain the generator matrix of any code in the class. Using a probabilistic method, we prove that, for any positive real number $δ<1/3$ such that the entropy at $3δ/2$ is less than $1/2$, the probability that the relative minimal distance of a random code in the class is greater than $δ$ is almost $1$; and the probability that the rate of the random code equals to $1/3$ is also almost $1$. As an obvious consequence, the $\mathbb{Z}_2\mathbb{Z}_4$-additive cyclic codes are asymptotically good.
Multi-Scale RCNN Model for Financial Time-series Classification
Venue: arXiv cs.LG 2019
Year: 2019

Financial time-series classification (FTC) is extremely valuable for investment management. In past decades, it draws a lot of attention from a wide extent of research areas, especially Artificial Intelligence (AI). Existing researches majorly focused on exploring the effects of the Multi-Scale (MS) property or the Temporal Dependency (TD) within financial time-series. Unfortunately, most previous researches fail to combine these two properties effectively and often fall short of accuracy and profitability. To effectively combine and utilize both properties of financial time-series, we propose a Multi-Scale Temporal Dependent Recurrent Convolutional Neural Network (MSTD-RCNN) for FTC. In the proposed method, the MS features are simultaneously extracted by convolutional units to precisely describe the state of the financial market. Moreover, the TD and complementary across different scales are captured through a Recurrent Neural Network. The proposed method is evaluated on three financial time-series datasets which source from the Chinese stock market. Extensive experimental results indicate that our model achieves the state-of-the-art performance in trend classification and simulated trading, compared with classical and advanced baseline models.
SolidityCheck : Quickly Detecting Smart Contract Problems Through Regular Expressions
Venue: arXiv cs.SE 2019
Year: 2019

As a blockchain platform that has developed vigorously in recent years, Ethereum is different from Bitcoin in that it introduces smart contracts into blockchain.Solidity is one of the most mature and widely used smart contract programming language,which is used to write smart contracts and deploy them on blockchain. However, once the data in the blockchain is written, it cannot be modified. Ethereum smart contract is stored in the block chain, which makes the smart contract can no longer repair the code problems such as re-entrancy vulnerabilities or integer overflow problems. Currently, there still lacks of an efficient and effective approach for detecting these problems in Solidity. In this paper, we first classify all the possible problems in Solidity, then propose a smart contract problem detection approach for Solidity, namely SolidityCheck. The approach uses regular expressions to define the characteristics of problematic statements and uses regular matching and program instrumentation to prevent or detect problems. Finally, a large number of experiments is performed to show that SolidityCheck is superior to existing approaches.
What Do You Mean `Why?': Resolving Sluices in Conversations
Venue: arXiv cs.CL 2019
Year: 2019

In conversation, we often ask one-word questions such as `Why?' or `Who?'. Such questions are typically easy for humans to answer, but can be hard for computers, because their resolution requires retrieving both the right semantic frames and the right arguments from context. This paper introduces the novel ellipsis resolution task of resolving such one-word questions, referred to as sluices in linguistics. We present a crowd-sourced dataset containing annotations of sluices from over 4,000 dialogues collected from conversational QA datasets, as well as a series of strong baseline architectures.
MUSE: Parallel Multi-Scale Attention for Sequence to Sequence Learning
Venue: arXiv cs.CL 2019
Year: 2019

In sequence to sequence learning, the self-attention mechanism proves to be highly effective, and achieves significant improvements in many tasks. However, the self-attention mechanism is not without its own flaws. Although self-attention can model extremely long dependencies, the attention in deep layers tends to overconcentrate on a single token, leading to insufficient use of local information and difficultly in representing long sequences. In this work, we explore parallel multi-scale representation learning on sequence data, striving to capture both long-range and short-range language structures. To this end, we propose the Parallel MUlti-Scale attEntion (MUSE) and MUSE-simple. MUSE-simple contains the basic idea of parallel multi-scale sequence representation learning, and it encodes the sequence in parallel, in terms of different scales with the help from self-attention, and pointwise transformation. MUSE builds on MUSE-simple and explores combining convolution and self-attention for learning sequence representations from more different scales. We focus on machine translation and the proposed approach achieves substantial performance improvements over Transformer, especially on long sequences. More importantly, we find that although conceptually simple, its success in practice requires intricate considerations, and the multi-scale attention must build on unified semantic space. Under common setting, the proposed model achieves substantial performance and outperforms all previous models on three main machine translation tasks. In addition, MUSE has potential for accelerating inference due to its parallelism. Code will be available at https://github.com/lancopku/MUSE
Analysing Time-Stamped Co-Editing Networks in Software Development Teams using git2net
Venue: arXiv cs.SE 2019
Year: 2019

Data from software repositories have become an important foundation for the empirical study of software engineering processes. A recurring theme in the repository mining literature is the inference of developer networks capturing e.g. collaboration, coordination, or communication from the commit history of projects. Most of the studied networks are based on the co-authorship of software artefacts. Because this neglects detailed information on code changes and code ownership we introduce git2net, a scalable python software that facilitates the extraction of fine-grained co-editing networks in large git repositories. It uses text mining techniques to analyse the detailed history of textual modifications within files. We apply our tool in two case studies using GitHub repositories of multiple Open Source as well as a commercial software project. Specifically, we use data on more than 1.2 million commits and more than 25'000 developers to test a hypothesis on the relation between developer productivity and co-editing patterns in software teams. We argue that git2net opens up a massive new source of high-resolution data on human collaboration patterns that can be used to advance theory in empirical software engineering, computational social science, and organisational studies.
Towards FAIR protocols and workflows: The OpenPREDICT case study
Venue: arXiv cs.LG 2019
Year: 2019

It is essential for the advancement of science that scientists and researchers share, reuse and reproduce workflows and protocols used by others. The FAIR principles are a set of guidelines that aim to maximize the value and usefulness of research data, and emphasize a number of important points regarding the means by which digital objects are found and reused by others. The question of how to apply these principles not just to the static input and output data but also to the dynamic workflows and protocols that consume and produce them is still under debate and poses a number of challenges. In this paper we describe our inclusive and overarching approach to apply the FAIR principles to workflows and protocols and demonstrate its benefits. We apply and evaluate our approach on a case study that consists of making the PREDICT workflow, a highly cited drug repurposing workflow, open and FAIR. This includes FAIRification of the involved datasets, as well as applying semantic technologies to represent and store data about the detailed versions of the general protocol, of the concrete workflow instructions, and of their execution traces. A semantic model was proposed to better address these specific requirements and were evaluated by answering competency questions. This semantic model consists of classes and relations from a number of existing ontologies, including Workflow4ever, PROV, EDAM, and BPMN. This allowed us then to formulate and answer new kinds of competency questions. Our evaluation shows the high degree to which our FAIRified OpenPREDICT workflow now adheres to the FAIR principles and the practicality and usefulness of being able to answer our new competency questions.
Active Re-identification Attacks on Periodically Released Dynamic Social Graphs
Venue: arXiv cs.SI 2019
Year: 2019

Active re-identification attacks pose a serious threat to privacy-preserving social graph publication. Active attackers create fake accounts to build structural patterns in social graphs which can be used to re-identify legitimate users on published anonymised graphs, even without additional background knowledge. So far, this type of attacks has only been studied in the scenario where the inherently dynamic social graph is published once. In this paper, we present the first active re-identification attack in the more realistic scenario where a dynamic social graph is periodically published. The new attack leverages tempo-structural patterns for strengthening the adversary. Through a comprehensive set of experiments on real-life and synthetic dynamic social graphs, we show that our new attack substantially outperforms the most effective static active attack in the literature by increasing the success probability of re-identification by more than two times and efficiency by almost 10 times. Moreover, unlike the static attack, our new attack is able to remain at the same level of effectiveness and efficiency as the publication process advances. We conduct a study on the factors that may thwart our new attack, which can help design graph anonymising methods with a better balance between privacy and utility.
Agent Probing Interaction Policies
Venue: arXiv cs.MA 2019
Year: 2019

Reinforcement learning in a multi agent system is difficult because these systems are inherently non-stationary in nature. In such a case, identifying the type of the opposite agent is crucial and can help us address this non-stationary environment. We have investigated if we can employ some probing policies which help us better identify the type of the other agent in the environment. We've made a simplifying assumption that the other agent has a stationary policy that our probing policy is trying to approximate. Our work extends Environmental Probing Interaction Policy framework to handle multi agent environments.
Feedback Motion Planning for Long-Range Autonomous Underwater Vehicles
Venue: arXiv cs.RO 2019
Year: 2019

Ocean ecosystems have spatiotemporal variability and dynamic complexity that require a long-term deployment of an autonomous underwater vehicle for data collection. A new long-range autonomous underwater vehicle called Tethys is adapted to study different oceanic phenomena. Additionally, an ocean environment has external forces and moments along with changing water currents which are generally not considered in a vehicle kinematic model. In this scenario, it is not enough to generate a simple trajectory from an initial location to a goal location in an uncertain ocean as the vehicle can deviate from its intended trajectory. As such, we propose to compute a feedback plan that adapts the vehicle trajectory in the presence of any modeled or unmodeled uncertainties. In this work, we present a feedback motion planning method for the Tethys vehicle by combining a predictive ocean model and its kinematic modeling. Given a goal location, the Tethys kinematic model, and the water flow pattern, our method computes a feedback plan for the vehicle in a dynamic ocean environment that reduces its energy consumption. The computed feedback plan provides the optimal action for the Tethys vehicle to take from any location of the environment to reach the goal location considering its orientation. Our results based on actual ocean model prediction data demonstrate the applicability of our method.
A brief chronology of Virtual Reality
Venue: arXiv cs.HC 2019
Year: 2019

In this article, we are going to review a brief history of the field of Virtual Reality (VR), VR systems, and applications and discuss how they evolved. After that, we will familiarize ourselves with the essential components of VR experiences and common VR terminology. Finally, we discuss the evolution of ubiquitous VR as a subfield of VR and its current trends.
Temporal Reasoning via Audio Question Answering
Venue: arXiv cs.CL 2019
Year: 2019

Multimodal question answering tasks can be used as proxy tasks to study systems that can perceive and reason about the world. Answering questions about different types of input modalities stresses different aspects of reasoning such as visual reasoning, reading comprehension, story understanding, or navigation. In this paper, we use the task of Audio Question Answering (AQA) to study the temporal reasoning abilities of machine learning models. To this end, we introduce the Diagnostic Audio Question Answering (DAQA) dataset comprising audio sequences of natural sound events and programmatically generated questions and answers that probe various aspects of temporal reasoning. We adapt several recent state-of-the-art methods for visual question answering to the AQA task, and use DAQA to demonstrate that they perform poorly on questions that require in-depth temporal reasoning. Finally, we propose a new model, Multiple Auxiliary Controllers for Linear Modulation (MALiMo) that extends the recent Feature-wise Linear Modulation (FiLM) model and significantly improves its temporal reasoning capabilities. We envisage DAQA to foster research on AQA and temporal reasoning and MALiMo a step towards models for AQA.
Efficient Drone Mobility Support Using Reinforcement Learning
Venue: arXiv cs.IT 2019
Year: 2019

Flying drones can be used in a wide range of applications and services from surveillance to package delivery. To ensure robust control and safety of drone operations, cellular networks need to provide reliable wireless connectivity to drone user equipments (UEs). To date, existing mobile networks have been primarily designed and optimized for serving ground UEs, thus making the mobility support in the sky challenging. In this paper, a novel handover (HO) mechanism is developed for a cellular-connected drone system to ensure robust wireless connectivity and mobility support for drone-UEs. By leveraging tools from reinforcement learning, HO decisions are dynamically optimized using a Q-learning algorithm to provide an efficient mobility support in the sky. The results show that the proposed approach can significantly reduce (e.g., by 80%) the number of HOs, while maintaining connectivity, compared to the baseline HO scheme in which the drone always connects to the strongest cell.
Are Noisy Sentences Useless for Distant Supervised Relation Extraction?
Venue: arXiv cs.CL 2019
Year: 2019

The noisy labeling problem has been one of the major obstacles for distant supervised relation extraction. Existing approaches usually consider that the noisy sentences are useless and will harm the model's performance. Therefore, they mainly alleviate this problem by reducing the influence of noisy sentences, such as applying bag-level selective attention or removing noisy sentences from sentence-bags. However, the underlying cause of the noisy labeling problem is not the lack of useful information, but the missing relation labels. Intuitively, if we can allocate credible labels for noisy sentences, they will be transformed into useful training data and benefit the model's performance. Thus, in this paper, we propose a novel method for distant supervised relation extraction, which employs unsupervised deep clustering to generate reliable labels for noisy sentences. Specifically, our model contains three modules: a sentence encoder, a noise detector and a label generator. The sentence encoder is used to obtain feature representations. The noise detector detects noisy sentences from sentence-bags, and the label generator produces high-confidence relation labels for noisy sentences. Extensive experimental results demonstrate that our model outperforms the state-of-the-art baselines on a popular benchmark dataset, and can indeed alleviate the noisy labeling problem.
Learning Feature Interactions with Lorentzian Factorization Machine
Venue: arXiv cs.IR 2019
Year: 2019

Learning representations for feature interactions to model user behaviors is critical for recommendation system and click-trough rate (CTR) predictions. Recent advances in this area are empowered by deep learning methods which could learn sophisticated feature interactions and achieve the state-of-the-art result in an end-to-end manner. These approaches require large number of training parameters integrated with the low-level representations, and thus are memory and computational inefficient. In this paper, we propose a new model named "LorentzFM" that can learn feature interactions embedded in a hyperbolic space in which the violation of triangle inequality for Lorentz distances is available. To this end, the learned representation is benefited by the peculiar geometric properties of hyperbolic triangles, and result in a significant reduction in the number of parameters (20\% to 80\%) because all the top deep learning layers are not required. With such a lightweight architecture, LorentzFM achieves comparable and even materially better results than the deep learning methods such as DeepFM, xDeepFM and Deep \& Cross in both recommendation and CTR prediction tasks.
Parallel Distributed Logistic Regression for Vertical Federated Learning without Third-Party Coordinator
Venue: arXiv cs.LG 2019
Year: 2019

Federated Learning is a new distributed learning mechanism which allows model training on a large corpus of decentralized data owned by different data providers, without sharing or leakage of raw data. According to the characteristics of data dis-tribution, it could be usually classified into three categories: horizontal federated learning, vertical federated learning, and federated transfer learning. In this paper we present a solution for parallel dis-tributed logistic regression for vertical federated learning. As compared with existing works, the role of third-party coordinator is removed in our proposed solution. The system is built on the pa-rameter server architecture and aims to speed up the model training via utilizing a cluster of servers in case of large volume of training data. We also evaluate the performance of the parallel distributed model training and the experimental results show the great scalability of the system.
Factorized Multimodal Transformer for Multimodal Sequential Learning
Venue: arXiv cs.LG 2019
Year: 2019

The complex world around us is inherently multimodal and sequential (continuous). Information is scattered across different modalities and requires multiple continuous sensors to be captured. As machine learning leaps towards better generalization to real world, multimodal sequential learning becomes a fundamental research area. Arguably, modeling arbitrarily distributed spatio-temporal dynamics within and across modalities is the biggest challenge in this research area. In this paper, we present a new transformer model, called the Factorized Multimodal Transformer (FMT) for multimodal sequential learning. FMT inherently models the intramodal and intermodal (involving two or more modalities) dynamics within its multimodal input in a factorized manner. The proposed factorization allows for increasing the number of self-attentions to better model the multimodal phenomena at hand; without encountering difficulties during training (e.g. overfitting) even on relatively low-resource setups. All the attention mechanisms within FMT have a full time-domain receptive field which allows them to asynchronously capture long-range multimodal dynamics. In our experiments we focus on datasets that contain the three commonly studied modalities of language, vision and acoustic. We perform a wide range of experiments, spanning across 3 well-studied datasets and 21 distinct labels. FMT shows superior performance over previously proposed models, setting new state of the art in the studied datasets.
This Car is Mine!: Automobile Theft Countermeasure Leveraging Driver Identification with Generative Adversarial Networks
Venue: arXiv cs.CR 2019
Year: 2019

As a car becomes more connected, a countermeasure against automobile theft has become a significant task in the real world. To respond to automobile theft, data mining, biometrics, and additional authentication methods are proposed. Among current countermeasures, data mining method is one of the efficient ways to capture the owner driver's unique characteristics. To identify the owner driver from thieves, previous works applied various algorithms toward driving data. Such data mining methods utilized supervised learning, thus required labeled data set. However, it is unrealistic to gather and apply the thief's driving pattern. To overcome this problem, we propose driver identification method with GAN. GAN has merit to build identification model by learning the owner driver's data only. We trained GAN only with owner driver's data and used trained discriminator to identify the owner driver. From actual driving data, we evaluated our identification model recognizes the owner driver well. By ensembling various driver authentication methods with the proposed model, we expect industry can develop automobile theft countermeasures available in the real world.
Neuron Interaction Based Representation Composition for Neural Machine Translation
Venue: arXiv cs.CL 2019
Year: 2019

Recent NLP studies reveal that substantial linguistic information can be attributed to single neurons, i.e., individual dimensions of the representation vectors. We hypothesize that modeling strong interactions among neurons helps to better capture complex information by composing the linguistic properties embedded in individual neurons. Starting from this intuition, we propose a novel approach to compose representations learned by different components in neural machine translation (e.g., multi-layer networks or multi-head attention), based on modeling strong interactions among neurons in the representation vectors. Specifically, we leverage bilinear pooling to model pairwise multiplicative interactions among individual neurons, and a low-rank approximation to make the model computationally feasible. We further propose extended bilinear pooling to incorporate first-order representations. Experiments on WMT14 English-German and English-French translation tasks show that our model consistently improves performances over the SOTA Transformer baseline. Further analyses demonstrate that our approach indeed captures more syntactic and semantic information as expected.
PAG-Net: Progressive Attention Guided Depth Super-resolution Network
Venue: arXiv cs.CV 2019
Year: 2019

In this paper, we propose a novel method for the challenging problem of guided depth map super-resolution, called PAGNet. It is based on residual dense networks and involves the attention mechanism to suppress the texture copying problem arises due to improper guidance by RGB images. The attention module mainly involves providing the spatial attention to guidance image based on the depth features. We evaluate the proposed trained models on test dataset and provide comparisons with the state-of-the-art depth super-resolution methods.