OpenRaiser/Intern-Atlas huggingface dataset:
Data-Driven Multi-step Demand Prediction for Ride-hailing Services Using Convolutional Neural Network
Venue: arXiv cs.LG 2019
Year: 2019

Ride-hailing services are growing rapidly and becoming one of the most disruptive technologies in the transportation realm. Accurate prediction of ride-hailing trip demand not only enables cities to better understand people's activity patterns, but also helps ride-hailing companies and drivers make informed decisions to reduce deadheading vehicle miles traveled, traffic congestion, and energy consumption. In this study, a convolutional neural network (CNN)-based deep learning model is proposed for multi-step ride-hailing demand prediction using the trip request data in Chengdu, China, offered by DiDi Chuxing. The CNN model is capable of accurately predicting the ride-hailing pick-up demand at each 1-km by 1-km zone in the city of Chengdu for every 10 minutes. Compared with another deep learning model based on long short-term memory, the CNN model is 30% faster for the training and predicting process. The proposed model can also be easily extended to make multi-step predictions, which would benefit the on-demand shared autonomous vehicles applications and fleet operators in terms of supply-demand rebalancing. The prediction error attenuation analysis shows that the accuracy stays acceptable as the model predicts more steps.
MindTheStep-AsyncPSGD: Adaptive Asynchronous Parallel Stochastic Gradient Descent
Venue: arXiv cs.DC 2019
Year: 2019

Stochastic Gradient Descent (SGD) is very useful in optimization problems with high-dimensional non-convex target functions, and hence constitutes an important component of several Machine Learning and Data Analytics methods. Recently there have been significant works on understanding the parallelism inherent to SGD, and its convergence properties. Asynchronous, parallel SGD (AsyncPSGD) has received particular attention, due to observed performance benefits. On the other hand, asynchrony implies inherent challenges in understanding the execution of the algorithm and its convergence, stemming from the fact that the contribution of a thread might be based on an old (stale) view of the state. In this work we aim to deepen the understanding of AsyncPSGD in order to increase the statistical efficiency in the presence of stale gradients. We propose new models for capturing the nature of the staleness distribution in a practical setting. Using the proposed models, we derive a staleness-adaptive SGD framework, MindTheStep-AsyncPSGD, for adapting the step size in an online-fashion, which provably reduces the negative impact of asynchrony. Moreover, we provide general convergence time bounds for a wide class of staleness-adaptive step size strategies for convex target functions. We also provide a detailed empirical study, showing how our approach implies faster convergence for deep learning applications.
The TV is Smart and Full of Trackers: Towards Understanding the Smart TV Advertising and Tracking Ecosystem
Venue: arXiv cs.NI 2019
Year: 2019

Motivated by the growing popularity of smart TVs, we present a large-scale measurement study of smart TVs by collecting and analyzing their network traffic from two different vantage points. First, we analyze aggregate network traffic of smart TVs in-the-wild, collected from residential gateways of tens of homes and several different smart TV platforms, including Apple, Samsung, Roku, and Chromecast. In addition to accessing video streaming and cloud services, we find that smart TVs frequently connect to well-known as well as platform-specific advertising and tracking services (ATS). Second, we instrument Roku and Amazon Fire TV, two popular smart TV platforms, by setting up a controlled testbed to systematically exercise the top-1000 apps on each platform, and analyze their network traffic at the granularity of the individual apps. We again find that smart TV apps connect to a wide range of ATS, and that the key players of the ATS ecosystems of the two platforms are different from each other and from that of the mobile platform. Third, we evaluate the (in)effectiveness of state-of-the-art DNS-based blocklists in filtering advertising and tracking traffic for smart TVs. We find that personally identifiable information (PII) is exfiltrated to platform-related Internet endpoints and third parties, and that blocklists are generally better at preventing exposure of PII to third parties than to platform-related endpoints. Our work demonstrates the segmentation of the smart TV ATS ecosystem across platforms and its differences from the mobile ATS ecosystem, thus motivating the need for designing privacy-enhancing tools specifically for each smart TV platform.
Improving Machine Reading Comprehension via Adversarial Training
Venue: arXiv cs.CL 2019
Year: 2019

Adversarial training (AT) as a regularization method has proved its effectiveness in various tasks, such as image classification and text classification. Though there are successful applications of AT in many tasks of natural language processing (NLP), the mechanism behind it is still unclear. In this paper, we aim to apply AT on machine reading comprehension (MRC) and study its effects from multiple perspectives. We experiment with three different kinds of RC tasks: span-based RC, span-based RC with unanswerable questions and multi-choice RC. The experimental results show that the proposed method can improve the performance significantly and universally on SQuAD1.1, SQuAD2.0 and RACE. With virtual adversarial training (VAT), we explore the possibility of improving the RC models with semi-supervised learning and prove that examples from a different task are also beneficial. We also find that AT helps little in defending against artificial adversarial examples, but AT helps the model to learn better on examples that contain more low-frequency words.
Smart Scheduling and Feedback Allocation over Non-stationary Wireless Channels
Venue: arXiv cs.NI 2019
Year: 2019

It is well known that opportunistic scheduling algorithms are throughput optimal under dynamic channel and network conditions. However, these algorithms achieve a hypothetical rate region which does not take into account the overhead associated with channel probing and feedback required to obtain the full channel state information at every slot. In this work, we design a joint scheduling and channel probing algorithm by considering the overhead of obtaining the channel state information. We adopt a correlated and non-stationary channel model, which is more realistic than those used in the literature. We use concepts from learning and information theory to accurately track channel variations to minimize the number of channels probed at every slot, while scheduling users to maximize the achievable rate region of the network. More specifically, we employ Gaussian Process Regression that enable us to predict future channel state and also provide a way to measure uncertainty in the learning and prediction. Simulation results show that with the proposed algorithm, the network can carry higher user traffic. The proposed solution can be utilized in more complex and heterogeneous environments such as 5G and beyond networks where the model free approach is needed.
Multi-Perspective Inferrer: Reasoning Sentences Relationship from Holistic Perspective
Venue: arXiv cs.CL 2019
Year: 2019

Natural Language Inference (NLI) aims to determine the logic relationships (i.e., entailment, neutral and contradiction) between a pair of premise and hypothesis. Recently, the alignment mechanism effectively helps NLI by capturing the aligned parts (i.e., the similar segments) in the sentence pairs, which imply the perspective of entailment and contradiction. However, these aligned parts will sometimes mislead the judgment of neutral relations. Intuitively, NLI should rely more on multiple perspectives to form a holistic view to eliminate bias. In this paper, we propose the Multi-Perspective Inferrer (MPI), a novel NLI model that reasons relationships from multiple perspectives associated with the three relationships. The MPI determines the perspectives of different parts of the sentences via a routing-by-agreement policy and makes the final decision from a holistic view. Additionally, we introduce an auxiliary supervised signal to ensure the MPI to learn the expected perspectives. Experiments on SNLI and MultiNLI show that 1) the MPI achieves substantial improvements on the base model, which verifies the motivation of multi-perspective inference; 2) visualized evidence verifies that the MPI learns highly interpretable perspectives as expected; 3) more importantly, the MPI is architecture-free and compatible with the powerful BERT.
Preservation of Anomalous Subgroups On Machine Learning Transformed Data
Venue: arXiv cs.LG 2019
Year: 2019

In this paper, we investigate the effect of machine learning based anonymization on anomalous subgroup preservation. In particular, we train a binary classifier to discover the most anomalous subgroup in a dataset by maximizing the bias between the group's predicted odds ratio from the model and observed odds ratio from the data. We then perform anonymization using a variational autoencoder (VAE) to synthesize an entirely new dataset that would ideally be drawn from the distribution of the original data. We repeat the anomalous subgroup discovery task on the new data and compare it to what was identified pre-anonymization. We evaluated our approach using publicly available datasets from the financial industry. Our evaluation confirmed that the approach was able to produce synthetic datasets that preserved a high level of subgroup differentiation as identified initially in the original dataset. Such a distinction was maintained while having distinctly different records between the synthetic and original dataset. Finally, we packed the above end to end process into what we call Utility Guaranteed Deep Privacy (UGDP) system. UGDP can be easily extended to onboard alternative generative approaches such as GANs to synthesize tabular data.
Hybrid Localization: A Low Cost, Low Complexity Approach Based on Wi-Fi and Odometry
Venue: arXiv cs.RO 2019
Year: 2019

Localization in indoor environments is essential to further support automation in a wide array of scenarios. Moreover, direction-of-arrival knowledge is essential to supporting high speed millimeter-wave (mmWave) links in indoor environments, since most mmWave links are of a line-of-sight nature to combat the high pathloss in this band. Accurate wireless localization in indoor environments, however, has proved a challenging task due to multi-path fading. Additionally, due to the effects of multi-path fading, methods such as trilateration alone do not result in accurate localization. As such, in this paper we propose to combine the knowledge of wireless localization methods with that of odometry sensors to track the location of a mobile robot. This paper presents significant real-world localization measurement results for both Wi-Fi and odometry in diverse environments at the Boise State University campus. Using these results, we devise an algorithm to combine data from both odometry and wireless localization. This algorithm is shown in hardware testing to reduce the localization error for a mobile robot
Nonlinearity + Networks: A 2020 Vision
Venue: arXiv cs.SI 2019
Year: 2019

I briefly survey several fascinating topics in networks and nonlinearity. I highlight a few methods and ideas, including several of personal interest, that I anticipate to be especially important during the next several years. These topics include temporal networks (in which the entities and/or their interactions change in time), stochastic and deterministic dynamical processes on networks, adaptive networks (in which a dynamical process on a network is coupled to dynamics of network structure), and network structure and dynamics that include "higher-order" interactions (which involve three or more entities in a network). I draw examples from a variety of scenarios, including contagion dynamics, opinion models, waves, and coupled oscillators.
Stylized Text Generation Using Wasserstein Autoencoders with a Mixture of Gaussian Prior
Venue: arXiv cs.CL 2019
Year: 2019

Wasserstein autoencoders are effective for text generation. They do not however provide any control over the style and topic of the generated sentences if the dataset has multiple classes and includes different topics. In this work, we present a semi-supervised approach for generating stylized sentences. Our model is trained on a multi-class dataset and learns the latent representation of the sentences using a mixture of Gaussian prior without any adversarial losses. This allows us to generate sentences in the style of a specified class or multiple classes by sampling from their corresponding prior distributions. Moreover, we can train our model on relatively small datasets and learn the latent representation of a specified class by adding external data with other styles/classes to our dataset. While a simple WAE or VAE cannot generate diverse sentences in this case, generated sentences with our approach are diverse, fluent, and preserve the style and the content of the desired classes.
An Overview of Data-Importance Aware Radio Resource Management for Edge Machine Learning
Venue: arXiv cs.IT 2019
Year: 2019

The 5G network connecting billions of Internet-of-Things (IoT) devices will make it possible to harvest an enormous amount of real-time mobile data. Furthermore, the 5G virtualization architecture will enable cloud computing at the (network) edge. The availability of both rich data and computation power at the edge has motivated Internet companies to deploy artificial intelligence (AI) there, creating the hot area of edge-AI. Edge learning, the theme of this project, concerns training edge-AI models, which endow on IoT devices intelligence for responding to real-time events. However, the transmission of high-dimensional data from many edge devices to servers can result in excessive communication latency, creating a bottleneck for edge learning. Traditional wireless techniques deigned for only radio access are ineffective in tackling the challenge. Attempts to overcome the communication bottleneck has led to the development of a new class of techniques for intelligent radio resource management (RRM), called data-importance aware RRM. Their designs feature the interplay of active machine learning and wireless communication. Specifically, the metrics that measure data importance in active learning (e.g., classification uncertainty and data diversity) are applied to RRM for efficient acquisition of distributed data in wireless networks to train AI models at servers. This article aims at providing an introduction to the emerging area of importance-aware RRM. To this end, we will introduce the design principles, survey recent advancements in the area, discuss some design examples, and suggest some promising research opportunities.
Can Monolingual Pretrained Models Help Cross-Lingual Classification?
Venue: arXiv cs.CL 2019
Year: 2019

Multilingual pretrained language models (such as multilingual BERT) have achieved impressive results for cross-lingual transfer. However, due to the constant model capacity, multilingual pre-training usually lags behind the monolingual competitors. In this work, we present two approaches to improve zero-shot cross-lingual classification, by transferring the knowledge from monolingual pretrained models to multilingual ones. Experimental results on two cross-lingual classification benchmarks show that our methods outperform vanilla multilingual fine-tuning.
Transformation of low-quality device-recorded speech to high-quality speech using improved SEGAN model
Venue: arXiv cs.SD 2019
Year: 2019

Nowadays vast amounts of speech data are recorded from low-quality recorder devices such as smartphones, tablets, laptops, and medium-quality microphones. The objective of this research was to study the automatic generation of high-quality speech from such low-quality device-recorded speech, which could then be applied to many speech-generation tasks. In this paper, we first introduce our new device-recorded speech dataset then propose an improved end-to-end method for automatically transforming the low-quality device-recorded speech into professional high-quality speech. Our method is an extension of a generative adversarial network (GAN)-based speech enhancement model called speech enhancement GAN (SEGAN), and we present two modifications to make model training more robust and stable. Finally, from a large-scale listening test, we show that our method can significantly enhance the quality of device-recorded speech signals.
A Multimodal CNN-based Tool to Censure Inappropriate Video Scenes
Venue: arXiv cs.MM 2019
Year: 2019

Due to the extensive use of video-sharing platforms and services for their storage, the amount of such media on the internet has become massive. This volume of data makes it difficult to control the kind of content that may be present in such video files. One of the main concerns regarding the video content is if it has an inappropriate subject matter, such as nudity, violence, or other potentially disturbing content. More than telling if a video is either appropriate or inappropriate, it is also important to identify which parts of it contain such content, for preserving parts that would be discarded in a simple broad analysis. In this work, we present a multimodal~(using audio and image features) architecture based on Convolutional Neural Networks (CNNs) for detecting inappropriate scenes in video files. In the task of classifying video files, our model achieved 98.95\% and 98.94\% of F1-score for the appropriate and inappropriate classes, respectively. We also present a censoring tool that automatically censors inappropriate segments of a video file.
Neural Cryptanalysis: Metrics, Methodology, and Applications in CPS Ciphers
Venue: arXiv cs.CR 2019
Year: 2019

Many real-world cyber-physical systems (CPS) use proprietary cipher algorithms. In this work, we describe an easy-to-use black-box security evaluation approach to measure the strength of proprietary ciphers without having to know the algorithms. We quantify the strength of a cipher by measuring how difficult it is for a neural network to mimic the cipher algorithm. We define new metrics (e.g., cipher match rate, training data complexity and training time complexity) that are computed from neural networks to quantitatively represent the cipher strength. This measurement approach allows us to directly compare the security of ciphers. Our experimental demonstration utilizes fully connected neural networks with multiple parallel binary classifiers at the output layer. The results show that when compared with round-reduced DES, the security strength of Hitag2 (a popular stream cipher used in the keyless entry of modern cars) is weaker than 3-round DES.
Parallel, Modular Fabric Actuators for a 2-DOF Pneumatic Sleeve
Venue: arXiv cs.RO 2019
Year: 2019

The shoulder is responsible for the movement of the entire upper limb. It is capable of articulating in three degrees-of-freedom (DOF), enabling the arm to perform manipulation actions with incomparable dexterity. Exoskeletons targeting the shoulder must be able to emulate its complex kinematics. While typical architectures have proven to be useful, they employ complex and usually inadequate techniques to match the mobility of the shoulder. A new 2-DOF soft robotic shoulder exoskeleton is presented. Modular soft robotic actuators with separated inflation modules on the exoskeleton are able to emulate the humerus' natural movement. These actuators were organized into two antagonistic pairs and operate in a parallel configuration. We present the design and functionality of the actuators powering the exoskeleton. The actuator design enables it to perform 3-D bending with insignificant resistance. We measured its performance through a series of static and dynamic tests. Depending on the configuration, the actuator can generate up to a maximum torque of 15.54 N-m and can respond with a minimum rise time of ~2s when excited by a step input. We demonstrated its ability to perform assistance of shoulder movements. It is capable of reaching any point on the humerus' workspace from any starting position.
DialogAct2Vec: Towards End-to-End Dialogue Agent by Multi-Task Representation Learning
Venue: arXiv cs.CL 2019
Year: 2019

In end-to-end dialogue modeling and agent learning, it is important to (1) effectively learn knowledge from data, and (2) fully utilize heterogeneous information, e.g., dialogue act flow and utterances. However, the majority of existing methods cannot simultaneously satisfy the two conditions. For example, rule definition and data labeling during system design take too much manual work, and sequence-to-sequence methods only model one-side utterance information. In this paper, we propose a novel joint end-to-end model by multi-task representation learning, which can capture the knowledge from heterogeneous information through automatically learning knowledgeable low-dimensional embeddings from data, named with DialogAct2Vec. The model requires little manual work for intervention in system design and we find that the multi-task learning can greatly improve the effectiveness of representation learning. Extensive experiments on a public dataset for restaurant reservation show that the proposed method leads to significant improvements against the state-of-the-art baselines on both the act prediction task and utterance prediction task.
Authentication of Smartphone Users Using Behavioral Biometrics
Venue: arXiv cs.CR 2019
Year: 2019

Smartphones and tablets have become ubiquitous in our daily lives. Smartphones, in particular, have become more than personal assistants. These devices have provided new avenues for consumers to play, work, and socialize whenever and wherever they want. Smartphones are small in size, so they are easy to handle and to stow and carry in users' pockets or purses. However, mobile devices are also susceptible to various problems. One of the greatest concerns is the possibility of breach in security and privacy if the device is seized by an outside party. It is possible that threats can come from friends as well as strangers. Due to the size of smart devices, they can be easily lost and may expose details of users' private lives. In addition, this might enable pervasive observation or imitation of one's movements and activities, such as sending messages to contacts, accessing private communication, shopping with a credit card, and relaying information about where one has been. This paper highlights the potential risks that occur when smartphones are stolen or seized, discusses the concept of continuous authentication, and analyzes current approaches and mechanisms of behavioral biometrics with respect to methodology, associated datasets and evaluation approaches.