OpenRaiser/Intern-Atlas huggingface dataset:
On the Effective Throughput of Coded Caching: A Game Theoretic Perspective
Venue: arXiv cs.IT 2019
Year: 2019

Recently coded caching has emerged as a promising means to handle continuously increasing wireless traffic. However, coded caching requires users to cooperate in order to minimize the overall transmission rate. How users with heterogeneous preferences cooperate in coded caching and how to calculate the resulting caching gains are still open problems. In this paper, a two-phase cache-aided network is investigated, in which users with heterogeneous preferences are served by a base station through a shared link. Effective throughput is considered as a performance metric. It is proved that the achievable domain of effective throughputs is a convex set and can be characterized by its boundary. A special type of caching schemes, named uncoded placement absolutely-fair (UPAF) caching, is studied. For the two-user case, games are formulated to allocate effective throughput gains for the two users. For the general multiuser case, a UPAF policy is proposed to organize user cooperation. It is shown that users with more concentrated preferences can obtain higher effective throughputs.
Merging Weak and Active Supervision for Semantic Parsing
Venue: arXiv cs.CL 2019
Year: 2019

A semantic parser maps natural language commands (NLs) from the users to executable meaning representations (MRs), which are later executed in certain environment to obtain user-desired results. The fully-supervised training of such parser requires NL/MR pairs, annotated by domain experts, which makes them expensive to collect. However, weakly-supervised semantic parsers are learnt only from pairs of NL and expected execution results, leaving the MRs latent. While weak supervision is cheaper to acquire, learning from this input poses difficulties. It demands that parsers search a large space with a very weak learning signal and it is hard to avoid spurious MRs that achieve the correct answer in the wrong way. These factors lead to a performance gap between parsers trained in weakly- and fully-supervised setting. To bridge this gap, we examine the intersection between weak supervision and active learning, which allows the learner to actively select examples and query for manual annotations as extra supervision to improve the model trained under weak supervision. We study different active learning heuristics for selecting examples to query, and various forms of extra supervision for such queries. We evaluate the effectiveness of our method on two different datasets. Experiments on the WikiSQL show that by annotating only 1.8% of examples, we improve over a state-of-the-art weakly-supervised baseline by 6.4%, achieving an accuracy of 79.0%, which is only 1.3% away from the model trained with full supervision. Experiments on WikiTableQuestions with human annotators show that our method can improve the performance with only 100 active queries, especially for weakly-supervised parsers learnt from a cold start.
Operational Framework for Recent Advances in Backtracking Search Optimisation Algorithm: A Systematic Review and Performance Evaluation
Venue: arXiv cs.NE 2019
Year: 2019

The experiments conducted in previous studies demonstrated the successful performance of BSA and its non-sensitivity toward the several types of optimisation problems. This success of BSA motivated researchers to work on expanding it, e.g., developing its improved versions or employing it for different applications and problem domains. However, there is a lack of literature review on BSA; therefore, reviewing the aforementioned modifications and applications systematically will aid further development of the algorithm. This paper provides a systematic review and meta-analysis that emphasise on reviewing the related studies and recent developments on BSA. Hence, the objectives of this work are two-fold: (i) First, two frameworks for depicting the main extensions and the uses of BSA are proposed. The first framework is a general framework to depict the main extensions of BSA, whereas the second is an operational framework to present the expansion procedures of BSA to guide the researchers who are working on improving it. (ii) Second, the experiments conducted in this study fairly compare the analytical performance of BSA with four other competitive algorithms: differential evolution (DE), particle swarm optimisation (PSO), artificial bee colony (ABC), and firefly (FF) on 16 different hardness scores of the benchmark functions with different initial control parameters such as problem dimensions and search space. The experimental results indicate that BSA is statistically superior than the aforementioned algorithms in solving different cohorts of numerical optimisation problems such as problems with different levels of hardness score, problem dimensions, and search spaces.
SOSD: A Benchmark for Learned Indexes
Venue: arXiv cs.DB 2019
Year: 2019

A groundswell of recent work has focused on improving data management systems with learned components. Specifically, work on learned index structures has proposed replacing traditional index structures, such as B-trees, with learned models. Given the decades of research committed to improving index structures, there is significant skepticism about whether learned indexes actually outperform state-of-the-art implementations of traditional structures on real-world data. To answer this question, we propose a new benchmarking framework that comes with a variety of real-world datasets and baseline implementations to compare against. We also show preliminary results for selected index structures, and find that learned models indeed often outperform state-of-the-art implementations, and are therefore a promising direction for future research.
A Multi-cascaded Deep Model for Bilingual SMS Classification
Venue: arXiv cs.CL 2019
Year: 2019

Most studies on text classification are focused on the English language. However, short texts such as SMS are influenced by regional languages. This makes the automatic text classification task challenging due to the multilingual, informal, and noisy nature of language in the text. In this work, we propose a novel multi-cascaded deep learning model called McM for bilingual SMS classification. McM exploits $n$-gram level information as well as long-term dependencies of text for learning. Our approach aims to learn a model without any code-switching indication, lexical normalization, language translation, or language transliteration. The model relies entirely upon the text as no external knowledge base is utilized for learning. For this purpose, a 12 class bilingual text dataset is developed from SMS feedbacks of citizens on public services containing mixed Roman Urdu and English languages. Our model achieves high accuracy for classification on this dataset and outperforms the previous model for multilingual text classification, highlighting language independence of McM.
DeStress: Deep Learning for Unsupervised Identification of Mental Stress in Firefighters from Heart-rate Variability (HRV) Data
Venue: arXiv cs.LG 2019
Year: 2019

In this work we perform a study of various unsupervised methods to identify mental stress in firefighter trainees based on unlabeled heart rate variability data. We collect RR interval time series data from nearly 100 firefighter trainees that participated in a drill. We explore and compare three methods in order to perform unsupervised stress detection: 1) traditional K-Means clustering with engineered time and frequency domain features 2) convolutional autoencoders and 3) long short-term memory (LSTM) autoencoders, both trained on the raw RRI measurements combined with DBSCAN clustering and K-Nearest-Neighbors classification. We demonstrate that K-Means combined with engineered features is unable to capture meaningful structure within the data. On the other hand, convolutional and LSTM autoencoders tend to extract varying structure from the data pointing to different clusters with different sizes of clusters. We attempt at identifying the true stressed and normal clusters using the HRV markers of mental stress reported in the literature. We demonstrate that the clusters produced by the convolutional autoencoders consistently and successfully stratify stressed versus normal samples, as validated by several established physiological stress markers such as RMSSD, Max-HR, Mean-HR and LF-HF ratio.
To Trust, or Not to Trust? A Study of Human Bias in Automated Video Interview Assessments
Venue: arXiv cs.HC 2019
Year: 2019

Supervised systems require human labels for training. But, are humans themselves always impartial during the annotation process? We examine this question in the context of automated assessment of human behavioral tasks. Specifically, we investigate whether human ratings themselves can be trusted at their face value when scoring video-based structured interviews, and whether such ratings can impact machine learning models that use them as training data. We present preliminary empirical evidence that indicates there might be biases in such annotations, most of which are visual in nature.
Unpaired Image Translation via Adaptive Convolution-based Normalization
Venue: arXiv cs.CV 2019
Year: 2019

Disentangling content and style information of an image has played an important role in recent success in image translation. In this setting, how to inject given style into an input image containing its own content is an important issue, but existing methods followed relatively simple approaches, leaving room for improvement especially when incorporating significant style changes. In response, we propose an advanced normalization technique based on adaptive convolution (AdaCoN), in order to properly impose style information into the content of an input image. In detail, after locally standardizing the content representation in a channel-wise manner, AdaCoN performs adaptive convolution where the convolution filter weights are dynamically estimated using the encoded style representation. The flexibility of AdaCoN can handle complicated image translation tasks involving significant style changes. Our qualitative and quantitative experiments demonstrate the superiority of our proposed method against various existing approaches that inject the style into the content.
Chain Rules for Hessian and Higher Derivatives Made Easy by Tensor Calculus
Venue: arXiv cs.SC 2019
Year: 2019

Computing multivariate derivatives of matrix-like expressions in the compact, coordinate free fashion is very important for both theory and applied computations (e.g. optimization and machine learning). The critical components of such computations are \emph{chain and product rules} for derivatives. Although they are taught early in simple scenarios, practical applications involve high-dimensional arrays; in this context it is very hard to find easy accessible and compact explanation. This paper discusses how to relatively simply carry such derivations based on the (simplified as adapted in applied computer science) concept of tensors. Numerical examples in modern Python libraries are provided. This discussion simplifies and illustrates an earlier exposition by Manton (2012).
Long Short-Term Network Based Unobtrusive Perceived Workload Monitoring with Consumer Grade Smartwatches in the Wild
Venue: arXiv cs.HC 2019
Year: 2019

Continuous high perceived workload has a negative impact on the individual's well-being. Prior works focused on detecting the workload with medical-grade wearable systems in the restricted settings, and the effect of applying deep learning techniques for perceived workload detection in the wild settings is not investigated. We present an unobtrusive, comfortable, pervasive and affordable Long Short-Term Memory Network based continuous workload monitoring system based on a smartwatch application that monitors the perceived workload of individuals in the wild. We make use of modern consumer-grade smartwatches. We have recorded physiological data from daily life with perceived workload questionnaires from subjects in their real-life environments over a month. The model was trained and evaluated with the daily-life physiological data coming from different days which makes it robust to daily changes in the heart rate variability, that we use with accelerometer features to asses low and high workload. Our system has the capability of removing motion-related artifacts and detecting perceived workload by using traditional and deep classifiers. We discussed the problems related to in the wild applications with the consumer-grade smartwatches. We showed that Long Short-Term Memory Network outperforms traditional classifiers on discrimination of low and high workload with smartwatches in the wild.
Model Embedded DRL for Intelligent Greenhouse Control
Venue: arXiv cs.LG 2019
Year: 2019

Greenhouse environment is the key to influence crops production. However, it is difficult for classical control methods to give precise environment setpoints, such as temperature, humidity, light intensity and carbon dioxide concentration for greenhouse because it is uncertain nonlinear system. Therefore, an intelligent close loop control framework based on model embedded deep reinforcement learning (MEDRL) is designed for greenhouse environment control. Specifically, computer vision algorithms are used to recognize growing periods and sex of crops, followed by the crop growth models, which can be trained with different growing periods and sex. These model outputs combined with the cost factor provide the setpoints for greenhouse and feedback to the control system in real-time. The whole MEDRL system has capability to conduct optimization control precisely and conveniently, and costs will be greatly reduced compared with traditional greenhouse control approaches.
FusionMapping: Learning Depth Prediction with Monocular Images and 2D Laser Scans
Venue: arXiv cs.CV 2019
Year: 2019

Acquiring accurate three-dimensional depth information conventionally requires expensive multibeam LiDAR devices. Recently, researchers have developed a less expensive option by predicting depth information from two-dimensional color imagery. However, there still exists a substantial gap in accuracy between depth information estimated from two-dimensional images and real LiDAR point-cloud. In this paper, we introduce a fusion-based depth prediction method, called FusionMapping. This is the first method that fuses colored imagery and two-dimensional laser scan to estimate depth in-formation. More specifically, we propose an autoencoder-based depth prediction network and a novel point-cloud refinement network for depth estimation. We analyze the performance of our FusionMapping approach on the KITTI LiDAR odometry dataset and an indoor mobile robot system. The results show that our introduced approach estimates depth with better accuracy when compared to existing methods.
Disentanglement Challenge: From Regularization to Reconstruction
Venue: arXiv cs.LG 2019
Year: 2019

The challenge of learning disentangled representation has recently attracted much attention and boils down to a competition using a new real world disentanglement dataset (Gondal et al., 2019). Various methods based on variational auto-encoder have been proposed to solve this problem, by enforcing the independence between the representation and modifying the regularization term in the variational lower bound. However recent work by Locatello et al. (2018) has demonstrated that the proposed methods are heavily influenced by randomness and the choice of the hyper-parameter. In this work, instead of designing a new regularization term, we adopt the FactorVAE but improve the reconstruction performance and increase the capacity of network and the training step. The strategy turns out to be very effective and achieve the 1st place in the challenge.
Urban Driving with Conditional Imitation Learning
Venue: arXiv cs.CV 2019
Year: 2019

Hand-crafting generalised decision-making rules for real-world urban autonomous driving is hard. Alternatively, learning behaviour from easy-to-collect human driving demonstrations is appealing. Prior work has studied imitation learning (IL) for autonomous driving with a number of limitations. Examples include only performing lane-following rather than following a user-defined route, only using a single camera view or heavily cropped frames lacking state observability, only lateral (steering) control, but not longitudinal (speed) control and a lack of interaction with traffic. Importantly, the majority of such systems have been primarily evaluated in simulation - a simple domain, which lacks real-world complexities. Motivated by these challenges, we focus on learning representations of semantics, geometry and motion with computer vision for IL from human driving demonstrations. As our main contribution, we present an end-to-end conditional imitation learning approach, combining both lateral and longitudinal control on a real vehicle for following urban routes with simple traffic. We address inherent dataset bias by data balancing, training our final policy on approximately 30 hours of demonstrations gathered over six months. We evaluate our method on an autonomous vehicle by driving 35km of novel routes in European urban streets.
Modeling Fluency and Faithfulness for Diverse Neural Machine Translation
Venue: arXiv cs.CL 2019
Year: 2019

Neural machine translation models usually adopt the teacher forcing strategy for training which requires the predicted sequence matches ground truth word by word and forces the probability of each prediction to approach a 0-1 distribution. However, the strategy casts all the portion of the distribution to the ground truth word and ignores other words in the target vocabulary even when the ground truth word cannot dominate the distribution. To address the problem of teacher forcing, we propose a method to introduce an evaluation module to guide the distribution of the prediction. The evaluation module accesses each prediction from the perspectives of fluency and faithfulness to encourage the model to generate the word which has a fluent connection with its past and future translation and meanwhile tends to form a translation equivalent in meaning to the source. The experiments on multiple translation tasks show that our method can achieve significant improvements over strong baselines.
Latent Semantic Search and Information Extraction Architecture
Venue: arXiv cs.IR 2019
Year: 2019

The motivation, concept, design and implementation of latent semantic search for search engines have limited semantic search, entity extraction and property attribution features, have insufficient accuracy and response time of latent search, may impose privacy concerns and the search results are unavailable in offline mode for robotic search operations. The alternative suggestion involves autonomous search engine with adaptive storage consumption, configurable search scope and latent search response time with built-in options for entity extraction and property attribution available as open source platform for mobile, desktop and server solutions. The suggested architecture attempts to implement artificial general intelligence (AGI) principles as long as autonomous behaviour constrained by limited resources is concerned, and it is applied for specific task of enabling Web search for artificial agents implementing the AGI.
Pruning at a Glance: Global Neural Pruning for Model Compression
Venue: arXiv cs.CV 2019
Year: 2019

Deep Learning models have become the dominant approach in several areas due to their high performance. Unfortunately, the size and hence computational requirements of operating such models can be considerably high. Therefore, this constitutes a limitation for deployment on memory and battery constrained devices such as mobile phones or embedded systems. To address these limitations, we propose a novel and simple pruning method that compresses neural networks by removing entire filters and neurons according to a global threshold across the network without any pre-calculation of layer sensitivity. The resulting model is compact, non-sparse, with the same accuracy as the non-compressed model, and most importantly requires no special infrastructure for deployment. We prove the viability of our method by producing highly compressed models, namely VGG-16, ResNet-56, and ResNet-110 respectively on CIFAR10 without losing any performance compared to the baseline, as well as ResNet-34 and ResNet-50 on ImageNet without a significant loss of accuracy. We also provide a well-retrained 30% compressed ResNet-50 that slightly surpasses the base model accuracy. Additionally, compressing more than 56% and 97% of AlexNet and LeNet-5 respectively. Interestingly, the resulted models' pruning patterns are highly similar to the other methods using layer sensitivity pre-calculation step. Our method does not only exhibit good performance but what is more also easy to implement.
Relation Graph Network for 3D Object Detection in Point Clouds
Venue: arXiv cs.CV 2019
Year: 2019

Convolutional Neural Networks (CNNs) have emerged as a powerful strategy for most object detection tasks on 2D images. However, their power has not been fully realised for detecting 3D objects in point clouds directly without converting them to regular grids. Existing state-of-art 3D object detection methods aim to recognize 3D objects individually without exploiting their relationships during learning or inference. In this paper, we first propose a strategy that associates the predictions of direction vectors and pseudo geometric centers together leading to a win-win solution for 3D bounding box candidates regression. Secondly, we propose point attention pooling to extract uniform appearance features for each 3D object proposal, benefiting from the learned direction features, semantic features and spatial coordinates of the object points. Finally, the appearance features are used together with the position features to build 3D object-object relationship graphs for all proposals to model their co-existence. We explore the effect of relation graphs on proposals' appearance features enhancement under supervised and unsupervised settings. The proposed relation graph network consists of a 3D object proposal generation module and a 3D relation module, makes it an end-to-end trainable network for detecting 3D object in point clouds. Experiments on challenging benchmarks ( SunRGB-Dand ScanNet datasets ) of 3D point clouds show that our algorithm can perform better than the existing state-of-the-art methods.
Transferable Force-Torque Dynamics Model for Peg-in-hole Task
Venue: arXiv cs.RO 2019
Year: 2019

We present a learning-based force-torque dynamics to achieve model-based control for contact-rich peg-in-hole task using force-only inputs. Learning the force-torque dynamics is challenging because of the ambiguity of the low-dimensional 6-d force signal and the requirement of excessive training data. To tackle these problems, we propose a multi-pose force-torque state representation, based on which a dynamics model is learned with the data generated in a sample-efficient offline fashion. In addition, by training the dynamics model with peg-and-holes of various shapes, scales, and elasticities, the model could quickly transfer to new peg-and-holes after a small number of trials. Extensive experiments show that our dynamics model could adapt to unseen peg-and-holes with 70% fewer samples required compared to learning from scratch. Along with the learned dynamics, model predictive control and model-based reinforcement learning policies achieve over 80% insertion success rate. Our video is available at https://youtu.be/ZAqldpVZgm4.