OpenRaiser/Intern-Atlas huggingface dataset:
Towards Pose-invariant Lip-Reading
Venue: arXiv cs.CV 2019
Year: 2019

Lip-reading models have been significantly improved recently thanks to powerful deep learning architectures. However, most works focused on frontal or near frontal views of the mouth. As a consequence, lip-reading performance seriously deteriorates in non-frontal mouth views. In this work, we present a framework for training pose-invariant lip-reading models on synthetic data instead of collecting and annotating non-frontal data which is costly and tedious. The proposed model significantly outperforms previous approaches on non-frontal views while retaining the superior performance on frontal and near frontal mouth views. Specifically, we propose to use a 3D Morphable Model (3DMM) to augment LRW, an existing large-scale but mostly frontal dataset, by generating synthetic facial data in arbitrary poses. The newly derived dataset, is used to train a state-of-the-art neural network for lip-reading. We conducted a cross-database experiment for isolated word recognition on the LRS2 dataset, and reported an absolute improvement of 2.55%. The benefit of the proposed approach becomes clearer in extreme poses where an absolute improvement of up to 20.64% over the baseline is achieved.
Learning Multi-Sense Word Distributions using Approximate Kullback-Leibler Divergence
Venue: arXiv cs.CL 2019
Year: 2019

Learning word representations has garnered greater attention in the recent past due to its diverse text applications. Word embeddings encapsulate the syntactic and semantic regularities of sentences. Modelling word embedding as multi-sense gaussian mixture distributions, will additionally capture uncertainty and polysemy of words. We propose to learn the Gaussian mixture representation of words using a Kullback-Leibler (KL) divergence based objective function. The KL divergence based energy function provides a better distance metric which can effectively capture entailment and distribution similarity among the words. Due to the intractability of KL divergence for Gaussian mixture, we go for a KL approximation between Gaussian mixtures. We perform qualitative and quantitative experiments on benchmark word similarity and entailment datasets which demonstrate the effectiveness of the proposed approach.
Towards Supervised Extractive Text Summarization via RNN-based Sequence Classification
Venue: arXiv cs.CL 2019
Year: 2019

This article briefly explains our submitted approach to the DocEng'19 competition on extractive summarization. We implemented a recurrent neural network based model that learns to classify whether an article's sentence belongs to the corresponding extractive summary or not. We bypass the lack of large annotated news corpora for extractive summarization by generating extractive summaries from abstractive ones, which are available from the CNN corpus.
A Bayesian/Information Theoretic Model of Bias Learning
Venue: arXiv cs.LG 2019
Year: 2019

In this paper the problem of learning appropriate bias for an environment of related tasks is examined from a Bayesian perspective. The environment of related tasks is shown to be naturally modelled by the concept of an {\em objective} prior distribution. Sampling from the objective prior corresponds to sampling different learning tasks from the environment. It is argued that for many common machine learning problems, although we don't know the true (objective) prior for the problem, we do have some idea of a set of possible priors to which the true prior belongs. It is shown that under these circumstances a learner can use Bayesian inference to learn the true prior by sampling from the objective prior. Bounds are given on the amount of information required to learn a task when it is simultaneously learnt with several other tasks. The bounds show that if the learner has little knowledge of the true prior, and the dimensionality of the true prior is small, then sampling multiple tasks is highly advantageous.
On Network Embedding for Machine Learning on Road Networks: A Case Study on the Danish Road Network
Venue: arXiv cs.LG 2019
Year: 2019

Road networks are a type of spatial network, where edges may be associated with qualitative information such as road type and speed limit. Unfortunately, such information is often incomplete; for instance, OpenStreetMap only has speed limits for 13% of all Danish road segments. This is problematic for analysis tasks that rely on such information for machine learning. To enable machine learning in such circumstances, one may consider the application of network embedding methods to extract structural information from the network. However, these methods have so far mostly been used in the context of social networks, which differ significantly from road networks in terms of, e.g., node degree and level of homophily (which are key to the performance of many network embedding methods). We analyze the use of network embedding methods, specifically node2vec, for learning road segment embeddings in road networks. Due to the often limited availability of information on other relevant road characteristics, the analysis focuses on leveraging the spatial network structure. Our results suggest that network embedding methods can indeed be used for deriving relevant network features (that may, e.g, be used for predicting speed limits), but that the qualities of the embeddings differ from embeddings for social networks.
Hardness of Learning DNFs using Halfspaces
Venue: arXiv cs.CC 2019
Year: 2019

The problem of learning $t$-term DNF formulas (for $t = O(1)$) has been studied extensively in the PAC model since its introduction by Valiant (STOC 1984). A $t$-term DNF can be efficiently learnt using a $t$-term DNF only if $t = 1$ i.e., when it is an AND, while even weakly learning a $2$-term DNF using a constant term DNF was shown to be NP-hard by Khot and Saket (FOCS 2008). On the other hand, Feldman et al. (FOCS 2009) showed the hardness of weakly learning a noisy AND using a halfspace -- the latter being a generalization of an AND, while Khot and Saket (STOC 2008) showed that an intersection of two halfspaces is hard to weakly learn using any function of constantly many halfspaces. The question of whether a $2$-term DNF is efficiently learnable using $2$ or constantly many halfspaces remained open. In this work we answer this question in the negative by showing the hardness of weakly learning a $2$-term DNF as well as a noisy AND using any function of a constant number of halfspaces. In particular we prove the following. For any constants $ν, ζ> 0$ and $\ell \in \mathbb{N}$, given a distribution over point-value pairs $\{0,1\}^n \times \{0,1\}$, it is NP-hard to decide whether, YES Case: There is a $2$-term DNF that classifies all the points of the distribution, and an AND that classifies at least $1-ζ$ fraction of the points correctly. NO Case: Any boolean function depending on at most $\ell$ halfspaces classifies at most $1/2 + ν$ fraction of the points of the distribution correctly. Our result generalizes and strengthens the previous best results mentioned above on the hardness of learning a $2$-term DNF, learning an intersection of two halfspaces, and learning a noisy AND.
Twitter Watch: Leveraging Social Media to Monitor and Predict Collective-Efficacy of Neighborhoods
Venue: arXiv cs.SI 2019
Year: 2019

Sociologists associate the spatial variation of crime within an urban setting, with the concept of collective efficacy. The collective efficacy of a neighborhood is defined as social cohesion among neighbors combined with their willingness to intervene on behalf of the common good. Sociologists measure collective efficacy by conducting survey studies designed to measure individuals' perception of their community. In this work, we employ the curated data from a survey study (ground truth) and examine the effectiveness of substituting costly survey questionnaires with proxies derived from social media. We enrich a corpus of tweets mentioning a local venue with several linguistic and topological features. We then propose a pairwise learning to rank model with the goal of identifying a ranking of neighborhoods that is similar to the ranking obtained from the ground truth collective efficacy values. In our experiments, we find that our generated ranking of neighborhoods achieves 0.77 Kendall tau-x ranking agreement with the ground truth ranking. Overall, our results are up to 37% better than traditional baselines.
Does Face Recognition Accuracy Get Better With Age? Deep Face Matchers Say No
Venue: arXiv cs.CV 2019
Year: 2019

Previous studies generally agree that face recognition accuracy is higher for older persons than for younger persons. But most previous studies were before the wave of deep learning matchers, and most considered accuracy only in terms of the verification rate for genuine pairs. This paper investigates accuracy for age groups 16-29, 30-49 and 50-70, using three modern deep CNN matchers, and considers differences in the impostor and genuine distributions as well as verification rates and ROC curves. We find that accuracy is lower for older persons and higher for younger persons. In contrast, a pre deep learning matcher on the same dataset shows the traditional result of higher accuracy for older persons, although its overall accuracy is much lower than that of the deep learning matchers. Comparing the impostor and genuine distributions, we conclude that impostor scores have a larger effect than genuine scores in causing lower accuracy for the older age group. We also investigate the effects of training data across the age groups. Our results show that fine-tuning the deep CNN models on additional images of older persons actually lowers accuracy for the older age group. Also, we fine-tune and train from scratch two models using age-balanced training datasets, and these results also show lower accuracy for older age group. These results argue that the lower accuracy for the older age group is not due to imbalance in the original training data.
Sparse associative memory based on contextual code learning for disambiguating word senses
Venue: arXiv cs.CL 2019
Year: 2019

In recent literature, contextual pretrained Language Models (LMs) demonstrated their potential in generalizing the knowledge to several Natural Language Processing (NLP) tasks including supervised Word Sense Disambiguation (WSD), a challenging problem in the field of Natural Language Understanding (NLU). However, word representations from these models are still very dense, costly in terms of memory footprint, as well as minimally interpretable. In order to address such issues, we propose a new supervised biologically inspired technique for transferring large pre-trained language model representations into a compressed representation, for the case of WSD. Our produced representation contributes to increase the general interpretability of the framework and to decrease memory footprint, while enhancing performance.
Human Annotations Improve GAN Performances
Venue: arXiv cs.CV 2019
Year: 2019

Generative Adversarial Networks (GANs) have shown great success in many applications. In this work, we present a novel method that leverages human annotations to improve the quality of generated images. Unlike previous paradigms that directly ask annotators to distinguish between real and fake data in a straightforward way, we propose and annotate a set of carefully designed attributes that encode important image information at various levels, to understand the differences between fake and real images. Specifically, we have collected an annotated dataset that contains 600 fake images and 400 real images. These images are evaluated by 10 workers from the Amazon Mechanical Turk (AMT) based on eight carefully defined attributes. Statistical analyses have revealed different distributions of the proposed attributes between real and fake images. These attributes are shown to be useful in discriminating fake images from real ones, and deep neural networks are developed to automatically predict the attributes. We further utilize the information by integrating the attributes into GANs to generate better images. Experimental results evaluated by multiple metrics show performance improvement of the proposed model.
Flexible Functional Split and Power Control for Energy Harvesting Cloud Radio Access Networks
Venue: arXiv cs.NI 2019
Year: 2019

Functional split is a promising technique to flexibly balance the processing cost at remote ends and the fronthaul rate in cloud radio access networks (C-RAN). By harvesting renewable energy, remote radio units (RRUs) can save grid power and be flexibly deployed. However, the randomness of energy arrival poses a major design challenge. To maximize the throughput under the average fronthaul rate constraint in C-RAN with renewable powered RRUs, we first study the offline problem of selecting the optimal functional split modes and the corresponding durations, jointly with the transmission power. We find that between successive energy arrivals, at most two functional split modes should be selected. Then the optimal online problem is formulated as an Markov decision process (MDP). To deal with the curse of dimensionality of solving MDP, we further analyze the special case with one instance of energy arrival and two candidate functional split modes as inspired by the offline solution, and then a heuristic online policy is proposed. Numerical results show that with flexible functional split, the throughput can be significantly improved compared with fixed functional split. Also, the proposed heuristic online policy has similar performance with the optimal online one, as validated by simulations.
$\ell_{\infty}$ Vector Contraction for Rademacher Complexity
Venue: arXiv cs.LG 2019
Year: 2019

We show that the Rademacher complexity of any $\mathbb{R}^{K}$-valued function class composed with an $\ell_{\infty}$-Lipschitz function is bounded by the maximum Rademacher complexity of the restriction of the function class along each coordinate, times a factor of $\tilde{O}(\sqrt{K})$.
"How do I fool you?": Manipulating User Trust via Misleading Black Box Explanations
Venue: arXiv cs.AI 2019
Year: 2019

As machine learning black boxes are increasingly being deployed in critical domains such as healthcare and criminal justice, there has been a growing emphasis on developing techniques for explaining these black boxes in a human interpretable manner. It has recently become apparent that a high-fidelity explanation of a black box ML model may not accurately reflect the biases in the black box. As a consequence, explanations have the potential to mislead human users into trusting a problematic black box. In this work, we rigorously explore the notion of misleading explanations and how they influence user trust in black-box models. More specifically, we propose a novel theoretical framework for understanding and generating misleading explanations, and carry out a user study with domain experts to demonstrate how these explanations can be used to mislead users. Our work is the first to empirically establish how user trust in black box models can be manipulated via misleading explanations.
Deep Long Audio Inpainting
Venue: arXiv cs.SD 2019
Year: 2019

Long (> 200 ms) audio inpainting, to recover a long missing part in an audio segment, could be widely applied to audio editing tasks and transmission loss recovery. It is a very challenging problem due to the high dimensional, complex and non-correlated audio features. While deep learning models have made tremendous progress in image and video inpainting, audio inpainting did not attract much attention. In this work, we take a pioneering step, exploring the possibility of adapting deep learning frameworks from various domains inclusive of audio synthesis and image inpainting for audio inpainting. Also, as the first to systematically analyze factors affecting audio inpainting performance, we explore how factors ranging from mask size, receptive field and audio representation could affect the performance. We also set up a benchmark for long audio inpainting. The code will be available on GitHub upon accepted.
A3GAN: An Attribute-aware Attentive Generative Adversarial Network for Face Aging
Venue: arXiv cs.CV 2019
Year: 2019

Face aging, which aims at aesthetically rendering a given face to predict its future appearance, has received significant research attention in recent years. Although great progress has been achieved with the success of Generative Adversarial Networks (GANs) in synthesizing realistic images, most existing GAN-based face aging methods have two main problems: 1) unnatural changes of high-level semantic information (e.g. facial attributes) due to the insufficient utilization of prior knowledge of input faces, and 2) distortions of low-level image content including ghosting artifacts and modifications in age-irrelevant regions. In this paper, we introduce A3GAN, an Attribute-Aware Attentive face aging model to address the above issues. Facial attribute vectors are regarded as the conditional information and embedded into both the generator and discriminator, encouraging synthesized faces to be faithful to attributes of corresponding inputs. To improve the visual fidelity of generation results, we leverage the attention mechanism to restrict modifications to age-related areas and preserve image details. Moreover, the wavelet packet transform is employed to capture textural features at multiple scales in the frequency space. Extensive experimental results demonstrate the effectiveness of our model in synthesizing photorealistic aged face images and achieving state-of-the-art performance on popular face aging datasets.
LIBRE: Learning Interpretable Boolean Rule Ensembles
Venue: arXiv cs.LG 2019
Year: 2019

We present a novel method - LIBRE - to learn an interpretable classifier, which materializes as a set of Boolean rules. LIBRE uses an ensemble of bottom-up weak learners operating on a random subset of features, which allows for the learning of rules that generalize well on unseen data even in imbalanced settings. Weak learners are combined with a simple union so that the final ensemble is also interpretable. Experimental results indicate that LIBRE efficiently strikes the right balance between prediction accuracy, which is competitive with black box methods, and interpretability, which is often superior to alternative methods from the literature.
The Markup Language for Designing Gaze Controlled Applications
Venue: arXiv cs.HC 2019
Year: 2019

The Gaze Interaction Markup Language (GIML) is presented, which is new language for designing the gaze-controlled application and psychological experiments (also by non-programmers).
AdvKnn: Adversarial Attacks On K-Nearest Neighbor Classifiers With Approximate Gradients
Venue: arXiv cs.CV 2019
Year: 2019

Deep neural networks have been shown to be vulnerable to adversarial examples---maliciously crafted examples that can trigger the target model to misbehave by adding imperceptible perturbations. Existing attack methods for k-nearest neighbor~(kNN) based algorithms either require large perturbations or are not applicable for large k. To handle this problem, this paper proposes a new method called AdvKNN for evaluating the adversarial robustness of kNN-based models. Firstly, we propose a deep kNN block to approximate the output of kNN methods, which is differentiable thus can provide gradients for attacks to cross the decision boundary with small distortions. Second, a new consistency learning for distribution instead of classification is proposed for the effectiveness in distribution based methods. Extensive experimental results indicate that the proposed method significantly outperforms state of the art in terms of attack success rate and the added perturbations.
Data Preparation in Agriculture Through Automated Semantic Annotation -- Basis for a Wide Range of Smart Services
Venue: arXiv cs.CY 2019
Year: 2019

Modern agricultural technology and the increasing digitalisation of such processes provide a wide range of data. However, their efficient and beneficial use suffers from legitimate concerns about data sovereignty and control, format inconsistencies and different interpretations. As a proposed solution, we present Wikinormia, a collaborative platform in which interested participants can describe and discuss their own new data formats. Once a finalized vocabulary has been created, specific parsers can semantically process the raw data into three basic representations: spatial information, time series and semantic facts (agricultural knowledge graph). Thanks to publicly accessible definitions and descriptions, developers can easily gain an overview of the concepts that are relevant to them. A variety of services will then (subject to individual access rights) be able to query their data simply via a query interface and retrieve results. We have already implemented this proposed solution in a prototype in the SDSD (Smart Data - Smart Services) project and demonstrate the benefits with a range of representative services. This provides an efficient system for the cooperative, flexible digitalisation of agricultural workflows.
Context Aware Family Dynamics based Internet of Things Access Control Towards Better Child Safety
Venue: arXiv cs.HC 2019
Year: 2019

Today, children are increasingly connected to the Internet and consume content and services through various means. It has been a challenge for less tech-savvy parents to protect children from harmful content and services. Internet of Things (IoT) has made the situation much worse as IoT devices allow children to connect to the Internet in novel ways (e.g., connected refrigerators, TVs, and so on). In this paper, we propose mySafeHome, an approach which utilises family dynamics to provide a more natural, and intuitive access control mechanism to protect children from harmful content and services in the context of IoT. In mySafeHome, access control dynamically adapts based on the physical distance between family members. For example, a particular type of content can only be consumed, through TV, by children if the parents are in the same room (or hearing distance). mySafeHome allows parents to assess a given content by themselves. Our approach also aims to create granular levels of access control (e.g., block / limit certain content, features, services, on certain devices when the parents are not in the vicinity). We developed a prototype using OpenHAB and several smart home devices to demonstrate the proposed approach. We believe that our approach also facilitates the creation of better relationships between family members. A demo can be viewed here: http://safehome.technology/demo.
CatGAN: Category-aware Generative Adversarial Networks with Hierarchical Evolutionary Learning for Category Text Generation
Venue: arXiv cs.CL 2019
Year: 2019

Generating multiple categories of texts is a challenging task and draws more and more attention. Since generative adversarial nets (GANs) have shown competitive results on general text generation, they are extended for category text generation in some previous works. However, the complicated model structures and learning strategies limit their performance and exacerbate the training instability. This paper proposes a category-aware GAN (CatGAN) which consists of an efficient category-aware model for category text generation and a hierarchical evolutionary learning algorithm for training our model. The category-aware model directly measures the gap between real samples and generated samples on each category, then reducing this gap will guide the model to generate high-quality category samples. The Gumbel-Softmax relaxation further frees our model from complicated learning strategies for updating CatGAN on discrete data. Moreover, only focusing on the sample quality normally leads the mode collapse problem, thus a hierarchical evolutionary learning algorithm is introduced to stabilize the training procedure and obtain the trade-off between quality and diversity while training CatGAN. Experimental results demonstrate that CatGAN outperforms most of the existing state-of-the-art methods.