OpenRaiser/Intern-Atlas huggingface dataset:
Augmentation adversarial training for self-supervised speaker recognition
Venue: arXiv cs.SD 2020
Year: 2020

The goal of this work is to train robust speaker recognition models without speaker labels. Recent works on unsupervised speaker representations are based on contrastive learning in which they encourage within-utterance embeddings to be similar and across-utterance embeddings to be dissimilar. However, since the within-utterance segments share the same acoustic characteristics, it is difficult to separate the speaker information from the channel information. To this end, we propose augmentation adversarial training strategy that trains the network to be discriminative for the speaker information, while invariant to the augmentation applied. Since the augmentation simulates the acoustic characteristics, training the network to be invariant to augmentation also encourages the network to be invariant to the channel information in general. Extensive experiments on the VoxCeleb and VOiCES datasets show significant improvements over previous works using self-supervision, and the performance of our self-supervised models far exceed that of humans.
Deep Active Learning by Model Interpretability
Venue: arXiv cs.LG 2020
Year: 2020

Recent successes of Deep Neural Networks (DNNs) in a variety of research tasks, however, heavily rely on the large amounts of labeled samples. This may require considerable annotation cost in real-world applications. Fortunately, active learning is a promising methodology to train high-performing model with minimal annotation cost. In the deep learning context, the critical question of active learning is how to precisely identify the informativeness of samples for DNN. In this paper, inspired by piece-wise linear interpretability in DNN, we introduce the linearly separable regions of samples to the problem of active learning, and propose a novel Deep Active learning approach by Model Interpretability (DAMI). To keep the maximal representativeness of the entire unlabeled data, DAMI tries to select and label samples on different linearly separable regions introduced by the piece-wise linear interpretability in DNN. We focus on modeling Multi-Layer Perception (MLP) for modeling tabular data. Specifically, we use the local piece-wise interpretation in MLP as the representation of each sample, and directly run K-Center clustering to select and label samples. To be noted, this whole process of DAMI does not require any hyper-parameters to tune manually. To verify the effectiveness of our approach, extensive experiments have been conducted on several tabular datasets. The experimental results demonstrate that DAMI constantly outperforms several state-of-the-art compared approaches.
Optimization of Processing Allocation in Vehicular Edge Cloud based Architecture
Venue: arXiv cs.NI 2020
Year: 2020

Vehicular edge computing is a new distributed processing architecture that exploits the revolution in the processing capabilities of vehicles to provide energy efficient services and low delay for Internet of Things (IoT)-based systems. Edge computing relies on a set of distributed processing nodes (i.e. vehicles) that are located close to the end user. In this paper, we consider a vehicular edge cloud (VEC) consisting of a set of vehicle clusters that form a temporal vehicular cloud by combining their computational resources in the cluster. We tackle the problem of processing allocation in the proposed vehicular edge architecture by developing a Mixed Integer Linear Programming (MILP) optimization model that jointly minimizes power consumption, propagation delay, and queuing delay. The results show that the closer the processing node (PN) is to the access point (AP), the lower the power consumption and delay, as the distance and number of hops affect the propagation delay and queuing delay. However, the queuing delay at the AP becomes a limiting factor when it operates at a low service rate compared to the traffic arrival rate. Thus, processing tasks at the vehicular nodes (VN) was avoided whenever the objective function included queueing delay and the AP operated at a low service rate. Increase in the AP service rate results in a lower queuing delay and better VN utilization.
HeRTA: Heaviside Real-Time Analysis
Venue: arXiv cs.OS 2020
Year: 2020

We investigate the mathematical properties of event bound functions as they are used in the worst-case response time analysis and utilization tests. We figure out the differences and similarities between the two approaches. Based on this analysis, we derive a more general form do describe events and event bounds. This new unified approach gives clear new insights in the investigation of real-time systems, simplifies the models and will support algebraic proofs in future work. In the end, we present a unified analysis which allows the algebraic definition of any scheduler. Introducing such functions to the real-time scheduling theory will lead two a more systematic way to integrate new concepts and applications to the theory. Last but not least, we show how the response time analysis in dynamic scheduling can be improved.
Silhouette Vectorization by Affine Scale-space
Venue: arXiv cs.GR 2020
Year: 2020

Silhouettes or 2D planar shapes are extremely important in human communication, which involves many logos, graphics symbols and fonts in vector form. Many more shapes can be extracted from image by binarization or segmentation, thus in raster form that requires a vectorization. There is a need for disposing of a mathematically well defined and justified shape vectorization process, which in addition provides a minimal set of control points with geometric meaning. In this paper we propose a silhouette vectorization method which extracts the outline of a 2D shape from a raster binary image, and converts it to a combination of cubic Bézier polygons and perfect circles. Starting from the boundary curvature extrema computed at sub-pixel level, we identify a set of control points based on the affine scale-space induced by the outline. These control points capture similarity invariant geometric features of the given silhouette and give precise locations of the shape's corners.of the given silhouette. Then, piecewise Bézier cubics are computed by least-square fitting combined with an adaptive splitting to guarantee a predefined accuracy. When there are no curvature extrema identified, either the outline is recognized as a circle using the isoperimetric inequality, or a pair of the most distant outline points are chosen to initiate the fitting. Given their construction, most of our control points are geometrically stable under affine transformations. By comparing with other feature detectors, we show that our method can be used as a reliable feature point detector for silhouettes. Compared to state-of-the-art image vectorization software, our algorithm demonstrates superior reduction on the number of control points, while maintaining high accuracy.
Health, Psychosocial, and Social issues emanating from COVID-19 pandemic based on Social Media Comments using Natural Language Processing
Venue: arXiv cs.CL 2020
Year: 2020

The COVID-19 pandemic has caused a global health crisis that affects many aspects of human lives. In the absence of vaccines and antivirals, several behavioural change and policy initiatives, such as physical distancing, have been implemented to control the spread of the coronavirus. Social media data can reveal public perceptions toward how governments and health agencies across the globe are handling the pandemic, as well as the impact of the disease on people regardless of their geographic locations in line with various factors that hinder or facilitate the efforts to control the spread of the pandemic globally. This paper aims to investigate the impact of the COVID-19 pandemic on people globally using social media data. We apply natural language processing (NLP) and thematic analysis to understand public opinions, experiences, and issues with respect to the COVID-19 pandemic using social media data. First, we collect over 47 million COVID-19-related comments from Twitter, Facebook, YouTube, and three online discussion forums. Second, we perform data preprocessing which involves applying NLP techniques to clean and prepare the data for automated theme extraction. Third, we apply context-aware NLP approach to extract meaningful keyphrases or themes from over 1 million randomly selected comments, as well as compute sentiment scores for each theme and assign sentiment polarity based on the scores using lexicon-based technique. Fourth, we categorize related themes into broader themes. A total of 34 negative themes emerged, out of which 15 are health-related issues, psychosocial issues, and social issues related to the COVID-19 pandemic from the public perspective. In addition, 20 positive themes emerged from our results. Finally, we recommend interventions that can help address the negative issues based on the positive themes and other remedial ideas rooted in research.
Sifting Convolution on the Sphere
Venue: arXiv cs.IT 2020
Year: 2020

A novel spherical convolution is defined through the sifting property of the Dirac delta on the sphere. The so-called sifting convolution is defined by the inner product of one function with a translated version of another, but with the adoption of an alternative translation operator on the sphere. This translation operator follows by analogy with the Euclidean translation when viewed in harmonic space. The sifting convolution satisfies a variety of desirable properties that are lacking in alternate definitions, namely: it supports directional kernels; it has an output which remains on the sphere; and is efficient to compute. An illustration of the sifting convolution on a topographic map of the Earth demonstrates that it supports directional kernels to perform anisotropic filtering, while its output remains on the sphere.
Are Visual Explanations Useful? A Case Study in Model-in-the-Loop Prediction
Venue: arXiv cs.LG 2020
Year: 2020

We present a randomized controlled trial for a model-in-the-loop regression task, with the goal of measuring the extent to which (1) good explanations of model predictions increase human accuracy, and (2) faulty explanations decrease human trust in the model. We study explanations based on visual saliency in an image-based age prediction task for which humans and learned models are individually capable but not highly proficient and frequently disagree. Our experimental design separates model quality from explanation quality, and makes it possible to compare treatments involving a variety of explanations of varying levels of quality. We find that presenting model predictions improves human accuracy. However, visual explanations of various kinds fail to significantly alter human accuracy or trust in the model - regardless of whether explanations characterize an accurate model, an inaccurate one, or are generated randomly and independently of the input image. These findings suggest the need for greater evaluation of explanations in downstream decision making tasks, better design-based tools for presenting explanations to users, and better approaches for generating explanations.
WiFi-based Crowd Monitoring and Workspace Planning for COVID-19 Recovery
Venue: arXiv cs.SI 2020
Year: 2020

The recovery phase of the COVID-19 pandemic requires careful planning and monitoring while people gradually return to work. Internet-of-Things (IoT) is widely regarded as a crucial tool to help combating COVID-19 pandemic in many areas and societies. In particular, the heterogeneous data captured by IoT solutions can inform policy making and quick responses to community events. This article introduces a novel IoT crowd monitoring solution which uses software defined networks (SDN) assisted WiFi access points as 24/7 sensors to monitor and analyze the use of physical space. Prototypes and crowd behavior models are developed using over 500 million records captured on a university campus. Besides supporting informed decision at institution level, the results can be used by individual visitors to plan or schedule their access to facilities.
MLJ: A Julia package for composable machine learning
Venue: arXiv cs.LG 2020
Year: 2020

MLJ (Machine Learing in Julia) is an open source software package providing a common interface for interacting with machine learning models written in Julia and other languages. It provides tools and meta-algorithms for selecting, tuning, evaluating, composing and comparing those models, with a focus on flexible model composition. In this design overview we detail chief novelties of the framework, together with the clear benefits of Julia over the dominant multi-language alternatives.
Evaluation metrics for behaviour modeling
Venue: arXiv cs.LG 2020
Year: 2020

A primary difficulty with unsupervised discovery of structure in large data sets is a lack of quantitative evaluation criteria. In this work, we propose and investigate several metrics for evaluating and comparing generative models of behavior learned using imitation learning. Compared to the commonly-used model log-likelihood, these criteria look at longer temporal relationships in behavior, are relevant if behavior has some properties that are inherently unpredictable, and highlight biases in the overall distribution of behaviors produced by the model. Pointwise metrics compare real to model-predicted trajectories given true past information. Distribution metrics compare statistics of the model simulating behavior in open loop, and are inspired by how experimental biologists evaluate the effects of manipulations on animal behavior. We show that the proposed metrics correspond with biologists' intuitions about behavior, and allow us to evaluate models, understand their biases, and enable us to propose new research directions.
Off-Policy Multi-Agent Decomposed Policy Gradients
Venue: arXiv cs.LG 2020
Year: 2020

Multi-agent policy gradient (MAPG) methods recently witness vigorous progress. However, there is a significant performance discrepancy between MAPG methods and state-of-the-art multi-agent value-based approaches. In this paper, we investigate causes that hinder the performance of MAPG algorithms and present a multi-agent decomposed policy gradient method (DOP). This method introduces the idea of value function decomposition into the multi-agent actor-critic framework. Based on this idea, DOP supports efficient off-policy learning and addresses the issue of centralized-decentralized mismatch and credit assignment in both discrete and continuous action spaces. We formally show that DOP critics have sufficient representational capability to guarantee convergence. In addition, empirical evaluations on the StarCraft II micromanagement benchmark and multi-agent particle environments demonstrate that DOP significantly outperforms both state-of-the-art value-based and policy-based multi-agent reinforcement learning algorithms. Demonstrative videos are available at https://sites.google.com/view/dop-mapg/.
Long-tail Session-based Recommendation
Venue: arXiv cs.IR 2020
Year: 2020

Session-based recommendation focuses on the prediction of user actions based on anonymous sessions and is a necessary method in the lack of user historical data. However, none of the existing session-based recommendation methods explicitly takes the long-tail recommendation into consideration, which plays an important role in improving the diversity of recommendation and producing the serendipity. As the distribution of items with long-tail is prevalent in session-based recommendation scenarios (e.g., e-commerce, music, and TV program recommendations), more attention should be put on the long-tail session-based recommendation. In this paper, we propose a novel network architecture, namely TailNet, to improve long-tail recommendation performance, while maintaining competitive accuracy performance compared with other methods. We start by classifying items into short-head (popular) and long-tail (niche) items based on click frequency. Then a novel is proposed and applied in TailNet to determine user preference between two types of items, so as to softly adjust and personalize recommendations. Extensive experiments on two real-world datasets verify the superiority of our method compared with state-of-the-art works.
Distributional Reinforcement Learning via Moment Matching
Venue: arXiv cs.LG 2020
Year: 2020

We consider the problem of learning a set of probability distributions from the empirical Bellman dynamics in distributional reinforcement learning (RL), a class of state-of-the-art methods that estimate the distribution, as opposed to only the expectation, of the total return. We formulate a method that learns a finite set of statistics from each return distribution via neural networks, as in (Bellemare, Dabney, and Munos 2017; Dabney et al. 2018b). Existing distributional RL methods however constrain the learned statistics to \emph{predefined} functional forms of the return distribution which is both restrictive in representation and difficult in maintaining the predefined statistics. Instead, we learn \emph{unrestricted} statistics, i.e., deterministic (pseudo-)samples, of the return distribution by leveraging a technique from hypothesis testing known as maximum mean discrepancy (MMD), which leads to a simpler objective amenable to backpropagation. Our method can be interpreted as implicitly matching all orders of moments between a return distribution and its Bellman target. We establish sufficient conditions for the contraction of the distributional Bellman operator and provide finite-sample analysis for the deterministic samples in distribution approximation. Experiments on the suite of Atari games show that our method outperforms the standard distributional RL baselines and sets a new record in the Atari games for non-distributed agents.
Dynamic Knowledge Distillation for Black-box Hypothesis Transfer Learning
Venue: arXiv cs.LG 2020
Year: 2020

In real world applications like healthcare, it is usually difficult to build a machine learning prediction model that works universally well across different institutions. At the same time, the available model is often proprietary, i.e., neither the model parameter nor the data set used for model training is accessible. In consequence, leveraging the knowledge hidden in the available model (aka. the hypothesis) and adapting it to a local data set becomes extremely challenging. Motivated by this situation, in this paper we aim to address such a specific case within the hypothesis transfer learning framework, in which 1) the source hypothesis is a black-box model and 2) the source domain data is unavailable. In particular, we introduce a novel algorithm called dynamic knowledge distillation for hypothesis transfer learning (dkdHTL). In this method, we use knowledge distillation with instance-wise weighting mechanism to adaptively transfer the "dark" knowledge from the source hypothesis to the target domain.The weighting coefficients of the distillation loss and the standard loss are determined by the consistency between the predicted probability of the source hypothesis and the target ground-truth label.Empirical results on both transfer learning benchmark datasets and a healthcare dataset demonstrate the effectiveness of our method.
Micro-expression spotting: A new benchmark
Venue: arXiv cs.CV 2020
Year: 2020

Micro-expressions (MEs) are brief and involuntary facial expressions that occur when people are trying to hide their true feelings or conceal their emotions. Based on psychology research, MEs play an important role in understanding genuine emotions, which leads to many potential applications. Therefore, ME analysis has become an attractive topic for various research areas, such as psychology, law enforcement, and psychotherapy. In the computer vision field, the study of MEs can be divided into two main tasks, spotting and recognition, which are used to identify positions of MEs in videos and determine the emotion category of the detected MEs, respectively. Recently, although much research has been done, no fully automatic system for analyzing MEs has yet been constructed on a practical level for two main reasons: most of the research on MEs only focuses on the recognition part, while abandoning the spotting task; current public datasets for ME spotting are not challenging enough to support developing a robust spotting algorithm. The contributions of this paper are threefold: (1) we introduce an extension of the SMIC-E database, namely the SMIC-E-Long database, which is a new challenging benchmark for ME spotting; (2) we suggest a new evaluation protocol that standardizes the comparison of various ME spotting techniques; (3) extensive experiments with handcrafted and deep learning-based approaches on the SMIC-E-Long database are performed for baseline evaluation.
Transferred Discrepancy: Quantifying the Difference Between Representations
Venue: arXiv cs.LG 2020
Year: 2020

Understanding what information neural networks capture is an essential problem in deep learning, and studying whether different models capture similar features is an initial step to achieve this goal. Previous works sought to define metrics over the feature matrices to measure the difference between two models. However, different metrics sometimes lead to contradictory conclusions, and there has been no consensus on which metric is suitable to use in practice. In this work, we propose a novel metric that goes beyond previous approaches. Recall that one of the most practical scenarios of using the learned representations is to apply them to downstream tasks. We argue that we should design the metric based on a similar principle. For that, we introduce the transferred discrepancy (TD), a new metric that defines the difference between two representations based on their downstream-task performance. Through an asymptotic analysis, we show how TD correlates with downstream tasks and the necessity to define metrics in such a task-dependent fashion. In particular, we also show that under specific conditions, the TD metric is closely related to previous metrics. Our experiments show that TD can provide fine-grained information for varied downstream tasks, and for the models trained from different initializations, the learned features are not the same in terms of downstream-task predictions. We find that TD may also be used to evaluate the effectiveness of different training strategies. For example, we demonstrate that the models trained with proper data augmentations that improve the generalization capture more similar features in terms of TD, while those with data augmentations that hurt the generalization will not. This suggests a training strategy that leads to more robust representation also trains models that generalize better.
Fine-grained Conflict Detection of IoT Services
Venue: arXiv cs.CY 2020
Year: 2020

We propose a novel framework to detect conflicts among IoT services in a multi-resident smart home. A fine-grained conflict model is proposed considering the functional and non-functional properties of IoT services. The proposed conflict model is designed using the concept of entropy and information gain from information theory. We use a novel algorithm based on temporal proximity to detect conflicts. Experimental results on real-world datasets show the efficiency of the proposed approach.