OpenRaiser/Intern-Atlas huggingface dataset:
On Fair Virtual Conference Scheduling: Achieving Equitable Participant and Speaker Satisfaction
Venue: arXiv cs.SI 2020
Year: 2020

The (COVID-19) pandemic-induced restrictions on travel and social gatherings have prompted most conference organizers to move their events online. However, in contrast to physical conferences, virtual conferences face a challenge in efficiently scheduling talks, accounting for the availability of participants from different time-zones as well as their interests in attending different talks. In such settings, a natural objective for the conference organizers would be to maximize some global welfare measure, such as the total expected audience participation across all talks. However, we show that optimizing for global welfare could result in a schedule that is unfair to the stakeholders, i.e., the individual utilities for participants and speakers can be highly unequal. To address the fairness concerns, we formally define fairness notions for participants and speakers, and subsequently derive suitable fairness objectives for them. We show that the welfare and fairness objectives can be in conflict with each other, and there is a need to maintain a balance between these objective while caring for them simultaneously. Thus, we propose a joint optimization framework that allows conference organizers to design talk schedules that balance (i.e., allow trade-offs) between global welfare, participant fairness and the speaker fairness objectives. We show that the optimization problem can be solved using integer linear programming, and empirically evaluate the necessity and benefits of such joint optimization approach in virtual conference scheduling.
Improving Text Relationship Modeling with Artificial Data
Venue: arXiv cs.DL 2020
Year: 2020

Data augmentation uses artificially-created examples to support supervised machine learning, adding robustness to the resulting models and helping to account for limited availability of labelled data. We apply and evaluate a synthetic data approach to relationship classification in digital libraries, generating artificial books with relationships that are common in digital libraries but not easier inferred from existing metadata. We find that for classification on whole-part relationships between books, synthetic data improves a deep neural network classifier by 91%. Further, we consider the ability of synthetic data to learn a useful new text relationship class from fully artificial training data.
Transformer in action: a comparative study of transformer-based acoustic models for large scale speech recognition applications
Venue: arXiv cs.CL 2020
Year: 2020

In this paper, we summarize the application of transformer and its streamable variant, Emformer based acoustic model for large scale speech recognition applications. We compare the transformer based acoustic models with their LSTM counterparts on industrial scale tasks. Specifically, we compare Emformer with latency-controlled BLSTM (LCBLSTM) on medium latency tasks and LSTM on low latency tasks. On a low latency voice assistant task, Emformer gets 24% to 26% relative word error rate reductions (WERRs). For medium latency scenarios, comparing with LCBLSTM with similar model size and latency, Emformer gets significant WERR across four languages in video captioning datasets with 2-3 times inference real-time factors reduction.
MILR: Mathematically Induced Layer Recovery for Plaintext Space Error Correction of CNNs
Venue: arXiv cs.LG 2020
Year: 2020

The increased use of Convolutional Neural Networks (CNN) in mission critical systems has increased the need for robust and resilient networks in the face of both naturally occurring faults as well as security attacks. The lack of robustness and resiliency can lead to unreliable inference results. Current methods that address CNN robustness require hardware modification, network modification, or network duplication. This paper proposes MILR a software based CNN error detection and error correction system that enables self-healing of the network from single and multi bit errors. The self-healing capabilities are based on mathematical relationships between the inputs,outputs, and parameters(weights) of a layers, exploiting these relationships allow the recovery of erroneous parameters (weights) throughout a layer and the network. MILR is suitable for plaintext-space error correction (PSEC) given its ability to correct whole-weight and even whole-layer errors in CNNs.
Melody-Conditioned Lyrics Generation with SeqGANs
Venue: arXiv cs.SD 2020
Year: 2020

Automatic lyrics generation has received attention from both music and AI communities for years. Early rule-based approaches have~---due to increases in computational power and evolution in data-driven models---~mostly been replaced with deep-learning-based systems. Many existing approaches, however, either rely heavily on prior knowledge in music and lyrics writing or oversimplify the task by largely discarding melodic information and its relationship with the text. We propose an end-to-end melody-conditioned lyrics generation system based on Sequence Generative Adversarial Networks (SeqGAN), which generates a line of lyrics given the corresponding melody as the input. Furthermore, we investigate the performance of the generator with an additional input condition: the theme or overarching topic of the lyrics to be generated. We show that the input conditions have no negative impact on the evaluation metrics while enabling the network to produce more meaningful results.
Design of Bilayer and Multi-layer LDPC Ensembles from Individual Degree Distributions
Venue: arXiv cs.IT 2020
Year: 2020

A new approach for designing bilayer and multi-layer LDPC codes is proposed and studied in the asymptotic regime. The ensembles are defined through individual uni-variate degree distributions, one for each layer. We present a construction that: 1) enables low-complexity decoding for high-SNR channel instances, 2) provably approaches capacity for low-SNR instances, 3) scales linearly (in terms of design complexity) in the number of layers. For the setup where decoding the second layer is significantly more costly than the first layer, we propose an optimal-cost decoding schedule and study the trade-off between code rate and decoding cost.
Deep Networks from the Principle of Rate Reduction
Venue: arXiv cs.LG 2020
Year: 2020

This work attempts to interpret modern deep (convolutional) networks from the principles of rate reduction and (shift) invariant classification. We show that the basic iterative gradient ascent scheme for optimizing the rate reduction of learned features naturally leads to a multi-layer deep network, one iteration per layer. The layered architectures, linear and nonlinear operators, and even parameters of the network are all explicitly constructed layer-by-layer in a forward propagation fashion by emulating the gradient scheme. All components of this "white box" network have precise optimization, statistical, and geometric interpretation. This principled framework also reveals and justifies the role of multi-channel lifting and sparse coding in early stage of deep networks. Moreover, all linear operators of the so-derived network naturally become multi-channel convolutions when we enforce classification to be rigorously shift-invariant. The derivation also indicates that such a convolutional network is significantly more efficient to construct and learn in the spectral domain. Our preliminary simulations and experiments indicate that so constructed deep network can already learn a good discriminative representation even without any back propagation training.
SFU-Store-Nav: A Multimodal Dataset for Indoor Human Navigation
Venue: arXiv cs.CV 2020
Year: 2020

This article describes a dataset collected in a set of experiments that involves human participants and a robot. The set of experiments was conducted in the computing science robotics lab in Simon Fraser University, Burnaby, BC, Canada, and its aim is to gather data containing common gestures, movements, and other behaviours that may indicate humans' navigational intent relevant for autonomous robot navigation. The experiment simulates a shopping scenario where human participants come in to pick up items from his/her shopping list and interact with a Pepper robot that is programmed to help the human participant. We collected visual data and motion capture data from 108 human participants. The visual data contains live recordings of the experiments and the motion capture data contains the position and orientation of the human participants in world coordinates. This dataset could be valuable for researchers in the robotics, machine learning and computer vision community.
No Perfect Outdoors: Towards A Deep Profiling of GNSS-based Location Contexts
Venue: arXiv cs.NI 2020
Year: 2020

While both outdoor and indoor localization methods are flourishing, how to properly marry them to offer pervasive localizability in urban areas remains open. Recently proposals on indoor-outdoor detection make the first step towards such an integration, yet complicated urban environments render such a binary classification incompetent. In this paper, we intend to fully explore raw GNSS measurements in order to better characterize the diversified urban environments. Essentially, we tackle the challenges introduced by the complex GNSS data and apply a deep learning model to identify representations for respective location contexts. We further develop two preliminary applications of our deep profiling. On one hand, we offer a more fine-grained semantic classification than binary indoor-outdoor detection. On the other hand, we derive a GPS error indicator more meaningful than that provided by Google Maps. These results are all corroborated by our extensive data collection and trace-driven evaluations.
Porting the microphysics model CASIM to GPU and KNL Cray machines
Venue: arXiv cs.DC 2020
Year: 2020

CASIM is a microphysics scheme which calculates the interaction between moisture droplets in the atmosphere and forms a critical part of weather and climate modelling codes. However the calculations involved are computationally intensive and so investigating whether CASIM can take advantage of novel hardware architectures and the likely increase in performance this might afford makes sense. In this paper we present work done in porting CASIM to GPUs via the directive driven OpenACC and also modifying CASIM to take advantage of the Knights Landing (KNL) processor using OpenMP. Due to the design, models extracting out specific computational kernels for offload to the GPU proved suboptimal and instead the entire scheme was ported over to the GPU. We consider the suitability and maturity of OpenACC for this approach as well as important optimisations that were identified. Enabling CASIM to take advantage of the KNL was significantly easier, but still required careful experimentation to understand the best design and configuration. The performance of both versions of CASIM, in comparison to the latest generation of CPUs is discussed, before identifying lessons learnt about the suitability of CASIM and other similar models for these architectures. The result of this work are versions of CASIM which show promising performance benefits when utilising both GPUs and KNLs and enable the communities to take advantage of these technologies, in addition to general techniques that can be applied to other similar weather and climate models.
INT8 Winograd Acceleration for Conv1D Equipped ASR Models Deployed on Mobile Devices
Venue: arXiv cs.SD 2020
Year: 2020

The intensive computation of Automatic Speech Recognition (ASR) models obstructs them from being deployed on mobile devices. In this paper, we present a novel quantized Winograd optimization pipeline, which combines the quantization and fast convolution to achieve efficient inference acceleration on mobile devices for ASR models. To avoid the information loss due to the combination of quantization and Winograd convolution, a Range-Scaled Quantization (RSQ) training method is proposed to expand the quantized numerical range and to distill knowledge from high-precision values. Moreover, an improved Conv1D equipped DFSMN (ConvDFSMN) model is designed for mobile deployment. We conduct extensive experiments on both ConvDFSMN and Wav2letter models. Results demonstrate the models can be effectively optimized with the proposed pipeline. Especially, Wav2letter achieves 1.48* speedup with an approximate 0.07% WER decrease on ARMv7-based mobile devices.
Sample-Optimal and Efficient Learning of Tree Ising models
Venue: arXiv cs.LG 2020
Year: 2020

We show that $n$-variable tree-structured Ising models can be learned computationally-efficiently to within total variation distance $ε$ from an optimal $O(n \ln n/ε^2)$ samples, where $O(\cdot)$ hides an absolute constant which, importantly, does not depend on the model being learned - neither its tree nor the magnitude of its edge strengths, on which we place no assumptions. Our guarantees hold, in fact, for the celebrated Chow-Liu [1968] algorithm, using the plug-in estimator for estimating mutual information. While this (or any other) algorithm may fail to identify the structure of the underlying model correctly from a finite sample, we show that it will still learn a tree-structured model that is $ε$-close to the true one in total variation distance, a guarantee called "proper learning." Our guarantees do not follow from known results for the Chow-Liu algorithm and the ensuing literature on learning graphical models, including a recent renaissance of algorithms on this learning challenge, which only yield asymptotic consistency results, or sample-inefficient and/or time-inefficient algorithms, unless further assumptions are placed on the graphical model, such as bounds on the "strengths" of the model's edges/hyperedges. While we establish guarantees for a widely known and simple algorithm, the analysis that this algorithm succeeds and is sample-optimal is quite complex, requiring a hierarchical classification of the edges into layers with different reconstruction guarantees, depending on their strength, combined with delicate uses of the subadditivity of the squared Hellinger distance over graphical models to control the error accumulation.
Dynamic Bayesian Approach for decision-making in Ego-Things
Venue: arXiv cs.LG 2020
Year: 2020

This paper presents a novel approach to detect abnormalities in dynamic systems based on multisensory data and feature selection. The proposed method produces multiple inference models by considering several features of the observed data. This work facilitates the obtainment of the most precise features for predicting future instances and detecting abnormalities. Growing neural gas (GNG) is employed for clustering multisensory data into a set of nodes that provide a semantic interpretation of data and define local linear models for prediction purposes. Our method uses a Markov Jump particle filter (MJPF) for state estimation and abnormality detection. The proposed method can be used for selecting the optimal set features to be shared in networking operations such that state prediction, decision-making, and abnormality detection processes are favored. This work is evaluated by using a real dataset consisting of a moving vehicle performing some tasks in a controlled environment.
Visually Guided Balloon Popping with an Autonomous MAV at MBZIRC 2020
Venue: arXiv cs.RO 2020
Year: 2020

Visually guided control of micro aerial vehicles (MAV) demands for robust real-time perception, fast trajectory generation, and a capable flight platform. We present a fully autonomous MAV that is able to pop balloons, relying only on onboard sensing and computing. The system is evaluated with real robot experiments during the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2020 where it showed its resilience and speed. In all three competition runs we were able to pop all five balloons in less than two minutes flight time with a single MAV.
Real-time Tropical Cyclone Intensity Estimation by Handling Temporally Heterogeneous Satellite Data
Venue: arXiv cs.CV 2020
Year: 2020

Analyzing big geophysical observational data collected by multiple advanced sensors on various satellite platforms promotes our understanding of the geophysical system. For instance, convolutional neural networks (CNN) have achieved great success in estimating tropical cyclone (TC) intensity based on satellite data with fixed temporal frequency (e.g., 3 h). However, to achieve more timely (under 30 min) and accurate TC intensity estimates, a deep learning model is demanded to handle temporally-heterogeneous satellite observations. Specifically, infrared (IR1) and water vapor (WV) images are available under every 15 minutes, while passive microwave rain rate (PMW) is available for about every 3 hours. Meanwhile, the visible (VIS) channel is severely affected by noise and sunlight intensity, making it difficult to be utilized. Therefore, we propose a novel framework that combines generative adversarial network (GAN) with CNN. The model utilizes all data, including VIS and PMW information, during the training phase and eventually uses only the high-frequent IR1 and WV data for providing intensity estimates during the predicting phase. Experimental results demonstrate that the hybrid GAN-CNN framework achieves comparable precision to the state-of-the-art models, while possessing the capability of increasing the maximum estimation frequency from 3 hours to less than 15 minutes.
On Learning Continuous Pairwise Markov Random Fields
Venue: arXiv cs.LG 2020
Year: 2020

We consider learning a sparse pairwise Markov Random Field (MRF) with continuous-valued variables from i.i.d samples. We adapt the algorithm of Vuffray et al. (2019) to this setting and provide finite-sample analysis revealing sample complexity scaling logarithmically with the number of variables, as in the discrete and Gaussian settings. Our approach is applicable to a large class of pairwise MRFs with continuous variables and also has desirable asymptotic properties, including consistency and normality under mild conditions. Further, we establish that the population version of the optimization criterion employed in Vuffray et al. (2019) can be interpreted as local maximum likelihood estimation (MLE). As part of our analysis, we introduce a robust variation of sparse linear regression a` la Lasso, which may be of interest in its own right.
Generative Adversarial Networks in Human Emotion Synthesis:A Review
Venue: arXiv cs.CV 2020
Year: 2020

Synthesizing realistic data samples is of great value for both academic and industrial communities. Deep generative models have become an emerging topic in various research areas like computer vision and signal processing. Affective computing, a topic of a broad interest in computer vision society, has been no exception and has benefited from generative models. In fact, affective computing observed a rapid derivation of generative models during the last two decades. Applications of such models include but are not limited to emotion recognition and classification, unimodal emotion synthesis, and cross-modal emotion synthesis. As a result, we conducted a review of recent advances in human emotion synthesis by studying available databases, advantages, and disadvantages of the generative models along with the related training strategies considering two principal human communication modalities, namely audio and video. In this context, facial expression synthesis, speech emotion synthesis, and the audio-visual (cross-modal) emotion synthesis is reviewed extensively under different application scenarios. Gradually, we discuss open research problems to push the boundaries of this research area for future works.
papaya2: 2D Irreducible Minkowski Tensor computation
Venue: arXiv cs.GR 2020
Year: 2020

A common challenge in scientific and technical domains is the quantitative description of geometries and shapes, e.g. in the analysis of microscope imagery or astronomical observation data. Frequently, it is desirable to go beyond scalar shape metrics such as porosity and surface to volume ratios because the samples are anisotropic or because direction-dependent quantities such as conductances or elasticity are of interest. Minkowski Tensors are a systematic family of versatile and robust higher-order shape descriptors that allow for shape characterization of arbitrary order and promise a path to systematic structure-function relationships for direction-dependent properties. Papaya2 is a software to calculate 2D higher-order shape metrics with a library interface, support for Irreducible Minkowski Tensors and interpolated marching squares. Extensions to Matlab, JavaScript and Python are provided as well. While the tensor of inertia is computed by many tools, we are not aware of other open-source software which provides higher-rank shape characterization in 2D.
Micromobility in Smart Cities: A Closer Look at Shared Dockless E-Scooters via Big Social Data
Venue: arXiv cs.SI 2020
Year: 2020

The micromobility is shaping first- and last-mile travels in urban areas. Recently, shared dockless electric scooters (e-scooters) have emerged as a daily alternative to driving for short-distance commuters in large cities due to the affordability, easy accessibility via an app, and zero emissions. Meanwhile, e-scooters come with challenges in city management, such as traffic rules, public safety, parking regulations, and liability issues. In this paper, we collected and investigated 5.8 million scooter-tagged tweets and 144,197 images, generated by 2.7 million users from October 2018 to March 2020, to take a closer look at shared e-scooters via crowdsourcing data analytics. We profiled e-scooter usages from spatial-temporal perspectives, explored different business roles (i.e., riders, gig workers, and ridesharing companies), examined operation patterns (e.g., injury types, and parking behaviors), and conducted sentiment analysis. To our best knowledge, this paper is the first large-scale systematic study on shared e-scooters using big social data.
Exploring complex networks with the ICON R package
Venue: arXiv cs.SI 2020
Year: 2020

We introduce ICON, an R package that contains 1075 complex network datasets in a standard edgelist format. All provided datasets have associated citations and have been indexed by the Colorado Index of Complex Networks - also referred to as ICON. In addition to supplying a large and diverse corpus of useful real-world networks, ICON also implements an S3 generic to work with the network and ggnetwork R packages for network analysis and visualization, respectively. Sample code in this report also demonstrates how ICON can be used in conjunction with the igraph package. Currently, the Comprehensive R Archive Network hosts ICON v0.4.0. We hope that ICON will serve as a standard corpus for complex network research and prevent redundant work that would be otherwise necessary by individual research groups. The open source code for ICON and for this reproducible report can be found at https://github.com/rrrlw/ICON.