pwc-archive/papers-with-abstracts huggingface dataset:
A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions
Authors: Pengzhen Ren, Yun Xiao, Xiaojun Chang, Po-Yao Huang, Zhihui Li, Xiaojiang Chen, Xin Wang
Date: 2020-06-01 00:00:00
Tasks: Neural Architecture Search, Survey

Deep learning has made breakthroughs and substantial in many fields due to its powerful automatic representation capabilities. It has been proven that neural architecture design is crucial to the feature representation of data and the final performance. However, the design of the neural architecture heavily relies on the researchers' prior knowledge and experience. And due to the limitations of human' inherent knowledge, it is difficult for people to jump out of their original thinking paradigm and design an optimal model. Therefore, an intuitive idea would be to reduce human intervention as much as possible and let the algorithm automatically design the neural architecture. Neural Architecture Search (NAS) is just such a revolutionary algorithm, and the related research work is complicated and rich. Therefore, a comprehensive and systematic survey on the NAS is essential. Previously related surveys have begun to classify existing work mainly based on the key components of NAS: search space, search strategy, and evaluation strategy. While this classification method is more intuitive, it is difficult for readers to grasp the challenges and the landmark work involved. Therefore, in this survey, we provide a new perspective: beginning with an overview of the characteristics of the earliest NAS algorithms, summarizing the problems in these early NAS algorithms, and then providing solutions for subsequent related research work. Besides, we conduct a detailed and comprehensive analysis, comparison, and summary of these works. Finally, we provide some possible future research directions.
Constrained Variational Autoencoder for improving EEG based Speech Recognition Systems
Authors: Gautam Krishna, Co Tran, Mason Carnahan, Ahmed Tewfik
Date: 2020-06-01 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Brain Computer Interface, EEG, Electroencephalogram (EEG), speech-recognition, Speech Recognition

In this paper we introduce a recurrent neural network (RNN) based variational autoencoder (VAE) model with a new constrained loss function that can generate more meaningful electroencephalography (EEG) features from raw EEG features to improve the performance of EEG based speech recognition systems. We demonstrate that both continuous and isolated speech recognition systems trained and tested using EEG features generated from raw EEG features using our VAE model results in improved performance and we demonstrate our results for a limited English vocabulary consisting of 30 unique sentences for continuous speech recognition and for an English vocabulary consisting of 2 unique sentences for isolated speech recognition. We compare our method with another recently introduced method described by authors in [1] to improve the performance of EEG based continuous speech recognition systems and we demonstrate that our method outperforms their method as vocabulary size increases when trained and tested using the same data set. Even though we demonstrate results only for automatic speech recognition (ASR) experiments in this paper, the proposed VAE model with constrained loss function can be extended to a variety of other EEG based brain computer interface (BCI) applications.
A Novel Approach for Generating SPARQL Queries from RDF Graphs
Authors: Emna Jabri
Date: 2020-05-30 00:00:00
Tasks: Management

This work is done as part of a research master's thesis project. The goal is to generate SPARQL queries based on user-supplied keywords to query RDF graphs. To do this, we first transformed the input ontology into an RDF graph that reflects the semantics represented in the ontology. Subsequently, we stored this RDF graph in the Neo4j graphical database to ensure efficient and persistent management of RDF data. At the time of the interrogation, we studied the different possible and desired interpretations of the request originally made by the user. We have also proposed to carry out a sort of transformation between the two query languages SPARQL and Cypher, which is specific to Neo4j. This allows us to implement the architecture of our system over a wide variety of BD-RDFs providing their query languages, without changing any of the other components of the system. Finally, we tested and evaluated our tool using different test bases, and it turned out that our tool is comprehensive, effective, and powerful enough.
Study on Key Technologies of Transit Passengers Travel Pattern Mining and Applications based on Multiple Sources of Data
Authors: Yongxin Liu
Date: 2020-05-26 00:00:00

In this research, we propose a series of methodologies to mine transit riders travel pattern and behavioral preferences, and then we use these knowledges to adjust and optimize the transit systems. Contributions are: 1) To increase the data validity: a) we propose a novel approach to rectify the time discrepancy of data between the AFC (Automated Fare Collection) systems and AVL (Automated Vehicle Location) system, our approach transforms data events into signals and applies time domain correlation the detect and rectify their relative discrepancies. b) By combining historical data and passengers ticketing time stamps, we induct and compensate missing information in AVL datasets. 2) To infer passengers alighting point, we introduce a maximum probabilistic model incorporating passengers home place to recover their complete transit trajectory from semi-complete boarding records.Then we propose an enhance activity identification algorithm which is capable of specifying passengers short-term activity from ordinary transfers. Finally, we analyze the temporal-spatial characteristic of transit ridership. 3) To discover passengers travel demands. We integrate each passengers trajectory data in multiple days and construct a Hybrid Trip Graph (HTG). We then use a depth search algorithm to derive the spatially closed transit trip chains; Finally, we use closed transit trip chains of passengers to study their travel pattern from various perspectives. Finally, we analyze urban transit corridors by aggregating the passengers critical transit chains.4) We derive eight influential factors, and then construct passengers choice models under various scenarios. Next, we validate our model using ridership re-distribute simulations. Finally, we conduct a comprehensive analysis on passengers temporal choice preference and use this information to optimize urban transit systems.
Event-based visual place recognition with ensembles of temporal windows
Authors: Tobias Fischer, Michael Milford
Date: 2020-05-22 00:00:00
Tasks: Image Reconstruction, Visual Place Recognition

Event cameras are bio-inspired sensors capable of providing a continuous stream of events with low latency and high dynamic range. As a single event only carries limited information about the brightness change at a particular pixel, events are commonly accumulated into spatio-temporal windows for further processing. However, the optimal window length varies depending on the scene, camera motion, the task being performed, and other factors. In this research, we develop a novel ensemble-based scheme for combining temporal windows of varying lengths that are processed in parallel. For applications where the increased computational requirements of this approach are not practical, we also introduce a new "approximate" ensemble scheme that achieves significant computational efficiencies without unduly compromising the original performance gains provided by the ensemble approach. We demonstrate our ensemble scheme on the visual place recognition (VPR) task, introducing a new Brisbane-Event-VPR dataset with annotated recordings captured using a DAVIS346 color event camera. We show that our proposed ensemble scheme significantly outperforms all the single-window baselines and conventional model-based ensembles, irrespective of the image reconstruction and feature extraction methods used in the VPR pipeline, and evaluate which ensemble combination technique performs best. These results demonstrate the significant benefits of ensemble schemes for event camera processing in the VPR domain and may have relevance to other related processes, including feature tracking, visual-inertial odometry, and steering prediction in driving.
A Spatial-Temporal Attention-Based Method and a New Dataset for Remote Sensing Image Change Detection
Authors: Hao Chen, Zhenwei Shi
Date: 2020-05-22 00:00:00
Proceeding: Remote Sensing 2020 5
Tasks: Change Detection

Remote sensing image change detection (CD) is done to identify desired significant changes between bitemporal images. Given two co-registered images taken at different times, the illumination variations and misregistration errors overwhelm the real object changes. Exploring the relationships among different spatial–temporal pixels may improve the performances of CD methods. In our work, we propose a novel Siamese-based spatial–temporal attention neural network. In contrast to previous methods that separately encode the bitemporal images without referring to any useful spatial–temporal dependency, we design a CD self-attention mechanism to model the spatial–temporal relationships. We integrate a new CD self-attention module in the procedure of feature extraction. Our self-attention module calculates the attention weights between any two pixels at different times and positions and uses them to generate more discriminative features. Considering that the object may have different scales, we partition the image into multi-scale subregions and introduce the self-attention in each subregion. In this way, we could capture spatial–temporal dependencies at various scales, thereby generating better representations to accommodate objects of various sizes. We also introduce a CD dataset LEVIR-CD, which is two orders of magnitude larger than other public datasets of this field. LEVIR-CD consists of a large set of bitemporal Google Earth images, with 637 image pairs (1024  1024) and over 31 k independently labeled change instances. Our proposed attention module improves the F1-score of our baseline model from 83.9 to 87.3 with acceptable computational overhead. Experimental results on a public remote sensing image CD dataset show our method outperforms several other state-of-the-art methods.
Epoch-Synchronous Overlap-Add (ESOLA) for Time- and Pitch-Scale Modification of Speech Signals
Authors: Sunil Rudresh, Aditya Vasisht, Karthika Vijayan, Chandra Sekhar Seelamantula
Date: 2018-01-19 00:00:00
Tasks: Speech Synthesis, Voice Conversion

Time- and pitch-scale modifications of speech signals find important applications in speech synthesis, playback systems, voice conversion, learning/hearing aids, etc.. There is a requirement for computationally efficient and real-time implementable algorithms. In this paper, we propose a high quality and computationally efficient time- and pitch-scaling methodology based on the glottal closure instants (GCIs) or epochs in speech signals. The proposed algorithm, termed as epoch-synchronous overlap-add time/pitch-scaling (ESOLA-TS/PS), segments speech signals into overlapping short-time frames and then the adjacent frames are aligned with respect to the epochs and the frames are overlap-added to synthesize time-scale modified speech. Pitch scaling is achieved by resampling the time-scaled speech by a desired sampling factor. We also propose a concept of epoch embedding into speech signals, which facilitates the identification and time-stamping of samples corresponding to epochs and using them for time/pitch-scaling to multiple scaling factors whenever desired, thereby contributing to faster and efficient implementation. The results of perceptual evaluation tests reported in this paper indicate the superiority of ESOLA over state-of-the-art techniques. ESOLA significantly outperforms the conventional pitch synchronous overlap-add (PSOLA) techniques in terms of perceptual quality and intelligibility of the modified speech. Unlike the waveform similarity overlap-add (WSOLA) or synchronous overlap-add (SOLA) techniques, the ESOLA technique has the capability to do exact time-scaling of speech with high quality to any desired modification factor within a range of 0.5 to 2. Compared to synchronous overlap-add with fixed synthesis (SOLAFS), the ESOLA is computationally advantageous and at least three times faster.
Implementation of UAV Coordination Based on a Hierarchical Multi-UAV Simulation Platform
Authors: Kun Xiao, Lan Ma, Shaochang Tan, Yirui Cong, Xiangke Wang
Date: 2020-05-03 00:00:00

In this paper, a hierarchical multi-UAV simulation platform,called XTDrone, is designed for UAV swarms, which is completely open-source 4 . There are six layers in XTDrone: communication, simulator,low-level control, high-level control, coordination, and human interac-tion layers. XTDrone has three advantages. Firstly, the simulation speedcan be adjusted to match the computer performance, based on the lock-step mode. Thus, the simulations can be conducted on a work stationor on a personal laptop, for different purposes. Secondly, a simplifiedsimulator is also developed which enables quick algorithm designing sothat the approximated behavior of UAV swarms can be observed inadvance. Thirdly, XTDrone is based on ROS, Gazebo, and PX4, andhence the codes in simulations can be easily transplanted to embeddedsystems. Note that XTDrone can support various types of multi-UAVmissions, and we provide two important demos in this paper: one is aground-station-based multi-UAV cooperative search, and the other is adistributed UAV formation flight, including consensus-based formationcontrol, task assignment, and obstacle avoidance.
Dynamic Fluid Surface Reconstruction Using Deep Neural Network
Authors: Simron Thapa,  Nianyi Li,  Jinwei Ye
Date: 2020-06-01 00:00:00
Proceeding: CVPR 2020 6
Tasks: Surface Reconstruction

Recovering the dynamic fluid surface is a long-standing challenging problem in computer vision. Most existing image-based methods require multiple views or a dedicated imaging system. Here we present a learning-based single-image approach for 3D fluid surface reconstruction. Specifically, we design a deep neural network that estimates the depth and normal maps of a fluid surface by analyzing the refractive distortion of a reference background image. Due to the dynamic nature of fluid surfaces, our network uses recurrent layers that carry temporal information from previous frames to achieve spatio-temporally consistent reconstruction given a video input. Due to the lack of fluid data, we synthesize a large fluid dataset using physics-based fluid modeling and rendering techniques for network training and validation. Through experiments on simulated and real captured fluid images, we demonstrate that our proposed deep neural network trained on our fluid dataset can recover dynamic 3D fluid surfaces with high accuracy.
A Lighting-Invariant Point Processor for Shading
Authors: Kathryn Heal,  Jialiang Wang,  Steven J. Gortler,  Todd Zickler
Date: 2020-06-01 00:00:00
Proceeding: CVPR 2020 6

Under the conventional diffuse shading model with unknown directional lighting, the set of quadratic surface shapes that are consistent with the spatial derivatives of intensity at a single image point is a two-dimensional algebraic variety embedded in the five-dimensional space of quadratic shapes. We describe the geometry of this variety, and we introduce a concise feedforward model that computes an explicit, differentiable approximation of the variety from the intensity and its derivatives at any single image point. The result is a parallelizable processor that operates at each image point and produces a lighting-invariant descriptor of the continuous set of compatible surface shapes at the point. We describe two applications of this processor: two-shot uncalibrated photometric stereo and quadratic-surface shape from shading.
Peek-a-Boo: Occlusion Reasoning in Indoor Scenes With Plane Representations
Authors: Ziyu Jiang,  Buyu Liu,  Samuel Schulter,  Zhangyang Wang,  Manmohan Chandraker
Date: 2020-06-01 00:00:00
Proceeding: CVPR 2020 6
Tasks: Scene Understanding

We address the challenging task of occlusion-aware indoor 3D scene understanding. We represent scenes by a set of planes, where each one is defined by its normal, offset and two masks outlining (i) the extent of the visible part and (ii) the full region that consists of both visible and occluded parts of the plane. We infer these planes from a single input image with a novel neural network architecture. It consists of a two-branch category-specific module that aims to predict layout and objects of the scene separately so that different types of planes can be handled better. We also introduce a novel loss function based on plane warping that can leverage multiple views at training time for improved occlusion-aware reasoning. In order to train and evaluate our occlusion-reasoning model, we use the ScanNet dataset and propose (i) a strategy to automatically extract ground truth for both visible and hidden regions and (ii) a new evaluation metric that specifically focuses on the prediction in hidden regions. We empirically demonstrate that our proposed approach can achieve higher accuracy for occlusion reasoning compared to competitive baselines on the ScanNet dataset, e.g. 42.65% relative improvement on hidden regions.
Skeleton-Based Action Recognition With Shift Graph Convolutional Network
Authors: Ke Cheng,  Yifan Zhang,  Xiangyu He,  Weihan Chen,  Jian Cheng,  Hanqing Lu
Date: 2020-06-01 00:00:00
Proceeding: CVPR 2020 6
Tasks: Action Recognition, Skeleton Based Action Recognition

Action recognition with skeleton data is attracting more attention in computer vision. Recently, graph convolutional networks (GCNs), which model the human body skeletons as spatiotemporal graphs, have obtained remarkable performance. However, the computational complexity of GCN-based methods are pretty heavy, typically over 15 GFLOPs for one action sample. Recent works even reach about 100 GFLOPs. Another shortcoming is that the receptive fields of both spatial graph and temporal graph are inflexible. Although some works enhance the expressiveness of spatial graph by introducing incremental adaptive modules, their performance is still limited by regular GCN structures. In this paper, we propose a novel shift graph convolutional network (Shift-GCN) to overcome both shortcomings. Instead of using heavy regular graph convolutions, our Shift-GCN is composed of novel shift graph operations and lightweight point-wise convolutions, where the shift graph operations provide flexible receptive fields for both spatial graph and temporal graph. On three datasets for skeleton-based action recognition, the proposed Shift-GCN notably exceeds the state-of-the-art methods with more than 10 times less computational complexity.
Dynamic Multiscale Graph Neural Networks for 3D Skeleton Based Human Motion Prediction
Authors: Maosen Li,  Siheng Chen,  Yangheng Zhao,  Ya Zhang,  Yanfeng Wang,  Qi Tian
Date: 2020-06-01 00:00:00
Proceeding: CVPR 2020 6
Tasks: Decoder, Human motion prediction, motion prediction

We propose novel dynamic multiscale graph neural networks (DMGNN) to predict 3D skeleton-based human motions. The core idea of DMGNN is to use a multiscale graph to comprehensively model the internal relations of a human body for motion feature learning. This multiscale graph is adaptive during training and dynamic across network layers. Based on this graph, we propose a multiscale graph computational unit (MGCU) to extract features at individual scales and fuse features across scales. The entire model is action-category-agnostic and follows an encoder-decoder framework. The encoder consists of a sequence of MGCUs to learn motion features. The decoder uses a proposed graph-based gate recurrent unit to generate future poses. Extensive experiments show that the proposed DMGNN outperforms state-of-the-art methods in both short and long-term predictions on the datasets of Human 3.6M and CMU Mocap. We further investigate the learned multiscale graphs for the interpretability. The codes could be downloaded from https://github.com/limaosen0/DMGNN.
Towards Verifying Robustness of Neural Networks Against A Family of Semantic Perturbations
Authors: Jeet Mohapatra,  Tsui-Wei Weng,  Pin-Yu Chen,  Sijia Liu,  Luca Daniel
Date: 2020-06-01 00:00:00
Proceeding: CVPR 2020 6
Tasks: image-classification, Image Classification

Verifying robustness of neural networks given a specified threat model is a fundamental yet challenging task. While current verification methods mainly focus on the l_p-norm threat model of the input instances, robustness verification against semantic adversarial attacks inducing large l_p-norm perturbations, such as color shifting and lighting adjustment, are beyond their capacity. To bridge this gap, we propose Semantify-NN, a model-agnostic and generic robustness verification approach against semantic perturbations for neural networks. By simply inserting our proposed semantic perturbation layers (SP-layers) to the input layer of any given model, Semantify-NN is model-agnostic, and any l_p-norm based verification tools can be used to verify the model robustness against semantic perturbations. We illustrate the principles of designing the SP-layers and provide examples including semantic perturbations to image classification in the space of hue, saturation, lightness, brightness, contrast and rotation, respectively. In addition, an efficient refinement technique is proposed to further significantly improve the semantic certificate. Experiments on various network architectures and different datasets demonstrate the superior verification performance of Semantify-NN over l_p-norm-based verification frameworks that naively convert semantic perturbation to l_p-norm. The results show that Semantify-NN can support robustness verification against a wide range of semantic perturbations.
How Does Noise Help Robustness? Explanation and Exploration under the Neural SDE Framework
Authors: Xuanqing Liu,  Tesi Xiao,  Si Si,  Qin Cao,  Sanjiv Kumar,  Cho-Jui Hsieh
Date: 2020-06-01 00:00:00
Proceeding: CVPR 2020 6

Neural Ordinary Differential Equation (Neural ODE) has been proposed as a continuous approximation to the ResNet architecture. Some commonly used regularization mechanisms in discrete neural networks (e.g., dropout, Gaussian noise) are missing in current Neural ODE networks. In this paper, we propose a new continuous neural network framework called Neural Stochastic Differential Equation (Neural SDE), which naturally incorporates various commonly used regularization mechanisms based on random noise injection. For regularization purposes, our framework includes multiple types of noise patterns, such as dropout, additive, and multiplicative noise, which are common in plain neural networks. We provide some theoretical analyses explaining the improved robustness of our models against input perturbations. Furthermore, we demonstrate that the Neural SDE network can achieve better generalization than the Neural ODE and is more resistant to adversarial and non-adversarial input perturbations.