pwc-archive/papers-with-abstracts huggingface dataset:
Sparse Array Beampattern Synthesis via Majorization-Based ADMM
Authors: Tong Wei, Linlong Wu, M. R. Bhavani Shankar
Date: 2021-04-09 00:00:00

Beampattern synthesis is a key problem in many wireless applications. With the increasing scale of MIMO antenna array, it is highly desired to conduct beampattern synthesis on a sparse array to reduce the power and hardware cost. In this paper, we consider conducting beampattern synthesis and sparse array construction jointly. In the formulated problem, the beampattern synthesis is designed by minimizing the matching error to the beampattern template, and the Shannon entropy function is first introduced to impose the sparsity of the array. Then, for this nonconvex problem, an iterative method is proposed by leveraging on the alternating direction multiplier method (ADMM) and the majorization minimization (MM). Simulation results demonstrate that, compared with the benchmark, our approach achieves a good trade-off between array sparsity and beampattern matching error with less runtime.
Cascade-Forward Neural Network Based on Resilient Backpropagation for Simultaneous Parameters and State Space Estimations of Brushed DC Machines
Authors: Hacene Mellah, Kamel Eddine Hemsas, Rachid Taleb
Date: 2021-03-31 00:00:00

A sensorless speed, average temperature and resistance estimation technique based on Neural Network (NN) for brushed DC machines is proposed in this paper. The literature on parameters and state spaces estimations of the Brushed DC machines, shows a variety of approaches. However, these observers are sensitive to a noise, on the model accuracy also are difficult to stabilize and to converge. Furthermore, the majority of earlier works, estimate either the speed or the temperature or the winding resistance. According to the literatures, the Resilient backpropagation (RBP) as is the known as the faster BP algorithm, Cascade-Forward Neural Network (CFNN), is known as the among accelerated learning backpropagation algorithms, that's why where it is found in several researches, also in several applications in these few years. The main objective of this paper is to introduce an intelligent sensor based on resilient BP to estimate simultaneously the speed, armature temperature and resistance of brushed DC machines only from the measured current and voltage. A comparison between the obtained results and the results of traditional estimator has been made to prove the ability of the proposed method. This method can be embedded in thermal monitoring systems, in high performance motor drives.
Oscillations and Bifurcation Structure of Reaction-Diffusion Model for Cell Polarity Formation
Authors: Masataka Kuwamura, Hirofumi Izuhara, Shin-ichiro Ei
Date: 2021-04-09 00:00:00

We investigate the oscillatory dynamics and bifurcation structure of a reaction-diffusion system with bistable nonlinearity and mass conservation, which was proposed by [Otsuji et al, PLoS Comp. Biol. 3 (2007), e108]. The system is a useful model for understanding cell polarity formation. We show that this model exhibits four different spatiotemporal patterns including two types of oscillatory patterns, which can be regarded as cell polarity oscillations with the reversal and non-reversal of polarity, respectively. The trigger causing these patterns is a diffusion-driven (Turing-like) instability. Moreover, we investigate the effects of extracellular signals on the cell polarity oscillations.
Functional quantization of rough volatility and applications to volatility derivatives
Authors: Ofelia Bonesini, Giorgia Callegaro, Antoine Jacquier
Date: 2021-04-09 00:00:00
Tasks: Quantization

We develop a product functional quantization of rough volatility. Since the quantizers can be computed offline, this new technique, built on the insightful works by Luschgy and Pages, becomes a strong competitor in the new arena of numerical tools for rough volatility. We concentrate our numerical analysis to pricing VIX Futures in the rough Bergomi model and compare our results to other recently suggested benchmarks.
Replay in Deep Learning: Current Approaches and Missing Biological Elements
Authors: Tyler L. Hayes, Giri P. Krishnan, Maxim Bazhenov, Hava T. Siegelmann, Terrence J. Sejnowski, Christopher Kanan
Date: 2021-04-01 00:00:00
Tasks: Deep Learning, Retrieval

Replay is the reactivation of one or more neural patterns, which are similar to the activation patterns experienced during past waking experiences. Replay was first observed in biological neural networks during sleep, and it is now thought to play a critical role in memory formation, retrieval, and consolidation. Replay-like mechanisms have been incorporated into deep artificial neural networks that learn over time to avoid catastrophic forgetting of previous knowledge. Replay algorithms have been successfully used in a wide range of deep learning methods within supervised, unsupervised, and reinforcement learning paradigms. In this paper, we provide the first comprehensive comparison between replay in the mammalian brain and replay in artificial neural networks. We identify multiple aspects of biological replay that are missing in deep learning systems and hypothesize how they could be utilized to improve artificial neural networks.
Risks of heterogeneously persistent higher moments
Authors: Jozef Barunik, Josef Kurka
Date: 2021-04-09 00:00:00

Using intraday data for the cross-section of individual stocks, we show that both transitory and persistent fluctuations in realized market and average idiosyncratic volatility, skewness and kurtosis are differentially priced in the cross-section of asset returns, implying a heterogeneous persistence structure of different sources of higher moment risks. Specifically, we find that idiosyncratic transitory shocks to volatility as well as idiosyncratic persistent shocks to skewness contain strong commonalities that are relevant to investors.
POSE-ID-on—A Novel Framework for Artwork Pose Clustering
Authors: Valerio Marsocci, Lorenzo Lastilla
Date: 2021-04-11 00:00:00
Tasks: Clustering, Pose Tracking

In this work, we focus our attention on the similarity among works of art based on human poses and the actions they represent, moving from the concept of Pathosformel in Aby Warburg. This form of similarity is investigated by performing a pose clustering of the human poses, which are modeled as 2D skeletons and are defined as sets of 14 points connected by limbs. To build a dataset of properly annotated artwork images (that is, including the 2D skeletons of the human figures represented), we relied on one of the most popular, recent, and accurate deep learning frameworks for pose tracking of human figures, namely OpenPose. To measure the similarity between human poses, two alternative distance functions are proposed. Moreover, we developed a modified version of the K-Medians algorithm to cluster similar poses and to find a limited number of poses that are representative of the whole dataset. The proposed approach was also compared to two popular clustering strategies, that is, K-Means and the Nearest Point Algorithm, showing higher robustness to outliers. Finally, we assessed the validity of the proposed framework, which we named POSE-ID-on, in both a qualitative and in a quantitative way by simulating a supervised setting, since we lacked a proper reference for comparison.
Low-Cost Stereovision System (Disparity Map) For Few Dollars
Authors: Ildar Rakhmatulin; Eugene Pomazov
Date: 2021-04-12 00:00:00

The paper presents an analysis of the latest developments in the field of stereo vision in the low-cost segment, both for prototypes and for industrial designs. We described the theory of stereo vision and presented information about cameras and data transfer protocols and their compatibility with various devices. The theory in the field of image processing for stereo vision processes is considered and the calibration process is described in detail. Ultimately, we presented the developed stereo vision system and provided the main points that need to be considered when developing such systems. The final, we presented software for adjusting stereo vision parameters in real-time in the python language in the Windows operating system.
CodedStereo: Learned Phase Masks for Large Depth-of-field Stereo
Authors: Shiyu Tan, Yicheng Wu, Shoou-I Yu, Ashok Veeraraghavan
Date: 2021-04-09 00:00:00
Proceeding: CVPR 2021 1
Tasks: Disparity Estimation, Image Reconstruction, Stereo Disparity Estimation

Conventional stereo suffers from a fundamental trade-off between imaging volume and signal-to-noise ratio (SNR) -- due to the conflicting impact of aperture size on both these variables. Inspired by the extended depth of field cameras, we propose a novel end-to-end learning-based technique to overcome this limitation, by introducing a phase mask at the aperture plane of the cameras in a stereo imaging system. The phase mask creates a depth-dependent point spread function, allowing us to recover sharp image texture and stereo correspondence over a significantly extended depth of field (EDOF) than conventional stereo. The phase mask pattern, the EDOF image reconstruction, and the stereo disparity estimation are all trained together using an end-to-end learned deep neural network. We perform theoretical analysis and characterization of the proposed approach and show a 6x increase in volume that can be imaged in simulation. We also build an experimental prototype and validate the approach using real-world results acquired using this prototype system.
Location-Sensitive Visual Recognition with Cross-IOU Loss
Authors: Kaiwen Duan, Lingxi Xie, Honggang Qi, Song Bai, Qingming Huang, Qi Tian
Date: 2021-04-11 00:00:00
Tasks: 2D Human Pose Estimation, Instance Segmentation, Object, object-detection, Object Detection, Pose Estimation, Semantic Segmentation

Object detection, instance segmentation, and pose estimation are popular visual recognition tasks which require localizing the object by internal or boundary landmarks. This paper summarizes these tasks as location-sensitive visual recognition and proposes a unified solution named location-sensitive network (LSNet). Based on a deep neural network as the backbone, LSNet predicts an anchor point and a set of landmarks which together define the shape of the target object. The key to optimizing the LSNet lies in the ability of fitting various scales, for which we design a novel loss function named cross-IOU loss that computes the cross-IOU of each anchor point-landmark pair to approximate the global IOU between the prediction and ground-truth. The flexibly located and accurately predicted landmarks also enable LSNet to incorporate richer contextual information for visual recognition. Evaluated on the MS-COCO dataset, LSNet set the new state-of-the-art accuracy for anchor-free object detection (a 53.5% box AP) and instance segmentation (a 40.2% mask AP), and shows promising performance in detecting multi-scale human poses. Code is available at https://github.com/Duankaiwen/LSNet
Continual Learning for Text Classification with Information Disentanglement Based Regularization
Authors: Yufan Huang, Yanzhe Zhang, Jiaao Chen, Xuezhi Wang, Diyi Yang
Date: 2021-04-12 00:00:00
Proceeding: NAACL 2021 4
Tasks: Continual Learning, Disentanglement, General Classification, Sentence, text-classification, Text Classification

Continual learning has become increasingly important as it enables NLP models to constantly learn and gain knowledge over time. Previous continual learning methods are mainly designed to preserve knowledge from previous tasks, without much emphasis on how to well generalize models to new tasks. In this work, we propose an information disentanglement based regularization method for continual learning on text classification. Our proposed method first disentangles text hidden spaces into representations that are generic to all tasks and representations specific to each individual task, and further regularizes these representations differently to better constrain the knowledge required to generalize. We also introduce two simple auxiliary tasks: next sentence prediction and task-id prediction, for learning better generic and specific representation spaces. Experiments conducted on large-scale benchmarks demonstrate the effectiveness of our method in continual text classification tasks with various sequences and lengths over state-of-the-art baselines. We have publicly released our code at https://github.com/GT-SALT/IDBR.
The World as a Graph: Improving El Niño Forecasts with Graph Neural Networks
Authors: Salva Rühling Cachay, Emma Erickson, Arthur Fender C. Bucker, Ernest Pokropek, Willa Potosnak, Suyash Bire, Salomey Osei, Björn Lütjens
Date: 2021-04-11 00:00:00
Tasks: Deep Learning

Deep learning-based models have recently outperformed state-of-the-art seasonal forecasting models, such as for predicting El Ni\~no-Southern Oscillation (ENSO). However, current deep learning models are based on convolutional neural networks which are difficult to interpret and can fail to model large-scale atmospheric patterns. In comparison, graph neural networks (GNNs) are capable of modeling large-scale spatial dependencies and are more interpretable due to the explicit modeling of information flow through edge connections. We propose the first application of graph neural networks to seasonal forecasting. We design a novel graph connectivity learning module that enables our GNN model to learn large-scale spatial interactions jointly with the actual ENSO forecasting task. Our model, \graphino, outperforms state-of-the-art deep learning-based models for forecasts up to six months ahead. Additionally, we show that our model is more interpretable as it learns sensible connectivity structures that correlate with the ENSO anomaly pattern.
Weak Form Generalized Hamiltonian Learning
Authors: Kevin L. Course, Trefor W. Evans, Prasanth B. Nair
Date: 2021-04-11 00:00:00
Tasks: Form, Time Series, Time Series Analysis

We present a method for learning generalized Hamiltonian decompositions of ordinary differential equations given a set of noisy time series measurements. Our method simultaneously learns a continuous time model and a scalar energy function for a general dynamical system. Learning predictive models in this form allows one to place strong, high-level, physics inspired priors onto the form of the learnt governing equations for general dynamical systems. Moreover, having shown how our method extends and unifies some previous work in deep learning with physics inspired priors, we present a novel method for learning continuous time models from the weak form of the governing equations which is less computationally taxing than standard adjoint methods.
RaidaR: A Rich Annotated Image Dataset of Rainy Street Scenes
Authors: Jiongchao Jin, Arezou Fatemi, Wallace Lira, Fenggen Yu, Biao Leng, Rui Ma, Ali Mahdavi-Amiri, Hao Zhang
Date: 2021-04-09 00:00:00
Tasks: Autonomous Driving, Data Augmentation, Image-to-Image Translation, Rain Removal, Segmentation, Translation

We introduce RaidaR, a rich annotated image dataset of rainy street scenes, to support autonomous driving research. The new dataset contains the largest number of rainy images (58,542) to date, 5,000 of which provide semantic segmentations and 3,658 provide object instance segmentations. The RaidaR images cover a wide range of realistic rain-induced artifacts, including fog, droplets, and road reflections, which can effectively augment existing street scene datasets to improve data-driven machine perception during rainy weather. To facilitate efficient annotation of a large volume of images, we develop a semi-automatic scheme combining manual segmentation and an automated processing akin to cross validation, resulting in 10-20 fold reduction on annotation time. We demonstrate the utility of our new dataset by showing how data augmentation with RaidaR can elevate the accuracy of existing segmentation algorithms. We also present a novel unpaired image-to-image translation algorithm for adding/removing rain artifacts, which directly benefits from RaidaR.
Unified Source-Filter GAN: Unified Source-filter Network Based On Factorization of Quasi-Periodic Parallel WaveGAN
Authors: Reo Yoneyama, Yi-Chiao Wu, Tomoki Toda
Date: 2021-04-10 00:00:00

We propose a unified approach to data-driven source-filter modeling using a single neural network for developing a neural vocoder capable of generating high-quality synthetic speech waveforms while retaining flexibility of the source-filter model to control their voice characteristics. Our proposed network called unified source-filter generative adversarial networks (uSFGAN) is developed by factorizing quasi-periodic parallel WaveGAN (QPPWG), one of the neural vocoders based on a single neural network, into a source excitation generation network and a vocal tract resonance filtering network by additionally implementing a regularization loss. Moreover, inspired by neural source filter (NSF), only a sinusoidal waveform is additionally used as the simplest clue to generate a periodic source excitation waveform while minimizing the effect of approximations in the source filter model. The experimental results demonstrate that uSFGAN outperforms conventional neural vocoders, such as QPPWG and NSF in both speech quality and pitch controllability.
AutoGL: A Library for Automated Graph Learning
Authors: Ziwei Zhang, Yijian Qin, Zeyang Zhang, Chaoyu Guan, Jie Cai, Heng Chang, Jiyan Jiang, Haoyang Li, Zixin Sun, Beini Xie, Yang Yao, YiPeng Zhang, Xin Wang, Wenwu Zhu
Date: 2021-04-11 00:00:00
Proceeding: ICLR Workshop GTRL 2021 5
Tasks: AutoML, BIG-bench Machine Learning, Feature Engineering, Graph Learning, Neural Architecture Search

Recent years have witnessed an upsurge in research interests and applications of machine learning on graphs. However, manually designing the optimal machine learning algorithms for different graph datasets and tasks is inflexible, labor-intensive, and requires expert knowledge, limiting its adaptivity and applicability. Automated machine learning (AutoML) on graphs, aiming to automatically design the optimal machine learning algorithm for a given graph dataset and task, has received considerable attention. However, none of the existing libraries can fully support AutoML on graphs. To fill this gap, we present Automated Graph Learning (AutoGL), the first dedicated library for automated machine learning on graphs. AutoGL is open-source, easy to use, and flexible to be extended. Specifically, we propose a three-layer architecture, consisting of backends to interface with devices, a complete automated graph learning pipeline, and supported graph applications. The automated machine learning pipeline further contains five functional modules: auto feature engineering, neural architecture search, hyper-parameter optimization, model training, and auto ensemble, covering the majority of existing AutoML methods on graphs. For each module, we provide numerous state-of-the-art methods and flexible base classes and APIs, which allow easy usage and customization. We further provide experimental results to showcase the usage of our AutoGL library. We also present AutoGL-light, a lightweight version of AutoGL to facilitate customizing pipelines and enriching applications, as well as benchmarks for graph neural architecture search. The codes of AutoGL are publicly available at https://github.com/THUMNLab/AutoGL.
Sentiment-based Candidate Selection for NMT
Authors: Alex Jones, Derry Tanti Wijaya
Date: 2021-04-10 00:00:00
Tasks: Binary Classification, Decoder, Machine Translation, NMT, Sentence, Sentiment Analysis, Translation

The explosion of user-generated content (UGC)--e.g. social media posts, comments, and reviews--has motivated the development of NLP applications tailored to these types of informal texts. Prevalent among these applications have been sentiment analysis and machine translation (MT). Grounded in the observation that UGC features highly idiomatic, sentiment-charged language, we propose a decoder-side approach that incorporates automatic sentiment scoring into the MT candidate selection process. We train separate English and Spanish sentiment classifiers, then, using n-best candidates generated by a baseline MT model with beam search, select the candidate that minimizes the absolute difference between the sentiment score of the source sentence and that of the translation, and perform a human evaluation to assess the produced translations. Unlike previous work, we select this minimally divergent translation by considering the sentiment scores of the source sentence and translation on a continuous interval, rather than using e.g. binary classification, allowing for more fine-grained selection of translation candidates. The results of human evaluations show that, in comparison to the open-source MT baseline model on top of which our sentiment-based pipeline is built, our pipeline produces more accurate translations of colloquial, sentiment-heavy source texts.
Sparse Coding Frontend for Robust Neural Networks
Authors: Can Bakiskan, Metehan Cekic, Ahmet Dundar Sezer, Upamanyu Madhow
Date: 2021-04-12 00:00:00

Deep Neural Networks are known to be vulnerable to small, adversarially crafted, perturbations. The current most effective defense methods against these adversarial attacks are variants of adversarial training. In this paper, we introduce a radically different defense trained only on clean images: a sparse coding based frontend which significantly attenuates adversarial attacks before they reach the classifier. We evaluate our defense on CIFAR-10 dataset under a wide range of attack types (including Linf , L2, and L1 bounded attacks), demonstrating its promise as a general-purpose approach for defense.
A Bop and Beyond: A Second Order Optimizer for Binarized Neural Networks
Authors: Cuauhtemoc Daniel Suarez-Ramirez, Miguel Gonzalez-Mendoza, Leonardo Chang-Fernandez, Gilberto Ochoa-Ruiz, Mario Alberto Duran-Vega
Date: 2021-04-11 00:00:00

The optimization of Binary Neural Networks (BNNs) relies on approximating the real-valued weights with their binarized representations. Current techniques for weight-updating use the same approaches as traditional Neural Networks (NNs) with the extra requirement of using an approximation to the derivative of the sign function - as it is the Dirac-Delta function - for back-propagation; thus, efforts are focused adapting full-precision techniques to work on BNNs. In the literature, only one previous effort has tackled the problem of directly training the BNNs with bit-flips by using the first raw moment estimate of the gradients and comparing it against a threshold for deciding when to flip a weight (Bop). In this paper, we take an approach parallel to Adam which also uses the second raw moment estimate to normalize the first raw moment before doing the comparison with the threshold, we call this method Bop2ndOrder. We present two versions of the proposed optimizer: a biased one and a bias-corrected one, each with its own applications. Also, we present a complete ablation study of the hyperparameters space, as well as the effect of using schedulers on each of them. For these studies, we tested the optimizer in CIFAR10 using the BinaryNet architecture. Also, we tested it in ImageNet 2012 with the XnorNet and BiRealNet architectures for accuracy. In both datasets our approach proved to converge faster, was robust to changes of the hyperparameters, and achieved better accuracy values.