pwc-archive/papers-with-abstracts huggingface dataset:
Embodied negation and levels of concreteness: A TMS Study on German and Italian language processing
Authors: Giorgio Papitto, Luisa Lugli, Anna M. Borghi, Antonello Pellicano, Ferdinand Binkofski
Date: 2021-06-08 00:00:00
Tasks: Negation, Sentence

According to the embodied cognition perspective, linguistic negation may block the motor simulations induced by language processing. Transcranial magnetic stimulation (TMS) was applied to the left primary motor cortex (hand area) of monolingual Italian and German healthy participants during a rapid serial visual presentation of sentences from their own language. In these languages, the negative particle is located at the beginning and at the end of the sentence, respectively. The study investigated whether the interruption of the motor simulation processes, accounted for by reduced motor evoked potentials (MEPs), takes place similarly in two languages differing on the position of the negative marker. Different levels of sentence concreteness were also manipulated to investigate if negation exerts generalized effects or if it is affected by the semantic features of the sentence. Our findings indicate that negation acts as a block on motor representations, but independently from the language and words concreteness level.
Impact of the activation rate of the hyperpolarization-activated current $I_{\rm h}$ on the neuronal membrane time constant and synaptic potential duration
Authors: Cesar C. Ceballos, Rodrigo F. O. Pena, Antonio C. Roque
Date: 2021-06-07 00:00:00

The temporal dynamics of membrane voltage changes in neurons is controlled by ionic currents. These currents are characterized by two main properties: conductance and kinetics. The hyperpolarization-activated current ($I_{\rm h}$) strongly modulates subthreshold potential changes by shortening the excitatory postsynaptic potentials and decreasing their temporal summation. Whereas the shortening of the synaptic potentials caused by the $I_{\rm h}$ conductance is well understood, the role of the $I_{\rm h}$ kinetics remains unclear. Here, we use a model of the $I_{\rm h}$ current model with either fast or slow kinetics to determine its influence on the membrane time constant ($\tau_m$) of a CA1 pyramidal cell model. Our simulation results show that the $I_{\rm h}$ with fast kinetics decreases $\tau_m$ and attenuates and shortens the excitatory postsynaptic potentials more than the slow $I_{\rm h}$. We conclude that the $I_{\rm h}$ activation kinetics is able to modulate $\tau_m$ and the temporal properties of excitatory postsynaptic potentials (EPSPs) in CA1 pyramidal cells. In order to elucidate the mechanisms by which $I_{\rm h}$ kinetics controls $\tau_m$, we propose a new concept called "time scaling factor". Our main finding is that the $I_{\rm h}$ kinetics influences $\tau_m$ by modulating the contribution of the $I_{\rm h}$ derivative conductance to $\tau_m$.
Assessing the impact of social activity permissiveness on the COVID-19 infection curve of several countries
Authors: Gerardo L. Febres
Date: 2021-06-08 00:00:00
Tasks: Math

This document aims to estimate and describe the effects of the social distancing measures implemented in several countries with the expectancy of controlling the spread of COVID-19. The procedure relies on the classic Susceptible-Infected-Removed (SIR) model, which is modified to incorporate a permissiveness index, representing the isolation achieved by the social distancing and the future development of vaccination campaigns and allowing the math model to reproduce more than one infection wave. The adjusted SIR models are used to study the compromise between the economy's reactivation and the resulting infection spreading increase. The document presents a graphical-abacus that describes the convenience of progressively relax social distancing measures while a feasible vaccination campaign develops
How does economic policy uncertainty comove with stock markets: New evidence from symmetric thermal optimal path method
Authors: Ying-Hui Shao, Yan-Hong Yang, Wei-Xing Zhou
Date: 2021-06-08 00:00:00

We revisit the dynamic relationship between domestic economic policy uncertainty and stock markets using the symmetric thermal optimal path (TOPS) method. We observe different interaction patterns in emerging and developed markets. Economic policy uncertainty drives the stock market in China, while stock markets play a leading role in the UK and the US. Meanwhile, the lead-lag relationship of the three countries reacts significantly to extreme events. Our findings have important implications for investors and policy makers.
Options on Bonds: Implied Volatilities from Affine Short-Rate Dynamics
Authors: Matthew Lorig, Natchanon Suaysom
Date: 2021-06-08 00:00:00

We derive an explicit asymptotic approximation for the implied volatilities of Call options written on bonds assuming the short-rate is described by an affine short-rate model. For specific affine short-rate models, we perform numerical experiments in order to gauge the accuracy of our approximation.
Fair Feature Distillation for Visual Recognition
Authors: Sangwon Jung, DongGyu Lee, TaeEon Park, Taesup Moon
Date: 2021-05-27 00:00:00
Proceeding: CVPR 2021 1
Tasks: Fairness, Knowledge Distillation

Fairness is becoming an increasingly crucial issue for computer vision, especially in the human-related decision systems. However, achieving algorithmic fairness, which makes a model produce indiscriminative outcomes against protected groups, is still an unresolved problem. In this paper, we devise a systematic approach which reduces algorithmic biases via feature distillation for visual recognition tasks, dubbed as MMD-based Fair Distillation (MFD). While the distillation technique has been widely used in general to improve the prediction accuracy, to the best of our knowledge, there has been no explicit work that also tries to improve fairness via distillation. Furthermore, We give a theoretical justification of our MFD on the effect of knowledge distillation and fairness. Throughout the extensive experiments, we show our MFD significantly mitigates the bias against specific minorities without any loss of the accuracy on both synthetic and real-world face datasets.
Stochastic Whitening Batch Normalization
Authors: Shengdong Zhang, Ehsan Nezhadarya, Homa Fashandi, Jiayi Liu, Darin Graham, Mohak Shah
Date: 2021-06-03 00:00:00
Proceeding: CVPR 2021 1
Tasks: image-classification, Image Classification

Batch Normalization (BN) is a popular technique for training Deep Neural Networks (DNNs). BN uses scaling and shifting to normalize activations of mini-batches to accelerate convergence and improve generalization. The recently proposed Iterative Normalization (IterNorm) method improves these properties by whitening the activations iteratively using Newton's method. However, since Newton's method initializes the whitening matrix independently at each training step, no information is shared between consecutive steps. In this work, instead of exact computation of whitening matrix at each time step, we estimate it gradually during training in an online fashion, using our proposed Stochastic Whitening Batch Normalization (SWBN) algorithm. We show that while SWBN improves the convergence rate and generalization of DNNs, its computational overhead is less than that of IterNorm. Due to the high efficiency of the proposed method, it can be easily employed in most DNN architectures with a large number of layers. We provide comprehensive experiments and comparisons between BN, IterNorm, and SWBN layers to demonstrate the effectiveness of the proposed technique in conventional (many-shot) image classification and few-shot classification tasks.
Asymmetrical Bi-RNN for pedestrian trajectory encoding
Authors: Raphaël Rozenberg, Joseph Gesnouin, Fabien Moutarde
Date: 2021-06-01 00:00:00
Tasks: Autonomous Driving, Autonomous Navigation, Pedestrian Trajectory Prediction, Trajectory Forecasting, Trajectory Prediction

Pedestrian motion behavior involves a combination of individual goals and social interactions with other agents. In this article, we present an asymmetrical bidirectional recurrent neural network architecture called U-RNN to encode pedestrian trajectories and evaluate its relevance to replace LSTMs for various forecasting models. Experimental results on the Trajnet++ benchmark show that the U-LSTM variant yields better results regarding every available metrics (ADE, FDE, Collision rate) than common trajectory encoders for a variety of approaches and interaction modules, suggesting that the proposed approach is a viable alternative to the de facto sequence encoding RNNs. Our implementation of the asymmetrical Bi-RNNs for the Trajnet++ benchmark is available at: github.com/JosephGesnouin/Asymmetrical-Bi-RNNs-to-encode-pedestrian-trajectories
DisTop: Discovering a Topological representation to learn diverse and rewarding skills
Authors: Arthur Aubret, Laetitia Matignon, Salima Hassas
Date: 2021-06-06 00:00:00
Tasks: Deep Reinforcement Learning, Hierarchical Reinforcement Learning, MuJoCo, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Representation Learning

The optimal way for a deep reinforcement learning (DRL) agent to explore is to learn a set of skills that achieves a uniform distribution of states. Following this,we introduce DisTop, a new model that simultaneously learns diverse skills and focuses on improving rewarding skills. DisTop progressively builds a discrete topology of the environment using an unsupervised contrastive loss, a growing network and a goal-conditioned policy. Using this topology, a state-independent hierarchical policy can select where the agent has to keep discovering skills in the state space. In turn, the newly visited states allows an improved learnt representation and the learning loop continues. Our experiments emphasize that DisTop is agnostic to the ground state representation and that the agent can discover the topology of its environment whether the states are high-dimensional binary data, images, or proprioceptive inputs. We demonstrate that this paradigm is competitiveon MuJoCo benchmarks with state-of-the-art algorithms on both single-task dense rewards and diverse skill discovery. By combining these two aspects, we showthat DisTop achieves state-of-the-art performance in comparison with hierarchical reinforcement learning (HRL) when rewards are sparse. We believe DisTop opens new perspectives by showing that bottom-up skill discovery combined with representation learning can unlock the exploration challenge in DRL.
Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts
Authors: Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, Ed Chi
Date: 2018-07-19 00:00:00
Tasks: Binary Classification, Click-Through Rate Prediction, Mixture-of-Experts, Multi-Task Learning, Recommendation Systems

Neural-based multi-task learning has been successfully used in many real-world large-scale applications such as recommendation systems. For example, in movie recommendations, beyond providing users movies which they tend to purchase and watch, the system might also optimize for users liking the movies afterwards. With multi-task learning, we aim to build a single model that learns these multiple goals and tasks simultaneously. However, the prediction quality of commonly used multi task models is often sensitive to the relationships between tasks. It is therefore important to study the modeling tradeoffs between task-specific objectives and inter-task relationships. In this work, we propose a novel multi-task learning approach, Multi-gate Mixture-of-Experts (MMoE), which explicitly learns to model task relationships from data. We adapt the Mixture-of-Experts (MoE) structure to multi-task learning by sharing the expert submodels across all tasks, while also having a gating network trained to optimize each task. To validate our approach on data with different levels of task relatedness, we first apply it to a synthetic dataset where we control the task relatedness. We show that the proposed approach performs better than baseline methods when the tasks are less related. We also show that the MMoE structure results in an additional trainability benefit, depending on different levels of randomness in the training data and model initialization. Furthermore, we demonstrate the performance improvements by MMoE on real tasks including a binary classification benchmark, and a large-scale content recommendation system at Google.
Personalized PercepNet: Real-time, Low-complexity Target Voice Separation and Enhancement
Authors: Ritwik Giri, Shrikant Venkataramani, Jean-Marc Valin, Umut Isik, Arvindh Krishnaswamy
Date: 2021-06-08 00:00:00

The presence of multiple talkers in the surrounding environment poses a difficult challenge for real-time speech communication systems considering the constraints on network size and complexity. In this paper, we present Personalized PercepNet, a real-time speech enhancement model that separates a target speaker from a noisy multi-talker mixture without compromising on complexity of the recently proposed PercepNet. To enable speaker-dependent speech enhancement, we first show how we can train a perceptually motivated speaker embedder network to produce a representative embedding vector for the given speaker. Personalized PercepNet uses the target speaker embedding as additional information to pick out and enhance only the target speaker while suppressing all other competing sounds. Our experiments show that the proposed model significantly outperforms PercepNet and other baselines, both in terms of objective speech enhancement metrics and human opinion scores.
TesseTrack: End-to-End Learnable Multi-Person Articulated 3D Pose Tracking
Authors: N. Dinesh Reddy, Laurent Guigues, Leonid Pischulini, Jayan Eledath, Srinivasa Narasimhan
Date: 2021-06-16 00:00:00
Proceeding: CVPR 2021 1
Tasks: 2D Pose Estimation, 3D Human Pose Estimation, 3D Human Pose Tracking, 3D Multi-Object Tracking, 3D Multi-Person Pose Estimation, 3D Pose Estimation, Human Detection, Pose Estimation, Pose Tracking

We consider the task of 3D pose estimation and tracking of multiple people seen in an arbitrary number of camera feeds. We propose TesseTrack, a novel top-down approach that simultaneously reasons about multiple individuals’ 3D body joint reconstructions and associations in space and time in a single end-to-end learnable framework. At the core of our approach is a novel spatio-temporal formulation that operates in a common voxelized feature space aggregated from single- or multiple camera views. After a person detection step, a 4D CNN produces short-term person-specific representations which are then linked across time by a differentiable matcher. The linked descriptions are then merged and deconvolved into 3D poses. This joint spatio-temporal formulation contrasts with previous piece-wise strategies that treat 2D pose estimation, 2D-to-3D lifting, and 3D pose tracking as independent sub-problems that are error-prone when solved in isolation. Furthermore, unlike previous methods, TesseTrack is robust to changes in the number of camera views and achieves very good results even if a single view is available at inference time. Quantitative evaluation of 3D pose reconstruction accuracy on standard benchmarks shows significant improvements over the state of the art. Evaluation of multi-person articulated 3D pose tracking in our novel evaluation framework demonstrates the superiority of TesseTrack over strong baselines.
SOS: Stereo Matching in O(1) with Slanted Support Windows
Date: 2018-01-01 00:00:00
Tasks: GPU, Stereo Depth Estimation, Stereo Disparity Estimation, Stereo Matching

Depth cameras have accelerated research in many areas of computer vision. Most triangulation-based depth cameras, whether structured light systems like the Kinect or active (assisted) stereo systems, are based on the principle of stereo matching. Depth from stereo is an active research topic dating back 30 years. Despite recent advances, algorithms usually trade-off accuracy for speed. In particular, efficient methods rely on fronto-parallel assumptions to reduce the search space and keep computation low. We present SOS (Slanted O(1) Stereo), the first algorithm capable of leveraging slanted support windows without sacrificing speed or accuracy. We use an active stereo configuration, where an illuminator textures the scene. Under this setting, local methods - such as PatchMatch Stereo - obtain state of the art results by jointly estimating disparities and slant, but at a large computational cost. We observe that these methods typically exploit local smoothness to simplify their initialization strategies. Our key insight is that local smoothness can in fact be used to amortize the computation not only within initialization, but across the entire stereo pipeline. Building on these insights, we propose a novel hierarchical initialization that is able to efficiently perform search over disparity and slants. We then show how this structure can be leveraged to provide high quality depth maps. Extensive quantitative evaluations demonstrate that the proposed technique yields significantly more precise results than current state of the art, but at a fraction of the computational cost. Our prototype implementation runs at 4000 fps on modern GPU architectures.
ST++: Make Self-training Work Better for Semi-supervised Semantic Segmentation
Authors: Lihe Yang, Wei Zhuo, Lei Qi, Yinghuan Shi, Yang Gao
Date: 2021-06-09 00:00:00
Proceeding: CVPR 2022 1
Tasks: Semantic Segmentation, Semi-Supervised Semantic Segmentation

Self-training via pseudo labeling is a conventional, simple, and popular pipeline to leverage unlabeled data. In this work, we first construct a strong baseline of self-training (namely ST) for semi-supervised semantic segmentation via injecting strong data augmentations (SDA) on unlabeled images to alleviate overfitting noisy labels as well as decouple similar predictions between the teacher and student. With this simple mechanism, our ST outperforms all existing methods without any bells and whistles, e.g., iterative re-training. Inspired by the impressive results, we thoroughly investigate the SDA and provide some empirical analysis. Nevertheless, incorrect pseudo labels are still prone to accumulate and degrade the performance. To this end, we further propose an advanced self-training framework (namely ST++), that performs selective re-training via prioritizing reliable unlabeled images based on holistic prediction-level stability. Concretely, several model checkpoints are saved in the first stage supervised training, and the discrepancy of their predictions on the unlabeled image serves as a measurement for reliability. Our image-level selection offers holistic contextual information for learning. We demonstrate that it is more suitable for segmentation than common pixel-wise selection. As a result, ST++ further boosts the performance of our ST. Code is available at https://github.com/LiheYoung/ST-PlusPlus.
Scaling Up Graph Neural Networks Via Graph Coarsening
Authors: Zengfeng Huang, Shengzhong Zhang, Chong Xi, Tang Liu, Min Zhou
Date: 2021-06-09 00:00:00
Tasks: Stochastic Optimization

Scalability of graph neural networks remains one of the major challenges in graph machine learning. Since the representation of a node is computed by recursively aggregating and transforming representation vectors of its neighboring nodes from previous layers, the receptive fields grow exponentially, which makes standard stochastic optimization techniques ineffective. Various approaches have been proposed to alleviate this issue, e.g., sampling-based methods and techniques based on pre-computation of graph filters. In this paper, we take a different approach and propose to use graph coarsening for scalable training of GNNs, which is generic, extremely simple and has sublinear memory and time costs during training. We present extensive theoretical analysis on the effect of using coarsening operations and provides useful guidance on the choice of coarsening methods. Interestingly, our theoretical analysis shows that coarsening can also be considered as a type of regularization and may improve the generalization. Finally, empirical results on real world datasets show that, simply applying off-the-shelf coarsening methods, we can reduce the number of nodes by up to a factor of ten without causing a noticeable downgrade in classification accuracy.
FastSeq: Make Sequence Generation Faster
Authors: Yu Yan, Fei Hu, Jiusheng Chen, Nikhil Bhendawade, Ting Ye, Yeyun Gong, Nan Duan, Desheng Cui, Bingyu Chi, Ruofei Zhang
Date: 2021-06-08 00:00:00
Proceeding: ACL 2021 5
Tasks: Text Generation

Transformer-based models have made tremendous impacts in natural language generation. However the inference speed is a bottleneck due to large model size and intensive computing involved in auto-regressive decoding process. We develop FastSeq framework to accelerate sequence generation without accuracy loss. The proposed optimization techniques include an attention cache optimization, an efficient algorithm for detecting repeated n-grams, and an asynchronous generation pipeline with parallel I/O. These optimizations are general enough to be applicable to Transformer-based models (e.g., T5, GPT2, and UniLM). Our benchmark results on a set of widely used and diverse models demonstrate 4-9x inference speed gain. Additionally, FastSeq is easy to use with a simple one-line code change. The source code is available at https://github.com/microsoft/fastseq.
Towards Scalable Uncertainty Aware DNN-based Wireless Localisation
Authors: Artan Salihu, Stefan Schwarz, Markus Rupp
Date: 2021-06-08 00:00:00

Existing deep neural network (DNN) based wireless localization approaches typically do not capture uncertainty inherent in their estimates. In this work, we propose and evaluate variational and scalable DNN approaches to measure the uncertainty as a result of changing propagation conditions and the finite number of training samples. Furthermore, we show that data uncertainty is sufficient to capture the uncertainty due to non-line-of-sight (NLOS) and, model uncertainty improves the overall reliability. To assess the robustness due to channel conditions and out-of-set regions, we evaluate the methods on challenging massive multiple-input multiple-output (MIMO) scenarios.
ParChain: A Framework for Parallel Hierarchical Agglomerative Clustering using Nearest-Neighbor Chain
Authors: Shangdi Yu, Yiqiu Wang, Yan Gu, Laxman Dhulipala, Julian Shun
Date: 2021-06-08 00:00:00
Tasks: Clustering

This paper studies the hierarchical clustering problem, where the goal is to produce a dendrogram that represents clusters at varying scales of a data set. We propose the ParChain framework for designing parallel hierarchical agglomerative clustering (HAC) algorithms, and using the framework we obtain novel parallel algorithms for the complete linkage, average linkage, and Ward's linkage criteria. Compared to most previous parallel HAC algorithms, which require quadratic memory, our new algorithms require only linear memory, and are scalable to large data sets. ParChain is based on our parallelization of the nearest-neighbor chain algorithm, and enables multiple clusters to be merged on every round. We introduce two key optimizations that are critical for efficiency: a range query optimization that reduces the number of distance computations required when finding nearest neighbors of clusters, and a caching optimization that stores a subset of previously computed distances, which are likely to be reused. Experimentally, we show that our highly-optimized implementations using 48 cores with two-way hyper-threading achieve 5.8--110.1x speedup over state-of-the-art parallel HAC algorithms and achieve 13.75--54.23x self-relative speedup. Compared to state-of-the-art algorithms, our algorithms require up to 237.3x less space. Our algorithms are able to scale to data set sizes with tens of millions of points, which existing algorithms are not able to handle.