pwc-archive/papers-with-abstracts huggingface dataset:
Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video
Authors: Edgar Tretschk, Ayush Tewari, Vladislav Golyanik, Michael Zollhöfer, Christoph Lassner, Christian Theobalt
Date: 2020-12-22 00:00:00
Proceeding: ICCV 2021 10
Tasks: NeRF, Novel View Synthesis, Video Editing

We present Non-Rigid Neural Radiance Fields (NR-NeRF), a reconstruction and novel view synthesis approach for general non-rigid dynamic scenes. Our approach takes RGB images of a dynamic scene as input (e.g., from a monocular video recording), and creates a high-quality space-time geometry and appearance representation. We show that a single handheld consumer-grade camera is sufficient to synthesize sophisticated renderings of a dynamic scene from novel virtual camera views, e.g. a `bullet-time' video effect. NR-NeRF disentangles the dynamic scene into a canonical volume and its deformation. Scene deformation is implemented as ray bending, where straight rays are deformed non-rigidly. We also propose a novel rigidity network to better constrain rigid regions of the scene, leading to more stable results. The ray bending and rigidity network are trained without explicit supervision. Our formulation enables dense correspondence estimation across views and time, and compelling video editing applications such as motion exaggeration. Our code will be open sourced.
Unadversarial Examples: Designing Objects for Robust Vision
Authors: Hadi Salman, Andrew Ilyas, Logan Engstrom, Sai Vemprala, Aleksander Madry, Ashish Kapoor
Date: 2020-12-22 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: BIG-bench Machine Learning, Sensitivity

We study a class of realistic computer vision settings wherein one can influence the design of the objects being recognized. We develop a framework that leverages this capability to significantly improve vision models' performance and robustness. This framework exploits the sensitivity of modern machine learning algorithms to input perturbations in order to design "robust objects," i.e., objects that are explicitly optimized to be confidently detected or classified. We demonstrate the efficacy of the framework on a wide variety of vision-based tasks ranging from standard benchmarks, to (in-simulation) robotics, to real-world experiments. Our code can be found at https://git.io/unadversarial .
Training Convolutional Neural Networks With Hebbian Principal Component Analysis
Authors: Gabriele Lagani, Giuseppe Amato, Fabrizio Falchi, Claudio Gennaro
Date: 2020-12-22 00:00:00
Tasks: Transfer Learning

Recent work has shown that biologically plausible Hebbian learning can be integrated with backpropagation learning (backprop), when training deep convolutional neural networks. In particular, it has been shown that Hebbian learning can be used for training the lower or the higher layers of a neural network. For instance, Hebbian learning is effective for re-training the higher layers of a pre-trained deep neural network, achieving comparable accuracy w.r.t. SGD, while requiring fewer training epochs, suggesting potential applications for transfer learning. In this paper we build on these results and we further improve Hebbian learning in these settings, by using a nonlinear Hebbian Principal Component Analysis (HPCA) learning rule, in place of the Hebbian Winner Takes All (HWTA) strategy used in previous work. We test this approach in the context of computer vision. In particular, the HPCA rule is used to train Convolutional Neural Networks in order to extract relevant features from the CIFAR-10 image dataset. The HPCA variant that we explore further improves the previous results, motivating further interest towards biologically plausible learning algorithms.
BKT-LSTM: Efficient Student Modeling for knowledge tracing and student performance prediction
Authors: Sein Minn
Date: 2020-12-22 00:00:00
Tasks: Knowledge Tracing, Prediction, Skill Mastery

Recently, we have seen a rapid rise in usage of online educational platforms. The personalized education became crucially important in future learning environments. Knowledge tracing (KT) refers to the detection of students' knowledge states and predict future performance given their past outcomes for providing adaptive solution to Intelligent Tutoring Systems (ITS). Bayesian Knowledge Tracing (BKT) is a model to capture mastery level of each skill with psychologically meaningful parameters and widely used in successful tutoring systems. However, it is unable to detect learning transfer across skills because each skill model is learned independently and shows lower efficiency in student performance prediction. While recent KT models based on deep neural networks shows impressive predictive power but it came with a price. Ten of thousands of parameters in neural networks are unable to provide psychologically meaningful interpretation that reflect to cognitive theory. In this paper, we proposed an efficient student model called BKT-LSTM. It contains three meaningful components: individual \textit{skill mastery} assessed by BKT, \textit{ability profile} (learning transfer across skills) detected by k-means clustering and \textit{problem difficulty}. All these components are taken into account in student's future performance prediction by leveraging predictive power of LSTM. BKT-LSTM outperforms state-of-the-art student models in student's performance prediction by considering these meaningful features instead of using binary values of student's past interaction in DKT. We also conduct ablation studies on each of BKT-LSTM model components to examine their value and each component shows significant contribution in student's performance prediction. Thus, it has potential for providing adaptive and personalized instruction in real-world educational systems.
Robust Kernel-based Feature Representation for 3D Point Cloud Analysis via Circular Convolutional Network
Authors: Seunghwan Jung, Yeong-Gil Shin, Minyoung Chung
Date: 2020-12-22 00:00:00
Tasks: Segmentation

Feature descriptors of point clouds are used in several applications, such as registration and part segmentation of 3D point clouds. Learning discriminative representations of local geometric features is unquestionably the most important task for accurate point cloud analyses. However, it is challenging to develop rotation or scale-invariant descriptors. Most previous studies have either ignored rotations or empirically studied optimal scale parameters, which hinders the applicability of the methods for real-world datasets. In this paper, we present a new local feature description method that is robust to rotation, density, and scale variations. Moreover, to improve representations of the local descriptors, we propose a global aggregation method. First, we place kernels aligned around each point in the normal direction. To avoid the sign problem of the normal vector, we use a symmetric kernel point distribution in the tangential plane. From each kernel point, we first projected the points from the spatial space to the feature space, which is robust to multiple scales and rotation, based on angles and distances. Subsequently, we perform graph convolutions by considering local kernel point structures and long-range global context, obtained by a global aggregation method. We experimented with our proposed descriptors on benchmark datasets (i.e., ModelNet40 and ShapeNetPart) to evaluate the performance of registration, classification, and part segmentation on 3D point clouds. Our method showed superior performances when compared to the state-of-the-art methods by reducing 70$\%$ of the rotation and translation errors in the registration task. Our method also showed comparable performance in the classification and part-segmentation tasks with simple and low-dimensional architectures.
Emergent Hand Morphology and Control from Optimizing Robust Grasps of Diverse Objects
Authors: Xinlei Pan, Animesh Garg, Animashree Anandkumar, Yuke Zhu
Date: 2020-12-22 00:00:00
Tasks: Bayesian Optimization, MORPH

Evolution in nature illustrates that the creatures' biological structure and their sensorimotor skills adapt to the environmental changes for survival. Likewise, the ability to morph and acquire new skills can facilitate an embodied agent to solve tasks of varying complexities. In this work, we introduce a data-driven approach where effective hand designs naturally emerge for the purpose of grasping diverse objects. Jointly optimizing morphology and control imposes computational challenges since it requires constant evaluation of a black-box function that measures the performance of a combination of embodiment and behavior. We develop a novel Bayesian Optimization algorithm that efficiently co-designs the morphology and grasping skills jointly through learned latent-space representations. We design the grasping tasks based on a taxonomy of three human grasp types: power grasp, pinch grasp, and lateral grasp. Through experimentation and comparative study, we demonstrate the effectiveness of our approach in discovering robust and cost-efficient hand morphologies for grasping novel objects.
FracBNN: Accurate and FPGA-Efficient Binary Neural Networks with Fractional Activations
Authors: Yichi Zhang, Junhao Pan, Xinheng Liu, Hongzheng Chen, Deming Chen, Zhiru Zhang
Date: 2020-12-22 00:00:00
Tasks: Binarization, image-classification, Image Classification

Binary neural networks (BNNs) have 1-bit weights and activations. Such networks are well suited for FPGAs, as their dominant computations are bitwise arithmetic and the memory requirement is also significantly reduced. However, compared to start-of-the-art compact convolutional neural network (CNN) models, BNNs tend to produce a much lower accuracy on realistic datasets such as ImageNet. In addition, the input layer of BNNs has gradually become a major compute bottleneck, because it is conventionally excluded from binarization to avoid a large accuracy loss. This work proposes FracBNN, which exploits fractional activations to substantially improve the accuracy of BNNs. Specifically, our approach employs a dual-precision activation scheme to compute features with up to two bits, using an additional sparse binary convolution. We further binarize the input layer using a novel thermometer encoding. Overall, FracBNN preserves the key benefits of conventional BNNs, where all convolutional layers are computed in pure binary MAC operations (BMACs). We design an efficient FPGA-based accelerator for our novel BNN model that supports the fractional activations. To evaluate the performance of FracBNN under a resource-constrained scenario, we implement the entire optimized network architecture on an embedded FPGA (Xilinx Ultra96v2). Our experiments on ImageNet show that FracBNN achieves an accuracy comparable to MobileNetV2, surpassing the best-known BNN design on FPGAs with an increase of 28.9% in top-1 accuracy and a 2.5x reduction in model size. FracBNN also outperforms a recently introduced BNN model with an increase of 2.4% in top-1 accuracy while using the same model size. On the embedded FPGA device, FracBNN demonstrates the ability of real-time image classification.
Autonomous sPOMDP Environment Modeling With Partial Model Exploitation
Authors: Andrew Wilhelm, Aaron Wilhelm, Garrett Fosdick
Date: 2020-12-22 00:00:00
Tasks: model

A state space representation of an environment is a classic and yet powerful tool used by many autonomous robotic systems for efficient and often optimal solution planning. However, designing these representations with high performance is laborious and costly, necessitating an effective and versatile tool for autonomous generation of state spaces for autonomous robots. We present a novel state space exploration algorithm by extending the original surprise-based partially-observable Markov Decision Processes (sPOMDP), and demonstrate its effective long-term exploration planning performance in various environments. Through extensive simulation experiments, we show the proposed model significantly increases efficiency and scalability of the original sPOMDP learning techniques with a range of 31-63% gain in training speed while improving robustness in environments with less deterministic transitions. Our results pave the way for extending sPOMDP solutions to a broader set of environments.
Virtual Source Synthetic Aperture for Accurate Lateral Displacement Estimation in Ultrasound Elastography
Authors: Morteza Mirzaei, Amir Asif, Hassan Rivaz
Date: 2020-12-19 00:00:00

Ultrasound elastography is an emerging noninvasive imaging technique wherein pathological alterations can be visualized by revealing the mechanical properties of the tissue. Estimating tissue displacement in all directions is required to accurately estimate the mechanical properties. Despite capabilities of elastography techniques in estimating displacement in both axial and lateral directions, estimation of axial displacement is more accurate than lateral direction due to higher sampling frequency, higher resolution and having a carrier signal propagating in the axial direction. Among different ultrasound imaging techniques, Synthetic Aperture (SA) has better lateral resolution than others, but it is not commonly used for ultrasound elastography due to its limitation in imaging depth of field. Virtual source synthetic aperture (VSSA) imaging is a technique to implement synthetic aperture beamforming on the focused transmitted data to overcome limitation of SA in depth of field while maintaining the same lateral resolution as SA. Besides lateral resolution, VSSA has the capability of increasing sampling frequency in the lateral direction without interpolation. In this paper, we utilize VSSA to perform beamforming to enable higher resolution and sampling frequency in the lateral direction. The beamformed data is then processed using our recently published elastography technique, OVERWIND [1]. Simulation and experimental results show substantial improvement in estimation of lateral displacements.
Learning to Play Imperfect-Information Games by Imitating an Oracle Planner
Authors: Rinu Boney, Alexander Ilin, Juho Kannala, Jarno Seppänen
Date: 2020-12-22 00:00:00
Tasks: Thompson Sampling

We consider learning to play multiplayer imperfect-information games with simultaneous moves and large state-action spaces. Previous attempts to tackle such challenging games have largely focused on model-free learning methods, often requiring hundreds of years of experience to produce competitive agents. Our approach is based on model-based planning. We tackle the problem of partial observability by first building an (oracle) planner that has access to the full state of the environment and then distilling the knowledge of the oracle to a (follower) agent which is trained to play the imperfect-information game by imitating the oracle's choices. We experimentally show that planning with naive Monte Carlo tree search does not perform very well in large combinatorial action spaces. We therefore propose planning with a fixed-depth tree search and decoupled Thompson sampling for action selection. We show that the planner is able to discover efficient playing strategies in the games of Clash Royale and Pommerman and the follower policy successfully learns to implement them by training on a few hundred battles.
Cloud removal in remote sensing images using generative adversarial networks and SAR-to-optical image translation
Authors: Faramarz Naderi Darbaghshahi, Mohammad Reza Mohammadi, Mohsen Soryani
Date: 2020-12-22 00:00:00
Tasks: Cloud Removal, SSIM, Translation

Satellite images are often contaminated by clouds. Cloud removal has received much attention due to the wide range of satellite image applications. As the clouds thicken, the process of removing the clouds becomes more challenging. In such cases, using auxiliary images such as near-infrared or synthetic aperture radar (SAR) for reconstructing is common. In this study, we attempt to solve the problem using two generative adversarial networks (GANs). The first translates SAR images into optical images, and the second removes clouds using the translated images of prior GAN. Also, we propose dilated residual inception blocks (DRIBs) instead of vanilla U-net in the generator networks and use structural similarity index measure (SSIM) in addition to the L1 Loss function. Reducing the number of downsamplings and expanding receptive fields by dilated convolutions increase the quality of output images. We used the SEN1-2 dataset to train and test both GANs, and we made cloudy images by adding synthetic clouds to optical images. The restored images are evaluated with PSNR and SSIM. We compare the proposed method with state-of-the-art deep learning models and achieve more accurate results in both SAR-to-optical translation and cloud removal parts.
Quantum Convolutional Neural Networks for High Energy Physics Data Analysis
Authors: Samuel Yen-Chi Chen, Tzu-Chieh Wei, Chao Zhang, Haiwang Yu, Shinjae Yoo
Date: 2020-12-22 00:00:00
Tasks: BIG-bench Machine Learning, Quantum Machine Learning, Vocal Bursts Intensity Prediction

This work presents a quantum convolutional neural network (QCNN) for the classification of high energy physics events. The proposed model is tested using a simulated dataset from the Deep Underground Neutrino Experiment. The proposed architecture demonstrates the quantum advantage of learning faster than the classical convolutional neural networks (CNNs) under a similar number of parameters. In addition to faster convergence, the QCNN achieves greater test accuracy compared to CNNs. Based on experimental results, it is a promising direction to study the application of QCNN and other quantum machine learning models in high energy physics and additional scientific fields.
Latent Feature Representation via Unsupervised Learning for Pattern Discovery in Massive Electron Microscopy Image Volumes
Authors: Gary B Huang, Huei-Fang Yang, Shin-ya Takemura, Pat Rivlin, Stephen M Plaza
Date: 2020-12-22 00:00:00
Tasks: Semantic Similarity, Semantic Textual Similarity

We propose a method to facilitate exploration and analysis of new large data sets. In particular, we give an unsupervised deep learning approach to learning a latent representation that captures semantic similarity in the data set. The core idea is to use data augmentations that preserve semantic meaning to generate synthetic examples of elements whose feature representations should be close to one another. We demonstrate the utility of our method applied to nano-scale electron microscopy data, where even relatively small portions of animal brains can require terabytes of image data. Although supervised methods can be used to predict and identify known patterns of interest, the scale of the data makes it difficult to mine and analyze patterns that are not known a priori. We show the ability of our learned representation to enable query by example, so that if a scientist notices an interesting pattern in the data, they can be presented with other locations with matching patterns. We also demonstrate that clustering of data in the learned space correlates with biologically-meaningful distinctions. Finally, we introduce a visualization tool and software ecosystem to facilitate user-friendly interactive analysis and uncover interesting biological patterns. In short, our work opens possible new avenues in understanding of and discovery in large data sets, arising in domains such as EM analysis.
Fundamental Limits of Controlled Stochastic Dynamical Systems: An Information-Theoretic Approach
Authors: Song Fang, Quanyan Zhu
Date: 2020-12-22 00:00:00

In this paper, we examine the fundamental performance limitations in the control of stochastic dynamical systems; more specifically, we derive generic $\mathcal{L}_p$ bounds that hold for any causal (stabilizing) controllers and any stochastic disturbances, by an information-theoretic analysis. We first consider the scenario where the plant (i.e., the dynamical system to be controlled) is linear time-invariant, and it is seen in general that the lower bounds are characterized by the unstable poles (or nonminimum-phase zeros) of the plant as well as the conditional entropy of the disturbance. We then analyze the setting where the plant is assumed to be (strictly) causal, for which case the lower bounds are determined by the conditional entropy of the disturbance. We also discuss the special cases of $p = 2$ and $p = \infty$, which correspond to minimum-variance control and controlling the maximum deviations, respectively. In addition, we investigate the power-spectral characterization of the lower bounds as well as its relation to the Kolmogorov-Szeg\"o formula.
C-Watcher: A Framework for Early Detection of High-Risk Neighborhoods Ahead of COVID-19 Outbreak
Authors: Congxi Xiao, Jingbo Zhou, Jizhou Huang, An Zhuo, Ji Liu, Haoyi Xiong, Dejing Dou
Date: 2020-12-22 00:00:00

The novel coronavirus disease (COVID-19) has crushed daily routines and is still rampaging through the world. Existing solution for nonpharmaceutical interventions usually needs to timely and precisely select a subset of residential urban areas for containment or even quarantine, where the spatial distribution of confirmed cases has been considered as a key criterion for the subset selection. While such containment measure has successfully stopped or slowed down the spread of COVID-19 in some countries, it is criticized for being inefficient or ineffective, as the statistics of confirmed cases are usually time-delayed and coarse-grained. To tackle the issues, we propose C-Watcher, a novel data-driven framework that aims at screening every neighborhood in a target city and predicting infection risks, prior to the spread of COVID-19 from epicenters to the city. In terms of design, C-Watcher collects large-scale long-term human mobility data from Baidu Maps, then characterizes every residential neighborhood in the city using a set of features based on urban mobility patterns. Furthermore, to transfer the firsthand knowledge (witted in epicenters) to the target city before local outbreaks, we adopt a novel adversarial encoder framework to learn "city-invariant" representations from the mobility-related features for precise early detection of high-risk neighborhoods, even before any confirmed cases known, in the target city. We carried out extensive experiments on C-Watcher using the real-data records in the early stage of COVID-19 outbreaks, where the results demonstrate the efficiency and effectiveness of C-Watcher for early detection of high-risk neighborhoods from a large number of cities.
How dark is the dark side of diversification?
Authors: Pedro Cadenas, Henryk Gzyl, Hyun Woong Park
Date: 2020-12-22 00:00:00

Against the widely held belief that diversification at banking institutions contributes to the stability of the financial system, Wagner (2010) found that diversification actually makes systemic crisis more likely. While it is true, as Wagner asserts, that the probability of joint default of the diversified portfolios is larger; we contend that, as common practice, the effect of diversification is examined with respect to a risk measure like VaR. We find that when banks use VaR, diversification does reduce individual and systemic risk. This, in turn, generates a different set of incentives for banks and regulators.