pwc-archive/papers-with-abstracts huggingface dataset:
Object-Centric Diagnosis of Visual Reasoning
Authors: Jianwei Yang, Jiayuan Mao, Jiajun Wu, Devi Parikh, David D. Cox, Joshua B. Tenenbaum, Chuang Gan
Date: 2020-12-21 00:00:00
Tasks: Diagnostic, Object, Question Answering, Visual Question Answering, Visual Question Answering (VQA), Visual Reasoning

When answering questions about an image, it not only needs knowing what -- understanding the fine-grained contents (e.g., objects, relationships) in the image, but also telling why -- reasoning over grounding visual cues to derive the answer for a question. Over the last few years, we have seen significant progress on visual question answering. Though impressive as the accuracy grows, it still lags behind to get knowing whether these models are undertaking grounding visual reasoning or just leveraging spurious correlations in the training data. Recently, a number of works have attempted to answer this question from perspectives such as grounding and robustness. However, most of them are either focusing on the language side or coarsely studying the pixel-level attention maps. In this paper, by leveraging the step-wise object grounding annotations provided in the GQA dataset, we first present a systematical object-centric diagnosis of visual reasoning on grounding and robustness, particularly on the vision side. According to the extensive comparisons across different models, we find that even models with high accuracy are not good at grounding objects precisely, nor robust to visual content perturbations. In contrast, symbolic and modular models have a relatively better grounding and robustness, though at the cost of accuracy. To reconcile these different aspects, we further develop a diagnostic model, namely Graph Reasoning Machine. Our model replaces purely symbolic visual representation with probabilistic scene graph and then applies teacher-forcing training for the visual reasoning module. The designed model improves the performance on all three metrics over the vanilla neural-symbolic model while inheriting the transparency. Further ablation studies suggest that this improvement is mainly due to more accurate image understanding and proper intermediate reasoning supervisions.
End-to-End Deep Structured Models for Drawing Crosswalks
Authors: Justin Liang, Raquel Urtasun
Date: 2020-12-21 00:00:00
Proceeding: ECCV 2018 9

In this paper we address the problem of detecting crosswalks from LiDAR and camera imagery. Towards this goal, given multiple LiDAR sweeps and the corresponding imagery, we project both inputs onto the ground surface to produce a top down view of the scene. We then leverage convolutional neural networks to extract semantic cues about the location of the crosswalks. These are then used in combination with road centerlines from freely available maps (e.g., OpenStreetMaps) to solve a structured optimization problem which draws the final crosswalk boundaries. Our experiments over crosswalks in a large city area show that 96.6% automation can be achieved.
Semantic Audio-Visual Navigation
Authors: Changan Chen, Ziad Al-Halah, Kristen Grauman
Date: 2020-12-21 00:00:00
Proceeding: CVPR 2021 1
Tasks: Position, Visual Navigation

Recent work on audio-visual navigation assumes a constantly-sounding target and restricts the role of audio to signaling the target's position. We introduce semantic audio-visual navigation, where objects in the environment make sounds consistent with their semantic meaning (e.g., toilet flushing, door creaking) and acoustic events are sporadic or short in duration. We propose a transformer-based model to tackle this new semantic AudioGoal task, incorporating an inferred goal descriptor that captures both spatial and semantic properties of the target. Our model's persistent multimodal memory enables it to reach the goal even long after the acoustic event stops. In support of the new task, we also expand the SoundSpaces audio simulations to provide semantically grounded sounds for an array of objects in Matterport3D. Our method strongly outperforms existing audio-visual navigation methods by learning to associate semantic, acoustic, and visual cues.
HyperSeg: Patch-wise Hypernetwork for Real-time Semantic Segmentation
Authors: Yuval Nirkin, Lior Wolf, Tal Hassner
Date: 2020-12-21 00:00:00
Proceeding: CVPR 2021 1
Tasks: Decoder, Dichotomous Image Segmentation, Real-Time Semantic Segmentation, Segmentation, Semantic Segmentation

We present a novel, real-time, semantic segmentation network in which the encoder both encodes and generates the parameters (weights) of the decoder. Furthermore, to allow maximal adaptivity, the weights at each decoder block vary spatially. For this purpose, we design a new type of hypernetwork, composed of a nested U-Net for drawing higher level context features, a multi-headed weight generating module which generates the weights of each block in the decoder immediately before they are consumed, for efficient memory utilization, and a primary network that is composed of novel dynamic patch-wise convolutions. Despite the usage of less-conventional blocks, our architecture obtains real-time performance. In terms of the runtime vs. accuracy trade-off, we surpass state of the art (SotA) results on popular semantic segmentation benchmarks: PASCAL VOC 2012 (val. set) and real-time semantic segmentation on Cityscapes, and CamVid. The code is available: https://nirkin.com/hyperseg.
Populating 3D Scenes by Learning Human-Scene Interaction
Authors: Mohamed Hassan, Partha Ghosh, Joachim Tesch, Dimitrios Tzionas, Michael J. Black
Date: 2020-12-21 00:00:00
Proceeding: CVPR 2021 1
Tasks: Contact Detection, Dense contact estimation, Pose Estimation

Humans live within a 3D space and constantly interact with it to perform tasks. Such interactions involve physical contact between surfaces that is semantically meaningful. Our goal is to learn how humans interact with scenes and leverage this to enable virtual characters to do the same. To that end, we introduce a novel Human-Scene Interaction (HSI) model that encodes proximal relationships, called POSA for "Pose with prOximitieS and contActs". The representation of interaction is body-centric, which enables it to generalize to new scenes. Specifically, POSA augments the SMPL-X parametric human body model such that, for every mesh vertex, it encodes (a) the contact probability with the scene surface and (b) the corresponding semantic scene label. We learn POSA with a VAE conditioned on the SMPL-X vertices, and train on the PROX dataset, which contains SMPL-X meshes of people interacting with 3D scenes, and the corresponding scene semantics from the PROX-E dataset. We demonstrate the value of POSA with two applications. First, we automatically place 3D scans of people in scenes. We use a SMPL-X model fit to the scan as a proxy and then find its most likely placement in 3D. POSA provides an effective representation to search for "affordances" in the scene that match the likely contact relationships for that pose. We perform a perceptual study that shows significant improvement over the state of the art on this task. Second, we show that POSA's learned representation of body-scene interaction supports monocular human pose estimation that is consistent with a 3D scene, improving on the state of the art. Our model and code are available for research purposes at https://posa.is.tue.mpg.de.
Multi-Agent Online Optimization with Delays: Asynchronicity, Adaptivity, and Optimism
Authors: Yu-Guan Hsieh, Franck Iutzeler, Jérôme Malick, Panayotis Mertikopoulos
Date: 2020-12-21 00:00:00

In this paper, we provide a general framework for studying multi-agent online learning problems in the presence of delays and asynchronicities. Specifically, we propose and analyze a class of adaptive dual averaging schemes in which agents only need to accumulate gradient feedback received from the whole system, without requiring any between-agent coordination. In the single-agent case, the adaptivity of the proposed method allows us to extend a range of existing results to problems with potentially unbounded delays between playing an action and receiving the corresponding feedback. In the multi-agent case, the situation is significantly more complicated because agents may not have access to a global clock to use as a reference point; to overcome this, we focus on the information that is available for producing each prediction rather than the actual delay associated with each feedback. This allows us to derive adaptive learning strategies with optimal regret bounds, even in a fully decentralized, asynchronous environment. Finally, we also analyze an "optimistic" variant of the proposed algorithm which is capable of exploiting the predictability of problems with a slower variation and leads to improved regret bounds.
Deep Learning in Detection and Diagnosis of Covid-19 using Radiology Modalities: A Systematic Review
Authors: Mustafa Ghaderzadeh, Farkhondeh Asadi
Date: 2020-12-21 00:00:00
Tasks: Articles, Deep Learning, Diagnostic, Specificity

Purpose: Early detection and diagnosis of Covid-19 and accurate separation of patients with non-Covid-19 cases at the lowest cost and in the early stages of the disease are one of the main challenges in the epidemic of Covid-19. Concerning the novelty of the disease, the diagnostic methods based on radiological images suffer shortcomings despite their many uses in diagnostic centers. Accordingly, medical and computer researchers tended to use machine-learning models to analyze radiology images. Methods: Present systematic review was conducted by searching three databases of PubMed, Scopus, and Web of Science from November 1, 2019, to July 20, 2020 Based on a search strategy, the keywords were Covid-19, Deep learning, Diagnosis and Detection leading to the extraction of 168 articles that ultimately, 37 articles were selected as the research population by applying inclusion and exclusion criteria. Result: This review study provides an overview of the current state of all models for the detection and diagnosis of Covid-19 through radiology modalities and their processing based on deep learning. According to the finding, Deep learning Based models have an extraordinary capacity to achieve an accurate and efficient system for the detection and diagnosis of Covid-19, which using of them in the processing of CT-Scan and X-Ray images, would lead to a significant increase in sensitivity and specificity values. Conclusion: The Application of Deep Learning (DL) in the field of Covid-19 radiologic image processing leads to the reduction of false-positive and negative errors in the detection and diagnosis of this disease and provides an optimal opportunity to provide fast, cheap, and safe diagnostic services to patients.
From Points to Multi-Object 3D Reconstruction
Authors: Francis Engelmann, Konstantinos Rematas, Bastian Leibe, Vittorio Ferrari
Date: 2020-12-21 00:00:00
Proceeding: CVPR 2021 1
Tasks: 3D Reconstruction, 3D Shape Reconstruction, Object

We propose a method to detect and reconstruct multiple 3D objects from a single RGB image. The key idea is to optimize for detection, alignment and shape jointly over all objects in the RGB image, while focusing on realistic and physically plausible reconstructions. To this end, we propose a keypoint detector that localizes objects as center points and directly predicts all object properties, including 9-DoF bounding boxes and 3D shapes -- all in a single forward pass. The proposed method formulates 3D shape reconstruction as a shape selection problem, i.e. it selects among exemplar shapes from a given database. This makes it agnostic to shape representations, which enables a lightweight reconstruction of realistic and visually-pleasing shapes based on CAD-models, while the training objective is formulated around point clouds and voxel representations. A collision-loss promotes non-intersecting objects, further increasing the reconstruction realism. Given the RGB image, the presented approach performs lightweight reconstruction in a single-stage, it is real-time capable, fully differentiable and end-to-end trainable. Our experiments compare multiple approaches for 9-DoF bounding box estimation, evaluate the novel shape-selection mechanism and compare to recent methods in terms of 3D bounding box estimation and 3D shape reconstruction quality.
Analyzing Dominance Move (MIP-DoM) Indicator for Multi- and Many-objective Optimization
Authors: Claudio Lucio do Val Lopes, Flávio Vinícius Cruzeiro Martins, Elizabeth Fialho Wanner, Kalyanmoy Deb
Date: 2020-12-21 00:00:00

Dominance move (DoM) is a binary quality indicator that can be used in multi-objective and many-objective optimization to compare two solution sets obtained from different algorithms. The DoM indicator can differentiate the sets for certain important features, such as convergence, spread, uniformity, and cardinality. DoM does not use any reference, and it has an intuitive and physical meaning, similar to the $\epsilon$-indicator, and calculates the minimum total move of members of one set so that all elements in another set are to be dominated or identical to at least one member of the first set. Despite the aforementioned properties, DoM is hard to calculate, particularly in higher dimensions. There is an efficient and exact method to calculate it in a bi-objective case only. This work proposes a novel approach to calculate DoM using a mixed integer programming (MIP) approach, which can handle sets with three or more objectives and is shown to overcome the $\epsilon$-indicator's information loss. Experiments, in the bi-objective space, are done to verify the model's correctness. Furthermore, other experiments, using 3, 5, 10, 15, 20, 25 and 30-objective problems are performed to show how the model behaves in higher-dimensional cases. Algorithms, such as IBEA, MOEA/D, NSGA-III, NSGA-II, and SPEA2 are used to generate the solution sets (however any other algorithms can also be used with the proposed MIP-DoM indicator). Further extensions are discussed to handle certain idiosyncrasies with some solution sets and also to improve the quality indicator and its use for other situations.
Variational Transport: A Convergent Particle-BasedAlgorithm for Distributional Optimization
Authors: Zhuoran Yang, Yufeng Zhang, Yongxin Chen, Zhaoran Wang
Date: 2020-12-21 00:00:00
Tasks: Generative Adversarial Network, Variational Inference

We consider the optimization problem of minimizing a functional defined over a family of probability distributions, where the objective functional is assumed to possess a variational form. Such a distributional optimization problem arises widely in machine learning and statistics, with Monte-Carlo sampling, variational inference, policy optimization, and generative adversarial network as examples. For this problem, we propose a novel particle-based algorithm, dubbed as variational transport, which approximately performs Wasserstein gradient descent over the manifold of probability distributions via iteratively pushing a set of particles. Specifically, we prove that moving along the geodesic in the direction of functional gradient with respect to the second-order Wasserstein distance is equivalent to applying a pushforward mapping to a probability distribution, which can be approximated accurately by pushing a set of particles. Specifically, in each iteration of variational transport, we first solve the variational problem associated with the objective functional using the particles, whose solution yields the Wasserstein gradient direction. Then we update the current distribution by pushing each particle along the direction specified by such a solution. By characterizing both the statistical error incurred in estimating the Wasserstein gradient and the progress of the optimization algorithm, we prove that when the objective function satisfies a functional version of the Polyak-\L{}ojasiewicz (PL) (Polyak, 1963) and smoothness conditions, variational transport converges linearly to the global minimum of the objective functional up to a certain statistical error, which decays to zero sublinearly as the number of particles goes to infinity.
AVAE: Adversarial Variational Auto Encoder
Authors: Antoine Plumerault, Hervé Le Borgne, Céline Hudelot
Date: 2020-12-21 00:00:00

Among the wide variety of image generative models, two models stand out: Variational Auto Encoders (VAE) and Generative Adversarial Networks (GAN). GANs can produce realistic images, but they suffer from mode collapse and do not provide simple ways to get the latent representation of an image. On the other hand, VAEs do not have these problems, but they often generate images less realistic than GANs. In this article, we explain that this lack of realism is partially due to a common underestimation of the natural image manifold dimensionality. To solve this issue we introduce a new framework that combines VAE and GAN in a novel and complementary way to produce an auto-encoding model that keeps VAEs properties while generating images of GAN-quality. We evaluate our approach both qualitatively and quantitatively on five image datasets.
Offline Reinforcement Learning from Images with Latent Space Models
Authors: Rafael Rafailov, Tianhe Yu, Aravind Rajeswaran, Chelsea Finn
Date: 2020-12-21 00:00:00
Tasks: Offline RL, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Offline reinforcement learning (RL) refers to the problem of learning policies from a static dataset of environment interactions. Offline RL enables extensive use and re-use of historical datasets, while also alleviating safety concerns associated with online exploration, thereby expanding the real-world applicability of RL. Most prior work in offline RL has focused on tasks with compact state representations. However, the ability to learn directly from rich observation spaces like images is critical for real-world applications such as robotics. In this work, we build on recent advances in model-based algorithms for offline RL, and extend them to high-dimensional visual observation spaces. Model-based offline RL algorithms have achieved state of the art results in state based tasks and have strong theoretical guarantees. However, they rely crucially on the ability to quantify uncertainty in the model predictions, which is particularly challenging with image observations. To overcome this challenge, we propose to learn a latent-state dynamics model, and represent the uncertainty in the latent space. Our approach is both tractable in practice and corresponds to maximizing a lower bound of the ELBO in the unknown POMDP. In experiments on a range of challenging image-based locomotion and manipulation tasks, we find that our algorithm significantly outperforms previous offline model-free RL methods as well as state-of-the-art online visual model-based RL methods. Moreover, we also find that our approach excels on an image-based drawer closing task on a real robot using a pre-existing dataset. All results including videos can be found online at https://sites.google.com/view/lompo/ .
Reflective Parametric Frequency Selective Limiters with sub-dB Loss and $μ$Watts Power Thresholds
Authors: Hussein M. E. Hussein, Mahmoud A. A. Ibrahim, Matteo Rinaldi, Marvin Onabajo, Cristian Cassella
Date: 2020-12-21 00:00:00

This article describes the design methodology to achieve reflective diode-based parametric frequency selective limiters (pFSLs) with low power thresholds ($P_{th}$) and sub-dB insertion-loss values ($IL^{s.s}$) for driving power levels ($P_{in}$) lower than $P_{th}$. In addition, we present the measured performance of a reflective pFSL designed through the discussed methodology and assembled on a FR-4 printed circuit board (PCB). Thanks to its optimally engineered dynamics, the built pFSL can operate around $\sim$2.1 GHz while exhibiting record-low $P_{th}$ (-3.4 dBm) and $IL^{s.s}$ (0.94 dB) values. Furthermore, while the pFSL can selectively attenuate undesired signals with power ranging from -3.4 dBm to 13 dBm, it provides a strong suppression level (IS > 12.0 dB) even when driven by much higher $P_{in}$ values approaching 28 dBm. Such measured performance metrics demonstrate how the unique nonlinear dynamics of parametric-based FSLs can be leveraged through components and systems compatible with conventional chip-scale manufacturing processes in order to increase the resilience to electromagnetic interference (EMI), even of wireless radios designed for a low-power consumption and consequently characterized by a narrow dynamic range.
Building LEGO Using Deep Generative Models of Graphs
Authors: Rylee Thompson, Elahe Ghalebi, Terrance DeVries, Graham W. Taylor
Date: 2020-12-21 00:00:00

Generative models are now used to create a variety of high-quality digital artifacts. Yet their use in designing physical objects has received far less attention. In this paper, we advocate for the construction toy, LEGO, as a platform for developing generative models of sequential assembly. We develop a generative model based on graph-structured neural networks that can learn from human-built structures and produce visually compelling designs. Our code is released at: https://github.com/uoguelph-mlrg/GenerativeLEGO.
An Efficient and Incentive-Compatible Mechanism for Energy Storage Markets
Authors: Bharadwaj Satchidanandan, Munther A. Dahleh
Date: 2020-12-21 00:00:00

A key obstacle to increasing renewable energy penetration in the power grid is the lack of utility-scale storage capacity. Transportation electrification has the potential to overcome this obstacle since Electric Vehicles (EVs) that are not in transit can provide battery storage as a service to the grid. This is referred to as EV-Power grid integration, and could potentially be a key milestone in the pathway to decarbonize the electricity and the transportation sectors. We first show that if EV-Power grid integration is not done carefully, then contrary to improving the cost efficiency of operating the grid, it could in fact be counterproductive to it. This fundamentally occurs due to two phenomena operating in tandem - the randomness of EV usage patterns and the possibility of strategic behavior by EV operators. We present a market-based solution to address this issue. Specifically, we develop a mechanism for energy storage markets using which the system operator can efficiently integrate a fleet of strategic EVs with random usage patterns into the grid, utilize them for storage, and satisfy the demand at minimum possible cost.
Instabilities in Multi-Asset and Multi-Agent Market Impact Games
Authors: Francesco Cordoni, Fabrizio Lillo
Date: 2020-04-07 00:00:00

We consider the general problem of a set of agents trading a portfolio of assets in the presence of transient price impact and additional quadratic transaction costs and we study, with analytical and numerical methods, the resulting Nash equilibria. Extending significantly the framework of Schied & Zhang (2019) and Luo & Schied (2020), who considered the single asset case, we prove the existence and uniqueness of the corresponding Nash equilibria for the related mean-variance optimization problem. We then focus our attention on the conditions on the model parameters making the trading profile of the agents at equilibrium, and as a consequence the price trajectory, wildly oscillating and the market unstable. While Schied & Zhang (2019) and Luo & Schied (2020) highlighted the importance of the value of transaction cost in determining the transition between a stable and an unstable phase, we show that also the scaling of market impact with the number of agents J and the number of assets M determines the asymptotic stability (in J and M ) of markets.