pwc-archive/papers-with-abstracts huggingface dataset:
CamLiFlow: Bidirectional Camera-LiDAR Fusion for Joint Optical Flow and Scene Flow Estimation
Authors: Haisong Liu, Tao Lu, Yihui Xu, Jia Liu, Wenjie Li, Lijun Chen
Date: 2021-11-20 00:00:00
Proceeding: CVPR 2022 1
Tasks: Optical Flow Estimation, Scene Flow Estimation

In this paper, we study the problem of jointly estimating the optical flow and scene flow from synchronized 2D and 3D data. Previous methods either employ a complex pipeline that splits the joint task into independent stages, or fuse 2D and 3D information in an "early-fusion" or "late-fusion" manner. Such one-size-fits-all approaches suffer from a dilemma of failing to fully utilize the characteristic of each modality or to maximize the inter-modality complementarity. To address the problem, we propose a novel end-to-end framework, called CamLiFlow. It consists of 2D and 3D branches with multiple bidirectional connections between them in specific layers. Different from previous work, we apply a point-based 3D branch to better extract the geometric features and design a symmetric learnable operator to fuse dense image features and sparse point features. Experiments show that CamLiFlow achieves better performance with fewer parameters. Our method ranks 1st on the KITTI Scene Flow benchmark, outperforming the previous art with 1/7 parameters. Code is available at https://github.com/MCG-NJU/CamLiFlow.
Discrete Representations Strengthen Vision Transformer Robustness
Authors: Chengzhi Mao, Lu Jiang, Mostafa Dehghani, Carl Vondrick, Rahul Sukthankar, Irfan Essa
Date: 2021-11-20 00:00:00
Proceeding: ICLR 2022 4
Tasks: Domain Generalization, Image Classification

Vision Transformer (ViT) is emerging as the state-of-the-art architecture for image recognition. While recent studies suggest that ViTs are more robust than their convolutional counterparts, our experiments find that ViTs trained on ImageNet are overly reliant on local textures and fail to make adequate use of shape information. ViTs thus have difficulties generalizing to out-of-distribution, real-world data. To address this deficiency, we present a simple and effective architecture modification to ViT's input layer by adding discrete tokens produced by a vector-quantized encoder. Different from the standard continuous pixel tokens, discrete tokens are invariant under small perturbations and contain less information individually, which promote ViTs to learn global information that is invariant. Experimental results demonstrate that adding discrete representation on four architecture variants strengthens ViT robustness by up to 12% across seven ImageNet robustness benchmarks while maintaining the performance on ImageNet.
Evaluation of automated airway morphological quantification for assessing fibrosing lung disease
Authors: Ashkan Pakzad, Wing Keung Cheung, Kin Quan, Nesrin Mogulkoc, Coline H. M. Van Moorsel, Brian J. Bartholmai, Hendrik W. Van Es, Alper Ezircan, Frouke Van Beek, Marcel Veltkamp, Ronald Karwoski, Tobias Peikert, Ryan D. Clay, Finbar Foley, Cassandra Braun, Recep Savas, Carole Sudre, Tom Doel, Daniel C. Alexander, Peter Wijeratne, David Hawkes, Yipeng Hu, John R Hurst, Joseph Jacob
Date: 2021-11-19 00:00:00
Tasks: Computed Tomography (CT)

Abnormal airway dilatation, termed traction bronchiectasis, is a typical feature of idiopathic pulmonary fibrosis (IPF). Volumetric computed tomography (CT) imaging captures the loss of normal airway tapering in IPF. We postulated that automated quantification of airway abnormalities could provide estimates of IPF disease extent and severity. We propose AirQuant, an automated computational pipeline that systematically parcellates the airway tree into its lobes and generational branches from a deep learning based airway segmentation, deriving airway structural measures from chest CT. Importantly, AirQuant prevents the occurrence of spurious airway branches by thick wave propagation and removes loops in the airway-tree by graph search, overcoming limitations of existing airway skeletonisation algorithms. Tapering between airway segments (intertapering) and airway tortuosity computed by AirQuant were compared between 14 healthy participants and 14 IPF patients. Airway intertapering was significantly reduced in IPF patients, and airway tortuosity was significantly increased when compared to healthy controls. Differences were most marked in the lower lobes, conforming to the typical distribution of IPF-related damage. AirQuant is an open-source pipeline that avoids limitations of existing airway quantification algorithms and has clinical interpretability. Automated airway measurements may have potential as novel imaging biomarkers of IPF severity and disease extent.
What Stops Learning-based 3D Registration from Working in the Real World?
Authors: Zheng Dang, Lizhou Wang, Junning Qiu, Minglei Lu, Mathieu Salzmann
Date: 2021-11-19 00:00:00
Tasks: Point Cloud Registration

Much progress has been made on the task of learning-based 3D point cloud registration, with existing methods yielding outstanding results on standard benchmarks, such as ModelNet40, even in the partial-to-partial matching scenario. Unfortunately, these methods still struggle in the presence of real data. In this work, we identify the sources of these failures, analyze the reasons behind them, and propose solutions to tackle them. We summarise our findings into a set of guidelines and demonstrate their effectiveness by applying them to different baseline methods, DCP and IDAM. In short, our guidelines improve both their training convergence and testing accuracy. Ultimately, this translates to a best-practice 3D registration network (BPNet), constituting the first learning-based method able to handle previously-unseen objects in real-world data. Despite being trained only on synthetic data, our model generalizes to real data without any fine-tuning, reaching an accuracy of up to 67% on point clouds of unseen objects obtained with a commercial sensor.
Diabetic Foot Ulcer Grand Challenge 2021: Evaluation and Summary
Authors: Bill Cassidy, Connah Kendrick, Neil D. Reeves, Joseph M. Pappachan, Claire O'Shea, David G. Armstrong, Moi Hoon Yap
Date: 2021-11-19 00:00:00

Diabetic foot ulcer classification systems use the presence of wound infection (bacteria present within the wound) and ischaemia (restricted blood supply) as vital clinical indicators for treatment and prediction of wound healing. Studies investigating the use of automated computerised methods of classifying infection and ischaemia within diabetic foot wounds are limited due to a paucity of publicly available datasets and severe data imbalance in those few that exist. The Diabetic Foot Ulcer Challenge 2021 provided participants with a more substantial dataset comprising a total of 15,683 diabetic foot ulcer patches, with 5,955 used for training, 5,734 used for testing and an additional 3,994 unlabelled patches to promote the development of semi-supervised and weakly-supervised deep learning techniques. This paper provides an evaluation of the methods used in the Diabetic Foot Ulcer Challenge 2021, and summarises the results obtained from each network. The best performing network was an ensemble of the results of the top 3 models, with a macro-average F1-score of 0.6307.
Urine Microscopic Image Dataset
Authors: Dipam Goswami, Hari Om Aggrawal, Rajiv Gupta, Vinti Agarwal
Date: 2021-11-19 00:00:00
Tasks: Diagnostic

Urinalysis is a standard diagnostic test to detect urinary system related problems. The automation of urinalysis will reduce the overall diagnostic time. Recent studies used urine microscopic datasets for designing deep learning based algorithms to classify and detect urine cells. But these datasets are not publicly available for further research. To alleviate the need for urine datsets, we prepare our urine sediment microscopic image (UMID) dataset comprising of around 3700 cell annotations and 3 categories of cells namely RBC, pus and epithelial cells. We discuss the several challenges involved in preparing the dataset and the annotations. We make the dataset publicly available.
Resistance-Time Co-Modulated PointNet for Temporal Super-Resolution Simulation of Blood Vessel Flows
Authors: Zhizheng Jiang, Fei Gao, Renshu Gu, Jinlan Xu, Gang Xu, Timon Rabczuk
Date: 2021-11-19 00:00:00
Tasks: Decoder, Super-Resolution

In this paper, a novel deep learning framework is proposed for temporal super-resolution simulation of blood vessel flows, in which a high-temporal-resolution time-varying blood vessel flow simulation is generated from a low-temporal-resolution flow simulation result. In our framework, point-cloud is used to represent the complex blood vessel model, resistance-time aided PointNet model is proposed for extracting the time-space features of the time-varying flow field, and finally we can reconstruct the high-accuracy and high-resolution flow field through the Decoder module. In particular, the amplitude loss and the orientation loss of the velocity are proposed from the vector characteristics of the velocity. And the combination of these two metrics constitutes the final loss function for network training. Several examples are given to illustrate the effective and efficiency of the proposed framework for temporal super-resolution simulation of blood vessel flows.
ColDE: A Depth Estimation Framework for Colonoscopy Reconstruction
Authors: Yubo Zhang, Jan-Michael Frahm, Samuel Ehrenstein, Sarah K. McGill, Julian G. Rosenman, Shuxian Wang, Stephen M. Pizer
Date: 2021-11-19 00:00:00
Tasks: 3D Reconstruction, Depth Estimation, Video Reconstruction

One of the key elements of reconstructing a 3D mesh from a monocular video is generating every frame's depth map. However, in the application of colonoscopy video reconstruction, producing good-quality depth estimation is challenging. Neural networks can be easily fooled by photometric distractions or fail to capture the complex shape of the colon surface, predicting defective shapes that result in broken meshes. Aiming to fundamentally improve the depth estimation quality for colonoscopy 3D reconstruction, in this work we have designed a set of training losses to deal with the special challenges of colonoscopy data. For better training, a set of geometric consistency objectives was developed, using both depth and surface normal information. Also, the classic photometric loss was extended with feature matching to compensate for illumination noise. With the training losses powerful enough, our self-supervised framework named ColDE is able to produce better depth maps of colonoscopy data as compared to the previous work utilizing prior depth knowledge. Used in reconstruction, our network is able to reconstruct good-quality colon meshes in real-time without any post-processing, making it the first to be clinically applicable.
Improved Model based Deep Learning using Monotone Operator Learning (MOL)
Authors: Aniket Pramanik, Mathews Jacob
Date: 2021-11-22 00:00:00
Tasks: Deep Learning, Operator learning, Rolling Shutter Correction

Model-based deep learning (MoDL) algorithms that rely on unrolling are emerging as powerful tools for image recovery. In this work, we introduce a novel monotone operator learning framework to overcome some of the challenges associated with current unrolled frameworks, including high memory cost, lack of guarantees on robustness to perturbations, and low interpretability. Unlike current unrolled architectures that use finite number of iterations, we use the deep equilibrium (DEQ) framework to iterate the algorithm to convergence and to evaluate the gradient of the convolutional neural network blocks using Jacobian iterations. This approach significantly reduces the memory demand, facilitating the extension of MoDL algorithms to high dimensional problems. We constrain the CNN to be a monotone operator, which allows us to introduce algorithms with guaranteed convergence properties and robustness guarantees. We demonstrate the utility of the proposed scheme in the context of parallel MRI.
Network-wide Multi-step Traffic Volume Prediction using Graph Convolutional Gated Recurrent Neural Network
Authors: Lei Lin, Weizi Li, Lei Zhu
Date: 2021-11-22 00:00:00

Accurate prediction of network-wide traffic conditions is essential for intelligent transportation systems. In the last decade, machine learning techniques have been widely used for this task, resulting in state-of-the-art performance. We propose a novel deep learning model, Graph Convolutional Gated Recurrent Neural Network (GCGRNN), to predict network-wide, multi-step traffic volume. GCGRNN can automatically capture spatial correlations between traffic sensors and temporal dependencies in historical traffic data. We have evaluated our model using two traffic datasets extracted from 150 sensors in Los Angeles, California, at the time resolutions one hour and 15 minutes, respectively. The results show that our model outperforms the other five benchmark models in terms of prediction accuracy. For instance, our model reduces MAE by 25.3%, RMSE by 29.2%, and MAPE by 20.2%, compared to the state-of-the-art Diffusion Convolutional Recurrent Neural Network (DCRNN) model using the hourly dataset. Our model also achieves faster training than DCRNN by up to 52%. The data and implementation of GCGRNN can be found at https://github.com/leilin-research/GCGRNN.
No-Regret Dynamics in the Fenchel Game: A Unified Framework for Algorithmic Convex Optimization
Authors: Jun-Kun Wang, Jacob Abernethy, Kfir Y. Levy
Date: 2021-11-22 00:00:00

We develop an algorithmic framework for solving convex optimization problems using no-regret game dynamics. By converting the problem of minimizing a convex function into an auxiliary problem of solving a min-max game in a sequential fashion, we can consider a range of strategies for each of the two-players who must select their actions one after the other. A common choice for these strategies are so-called no-regret learning algorithms, and we describe a number of such and prove bounds on their regret. We then show that many classical first-order methods for convex optimization -- including average-iterate gradient descent, the Frank-Wolfe algorithm, Nesterov's acceleration methods, and the accelerated proximal method -- can be interpreted as special cases of our framework as long as each player makes the correct choice of no-regret strategy. Proving convergence rates in this framework becomes very straightforward, as they follow from plugging in the appropriate known regret bounds. Our framework also gives rise to a number of new first-order methods for special cases of convex optimization that were not previously known.
Machine Learning of Thermodynamic Observables in the Presence of Mode Collapse
Authors: Kim A. Nicoli, Christopher Anders, Lena Funcke, Tobias Hartung, Karl Jansen, Pan Kessel, Shinichi Nakajima, Paolo Stornati
Date: 2021-11-22 00:00:00
Tasks: BIG-bench Machine Learning

Estimating the free energy, as well as other thermodynamic observables, is a key task in lattice field theories. Recently, it has been pointed out that deep generative models can be used in this context [1]. Crucially, these models allow for the direct estimation of the free energy at a given point in parameter space. This is in contrast to existing methods based on Markov chains which generically require integration through parameter space. In this contribution, we will review this novel machine-learning-based estimation method. We will in detail discuss the issue of mode collapse and outline mitigation techniques which are particularly suited for applications at finite temperature.
Teaching Humans When To Defer to a Classifier via Exemplars
Authors: Hussein Mozannar, Arvind Satyanarayan, David Sontag
Date: 2021-11-22 00:00:00
Tasks: Multi-hop Question Answering, Question Answering, valid

Expert decision makers are starting to rely on data-driven automated agents to assist them with various tasks. For this collaboration to perform properly, the human decision maker must have a mental model of when and when not to rely on the agent. In this work, we aim to ensure that human decision makers learn a valid mental model of the agent's strengths and weaknesses. To accomplish this goal, we propose an exemplar-based teaching strategy where humans solve the task with the help of the agent and try to formulate a set of guidelines of when and when not to defer. We present a novel parameterization of the human's mental model of the AI that applies a nearest neighbor rule in local regions surrounding the teaching examples. Using this model, we derive a near-optimal strategy for selecting a representative teaching set. We validate the benefits of our teaching strategy on a multi-hop question answering task using crowd workers and find that when workers draw the right lessons from the teaching stage, their task performance improves, we furthermore validate our method on a set of synthetic experiments.
Bridging the reality gap in quantum devices with physics-aware machine learning
Authors: D. L. Craig, H. Moon, F. Fedele, D. T. Lennon, B. Van Straaten, F. Vigneau, L. C. Camenzind, D. M. Zumbühl, G. A. D. Briggs, M. A. Osborne, D. Sejdinovic, N. Ares
Date: 2021-11-22 00:00:00
Tasks: Bayesian Inference, BIG-bench Machine Learning

The discrepancies between reality and simulation impede the optimisation and scalability of solid-state quantum devices. Disorder induced by the unpredictable distribution of material defects is one of the major contributions to the reality gap. We bridge this gap using physics-aware machine learning, in particular, using an approach combining a physical model, deep learning, Gaussian random field, and Bayesian inference. This approach has enabled us to infer the disorder potential of a nanoscale electronic device from electron transport data. This inference is validated by verifying the algorithm's predictions about the gate voltage values required for a laterally-defined quantum dot device in AlGaAs/GaAs to produce current features corresponding to a double quantum dot regime.
BarrierNet: A Safety-Guaranteed Layer for Neural Networks
Authors: Wei Xiao, Ramin Hasani, Xiao Li, Daniela Rus
Date: 2021-11-22 00:00:00

This paper introduces differentiable higher-order control barrier functions (CBF) that are end-to-end trainable together with learning systems. CBFs are usually overly conservative, while guaranteeing safety. Here, we address their conservativeness by softening their definitions using environmental dependencies without loosing safety guarantees, and embed them into differentiable quadratic programs. These novel safety layers, termed a BarrierNet, can be used in conjunction with any neural network-based controller, and can be trained by gradient descent. BarrierNet allows the safety constraints of a neural controller be adaptable to changing environments. We evaluate them on a series of control problems such as traffic merging and robot navigations in 2D and 3D space, and demonstrate their effectiveness compared to state-of-the-art approaches.
Policy Gradient and Actor-Critic Learning in Continuous Time and Space: Theory and Algorithms
Authors: Yanwei Jia, Xun Yu Zhou
Date: 2021-11-22 00:00:00

We study policy gradient (PG) for reinforcement learning in continuous time and space under the regularized exploratory formulation developed by Wang et al. (2020). We represent the gradient of the value function with respect to a given parameterized stochastic policy as the expected integration of an auxiliary running reward function that can be evaluated using samples and the current value function. This effectively turns PG into a policy evaluation (PE) problem, enabling us to apply the martingale approach recently developed by Jia and Zhou (2021) for PE to solve our PG problem. Based on this analysis, we propose two types of the actor-critic algorithms for RL, where we learn and update value functions and policies simultaneously and alternatingly. The first type is based directly on the aforementioned representation which involves future trajectories and hence is offline. The second type, designed for online learning, employs the first-order condition of the policy gradient and turns it into martingale orthogonality conditions. These conditions are then incorporated using stochastic approximation when updating policies. Finally, we demonstrate the algorithms by simulations in two concrete examples.
Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectification
Authors: Ling Pan, Longbo Huang, Tengyu Ma, Huazhe Xu
Date: 2021-11-22 00:00:00
Tasks: Continuous Control, Multi-agent Reinforcement Learning, Offline RL, reinforcement-learning, Reinforcement Learning (RL)

Conservatism has led to significant progress in offline reinforcement learning (RL) where an agent learns from pre-collected datasets. However, as many real-world scenarios involve interaction among multiple agents, it is important to resolve offline RL in the multi-agent setting. Given the recent success of transferring online RL algorithms to the multi-agent setting, one may expect that offline RL algorithms will also transfer to multi-agent settings directly. Surprisingly, we empirically observe that conservative offline RL algorithms do not work well in the multi-agent setting -- the performance degrades significantly with an increasing number of agents. Towards mitigating the degradation, we identify a key issue that non-concavity of the value function makes the policy gradient improvements prone to local optima. Multiple agents exacerbate the problem severely, since the suboptimal policy by any agent can lead to uncoordinated global failure. Following this intuition, we propose a simple yet effective method, Offline Multi-Agent RL with Actor Rectification (OMAR), which combines the first-order policy gradients and zeroth-order optimization methods to better optimize the conservative value functions over the actor parameters. Despite the simplicity, OMAR achieves state-of-the-art results in a variety of multi-agent control tasks.
Towards Understanding the Unreasonable Effectiveness of Learning AC-OPF Solutions
Authors: My H. Dinh, Ferdinando Fioretto, Mostafa Mohammadian, Kyri Baker
Date: 2021-11-22 00:00:00

Optimal Power Flow (OPF) is a fundamental problem in power systems. It is computationally challenging and a recent line of research has proposed the use of Deep Neural Networks (DNNs) to find OPF approximations at vastly reduced runtimes when compared to those obtained by classical optimization methods. While these works show encouraging results in terms of accuracy and runtime, little is known on why these models can predict OPF solutions accurately, as well as about their robustness. This paper provides a step forward to address this knowledge gap. The paper connects the volatility of the outputs of the generators to the ability of a learning model to approximate them, it sheds light on the characteristics affecting the DNN models to learn good predictors, and it proposes a new model that exploits the observations made by this paper to produce accurate and robust OPF predictions.
Feature extraction of machine learning and phase transition point of Ising model
Authors: Shotaro Shiba Funai
Date: 2021-11-22 00:00:00
Tasks: BIG-bench Machine Learning

We study the features extracted by the Restricted Boltzmann Machine (RBM) when it is trained with spin configurations of Ising model at various temperatures. Using the trained RBM, we obtain the flow of iterative reconstructions (RBM flow) of the spin configurations and find that in some cases the flow approaches the phase transition point $T=T_c$ in Ising model. Since the extracted features are emphasized in the reconstructed configurations, the configurations at such a fixed point should describe nothing but the extracted features. Then we investigate the dependence of the fixed point on various parameters and conjecture the condition where the fixed point of the RBM flow is at the phase transition point. We also provide supporting evidence for the conjecture by analyzing the weight matrix of the trained RBM.