pwc-archive/papers-with-abstracts huggingface dataset:
YOLO-Z: Improving small object detection in YOLOv5 for autonomous vehicles
Authors: Aduen Benjumea, Izzeddin Teeti, Fabio Cuzzolin, Andrew Bradley
Date: 2021-12-22 00:00:00
Tasks: Autonomous Racing, Autonomous Vehicles, object-detection, Object Detection, Small Object Detection

As autonomous vehicles and autonomous racing rise in popularity, so does the need for faster and more accurate detectors. While our naked eyes are able to extract contextual information almost instantly, even from far away, image resolution and computational resources limitations make detecting smaller objects (that is, objects that occupy a small pixel area in the input image) a genuinely challenging task for machines and a wide-open research field. This study explores how the popular YOLOv5 object detector can be modified to improve its performance in detecting smaller objects, with a particular application in autonomous racing. To achieve this, we investigate how replacing certain structural elements of the model (as well as their connections and other parameters) can affect performance and inference time. In doing so, we propose a series of models at different scales, which we name `YOLO-Z', and which display an improvement of up to 6.9% in mAP when detecting smaller objects at 50% IOU, at the cost of just a 3ms increase in inference time compared to the original YOLOv5. Our objective is to inform future research on the potential of adjusting a popular detector such as YOLOv5 to address specific tasks and provide insights on how specific changes can impact small object detection. Such findings, applied to the broader context of autonomous vehicles, could increase the amount of contextual information available to such systems.
Causal Inference Despite Limited Global Confounding via Mixture Models
Authors: Spencer L. Gordon, Bijan Mazaheri, Yuval Rabani, Leonard J. Schulman
Date: 2021-12-22 00:00:00
Tasks: Causal Inference

A Bayesian Network is a directed acyclic graph (DAG) on a set of $n$ random variables (the vertices); a Bayesian Network Distribution (BND) is a probability distribution on the random variables that is Markovian on the graph. A finite $k$-mixture of such models is graphically represented by a larger graph which has an additional ``hidden'' (or ``latent'') random variable $U$, ranging in $\{1,\ldots,k\}$, and a directed edge from $U$ to every other vertex. Models of this type are fundamental to causal inference, where $U$ models an unobserved confounding effect of multiple populations, obscuring the causal relationships in the observable DAG. By solving the mixture problem and recovering the joint probability distribution with $U$, traditionally unidentifiable causal relationships become identifiable. Using a reduction to the more well-studied ``product'' case on empty graphs, we give the first algorithm to learn mixtures of non-empty DAGs.
Impact of class imbalance on chest x-ray classifiers: towards better evaluation practices for discrimination and calibration performance
Authors: Candelaria Mosquera, Luciana Ferrer, Diego Milone, Daniel Luna, Enzo Ferrante
Date: 2021-12-23 00:00:00

This work aims to analyze standard evaluation practices adopted by the research community when assessing chest x-ray classifiers, particularly focusing on the impact of class imbalance in such appraisals. Our analysis considers a comprehensive definition of model performance, covering not only discriminative performance but also model calibration, a topic of research that has received increasing attention during the last years within the machine learning community. Firstly, we conducted a literature study to analyze common scientific practices and confirmed that: (1) even when dealing with highly imbalanced datasets, the community tends to use metrics that are dominated by the majority class; and (2) it is still uncommon to include calibration studies for chest x-ray classifiers, albeit its importance in the context of healthcare. Secondly, we perform a systematic experiment on two major chest x-ray datasets to explore the behavior of several performance metrics under different class ratios and show that widely adopted metrics can conceal the performance in the minority class. Finally, we recommend the inclusion of complementary metrics to better reflect the system's performance in such scenarios. Our study indicates that current evaluation practices adopted by the research community for chest x-ray computer-aided diagnosis systems may not reflect their performance in real clinical scenarios, and suggest alternatives to improve this situation.
Faster Deep Ensemble Averaging for Quantification of DNA Damage from Comet Assay Images With Uncertainty Estimates
Authors: Srikanth Namuduri, Prateek Mehta, Lise Barbe, Stephanie Lam, Zohreh Faghihmonzavi, Steve Finkbeiner, Shekhar Bhansali
Date: 2021-12-23 00:00:00
Tasks: Transfer Learning

Several neurodegenerative diseases involve the accumulation of cellular DNA damage. Comet assays are a popular way of estimating the extent of DNA damage. Current literature on the use of deep learning to quantify DNA damage presents an empirical approach to hyper-parameter optimization and does not include uncertainty estimates. Deep ensemble averaging is a standard approach to estimating uncertainty but it requires several iterations of network training, which makes it time-consuming. Here we present an approach to quantify the extent of DNA damage that combines deep learning with a rigorous and comprehensive method to optimize the hyper-parameters with the help of statistical tests. We also use an architecture that allows for a faster computation of deep ensemble averaging and performs statistical tests applicable to networks using transfer learning. We applied our approach to a comet assay dataset with more than 1300 images and achieved an $R^2$ of 0.84, where the output included the confidence interval for each prediction. The proposed architecture is an improvement over the current approaches since it speeds up the uncertainty estimation by 30X while being statistically more rigorous.
Self-Attention Generative Adversarial Network for Iterative Reconstruction of CT Images
Authors: Ruiwen Xing, Thomas Humphries, Dong Si
Date: 2021-12-23 00:00:00
Tasks: Computed Tomography (CT), Denoising, Generative Adversarial Network, Image Reconstruction

Computed tomography (CT) uses X-ray measurements taken from sensors around the body to generate tomographic images of the human body. Conventional reconstruction algorithms can be used if the X-ray data are adequately sampled and of high quality; however, concerns such as reducing dose to the patient, or geometric limitations on data acquisition, may result in low quality or incomplete data. Images reconstructed from these data using conventional methods are of poor quality, due to noise and other artifacts. The aim of this study is to train a single neural network to reconstruct high-quality CT images from noisy or incomplete CT scan data, including low-dose, sparse-view, and limited-angle scenarios. To accomplish this task, we train a generative adversarial network (GAN) as a signal prior, to be used in conjunction with the iterative simultaneous algebraic reconstruction technique (SART) for CT data. The network includes a self-attention block to model long-range dependencies in the data. We compare our Self-Attention GAN for CT image reconstruction with several state-of-the-art approaches, including denoising cycle GAN, CIRCLE GAN, and a total variation superiorized algorithm. Our approach is shown to have comparable overall performance to CIRCLE GAN, while outperforming the other two approaches.
Radiomic biomarker extracted from PI-RADS 3 patients support more eìcient and robust prostate cancer diagnosis: a multi-center study
Authors: Longfei Li, Rui Yang, Xin Chen, Cheng Li, Hairong Zheng, Yusong Lin, Zaiyi Liu, Shanshan Wang
Date: 2021-12-23 00:00:00

Prostate Imaging Reporting and Data System (PI-RADS) based on multi-parametric MRI classi\^ees patients into 5 categories (PI-RADS 1-5) for routine clinical diagnosis guidance. However, there is no consensus on whether PI-RADS 3 patients should go through biopsies. Mining features from these hard samples (HS) is meaningful for physicians to achieve accurate diagnoses. Currently, the mining of HS biomarkers is insu\`icient, and the e\'eectiveness and robustness of HS biomarkers for prostate cancer diagnosis have not been explored. In this study, biomarkers from di\'eerent data distributions are constructed. Results show that HS biomarkers can achieve better performances in di\'eerent data distributions.
Weisfeiler and Leman go Machine Learning: The Story so far
Authors: Christopher Morris, Yaron Lipman, Haggai Maron, Bastian Rieck, Nils M. Kriege, Martin Grohe, Matthias Fey, Karsten Borgwardt
Date: 2021-12-18 00:00:00
Tasks: BIG-bench Machine Learning, Representation Learning

In recent years, algorithms and neural architectures based on the Weisfeiler--Leman algorithm, a well-known heuristic for the graph isomorphism problem, have emerged as a powerful tool for machine learning with graphs and relational data. Here, we give a comprehensive overview of the algorithm's use in a machine-learning setting, focusing on the supervised regime. We discuss the theoretical background, show how to use it for supervised graph and node representation learning, discuss recent extensions, and outline the algorithm's connection to (permutation-)equivariant neural architectures. Moreover, we give an overview of current applications and future directions to stimulate further research.
Cloud Removal from Satellite Images
Authors: Rutvik Chauhan, Antarpuneet Singh, Sujoy Saha
Date: 2021-12-23 00:00:00
Tasks: Cloud Detection, Cloud Removal, Generative Adversarial Network

In this report, we have analyzed available cloud detection technique using sentinel hub. We have also implemented spatial attention generative adversarial network and improved quality of generated image compared to previous solution [7].
Tight query complexity bounds for learning graph partitions
Authors: Xizhi Liu, Sayan Mukherjee
Date: 2021-12-15 00:00:00

Given a partition of a graph into connected components, the membership oracle asserts whether any two vertices of the graph lie in the same component or not. We prove that for $n\ge k\ge 2$, learning the components of an $n$-vertex hidden graph with $k$ components requires at least $(k-1)n-\binom k2$ membership queries. Our result improves on the best known information-theoretic bound of $\Omega(n\log k)$ queries, and exactly matches the query complexity of the algorithm introduced by [Reyzin and Srivastava, 2007] for this problem. Additionally, we introduce an oracle, with access to which one can learn the number of components of $G$ in asymptotically fewer queries than learning the full partition, thus answering another question posed by the same authors. Lastly, we introduce a more applicable version of this oracle, and prove asymptotically tight bounds of $\widetilde\Theta(m)$ queries for both learning and verifying an $m$-edge hidden graph $G$ using it.
Combating Collusion Rings is Hard but Possible
Authors: Niclas Boehmer, Robert Bredereck, André Nichterlein
Date: 2021-12-14 00:00:00

A recent report of Littmann [Commun. ACM '21] outlines the existence and the fatal impact of collusion rings in academic peer reviewing. We introduce and analyze the problem Cycle-Free Reviewing that aims at finding a review assignment without the following kind of collusion ring: A sequence of reviewers each reviewing a paper authored by the next reviewer in the sequence (with the last reviewer reviewing a paper of the first), thus creating a review cycle where each reviewer gives favorable reviews. As a result, all papers in that cycle have a high chance of acceptance independent of their respective scientific merit. We observe that review assignments computed using a standard Linear Programming approach typically admit many short review cycles. On the negative side, we show that Cycle-Free Reviewing is NP-hard in various restricted cases (i.e., when every author is qualified to review all papers and one wants to prevent that authors review each other's or their own papers or when every author has only one paper and is only qualified to review few papers). On the positive side, among others, we show that, in some realistic settings, an assignment without any review cycles of small length always exists. This result also gives rise to an efficient heuristic for computing (weighted) cycle-free review assignments, which we show to be of excellent quality in practice.
Revisiting PGD Attacks for Stability Analysis of Large-Scale Nonlinear Systems and Perception-Based Control
Authors: Aaron Havens, Darioush Keivan, Peter Seiler, Geir Dullerud, Bin Hu
Date: 2022-01-03 00:00:00

Many existing region-of-attraction (ROA) analysis tools find difficulty in addressing feedback systems with large-scale neural network (NN) policies and/or high-dimensional sensing modalities such as cameras. In this paper, we tailor the projected gradient descent (PGD) attack method developed in the adversarial learning community as a general-purpose ROA analysis tool for large-scale nonlinear systems and end-to-end perception-based control. We show that the ROA analysis can be approximated as a constrained maximization problem whose goal is to find the worst-case initial condition which shifts the terminal state the most. Then we present two PGD-based iterative methods which can be used to solve the resultant constrained maximization problem. Our analysis is not based on Lyapunov theory, and hence requires minimum information of the problem structures. In the model-based setting, we show that the PGD updates can be efficiently performed using back-propagation. In the model-free setting (which is more relevant to ROA analysis of perception-based control), we propose a finite-difference PGD estimate which is general and only requires a black-box simulator for generating the trajectories of the closed-loop system given any initial state. We demonstrate the scalability and generality of our analysis tool on several numerical examples with large-scale NN policies and high-dimensional image observations. We believe that our proposed analysis serves as a meaningful initial step toward further understanding of closed-loop stability of large-scale nonlinear systems and perception-based control.
Descriptors for Machine Learning Model of Generalized Force Field in Condensed Matter Systems
Authors: Puhan Zhang, Sheng Zhang, Gia-Wei Chern
Date: 2022-01-03 00:00:00
Tasks: BIG-bench Machine Learning

We outline the general framework of machine learning (ML) methods for multi-scale dynamical modeling of condensed matter systems, and in particular of strongly correlated electron models. Complex spatial temporal behaviors in these systems often arise from the interplay between quasi-particles and the emergent dynamical classical degrees of freedom, such as local lattice distortions, spins, and order-parameters. Central to the proposed framework is the ML energy model that, by successfully emulating the time-consuming electronic structure calculation, can accurately predict a local energy based on the classical field in the intermediate neighborhood. In order to properly include the symmetry of the electron Hamiltonian, a crucial component of the ML energy model is the descriptor that transforms the neighborhood configuration into invariant feature variables, which are input to the learning model. A general theory of the descriptor for the classical fields is formulated, and two types of models are distinguished depending on the presence or absence of an internal symmetry for the classical field. Several specific approaches to the descriptor of the classical fields are presented. Our focus is on the group-theoretical method that offers a systematic and rigorous approach to compute invariants based on the bispectrum coefficients. We propose an efficient implementation of the bispectrum method based on the concept of reference irreducible representations. Finally, the implementations of the various descriptors are demonstrated on well-known electronic lattice models.
Class-Incremental Continual Learning into the eXtended DER-verse
Authors: Matteo Boschini, Lorenzo Bonicelli, Pietro Buzzega, Angelo Porrello, Simone Calderara
Date: 2022-01-03 00:00:00
Tasks: Continual Learning, Knowledge Distillation

The staple of human intelligence is the capability of acquiring knowledge in a continuous fashion. In stark contrast, Deep Networks forget catastrophically and, for this reason, the sub-field of Class-Incremental Continual Learning fosters methods that learn a sequence of tasks incrementally, blending sequentially-gained knowledge into a comprehensive prediction. This work aims at assessing and overcoming the pitfalls of our previous proposal Dark Experience Replay (DER), a simple and effective approach that combines rehearsal and Knowledge Distillation. Inspired by the way our minds constantly rewrite past recollections and set expectations for the future, we endow our model with the abilities to i) revise its replay memory to welcome novel information regarding past data ii) pave the way for learning yet unseen classes. We show that the application of these strategies leads to remarkable improvements; indeed, the resulting method - termed eXtended-DER (X-DER) - outperforms the state of the art on both standard benchmarks (such as CIFAR-100 and miniImagenet) and a novel one here introduced. To gain a better understanding, we further provide extensive ablation studies that corroborate and extend the findings of our previous research (e.g. the value of Knowledge Distillation and flatter minima in continual learning setups).
Faster Unbalanced Optimal Transport: Translation invariant Sinkhorn and 1-D Frank-Wolfe
Authors: Thibault Séjourné, François-Xavier Vialard, Gabriel Peyré
Date: 2022-01-03 00:00:00
Tasks: Translation

Unbalanced optimal transport (UOT) extends optimal transport (OT) to take into account mass variations to compare distributions. This is crucial to make OT successful in ML applications, making it robust to data normalization and outliers. The baseline algorithm is Sinkhorn, but its convergence speed might be significantly slower for UOT than for OT. In this work, we identify the cause for this deficiency, namely the lack of a global normalization of the iterates, which equivalently corresponds to a translation of the dual OT potentials. Our first contribution leverages this idea to develop a provably accelerated Sinkhorn algorithm (coined 'translation invariant Sinkhorn') for UOT, bridging the computational gap with OT. Our second contribution focusses on 1-D UOT and proposes a Frank-Wolfe solver applied to this translation invariant formulation. The linear oracle of each steps amounts to solving a 1-D OT problems, resulting in a linear time complexity per iteration. Our last contribution extends this method to the computation of UOT barycenter of 1-D measures. Numerical simulations showcase the convergence speed improvement brought by these three approaches.
A Unified Analysis Method for Online Optimization in Normed Vector Space
Authors: Qing-xin Meng, Jian-wei Liu
Date: 2021-12-22 00:00:00

This paper studies online optimization from a high-level unified theoretical perspective. We not only generalize both Optimistic-DA and Optimistic-MD in normed vector space, but also unify their analysis methods for dynamic regret. Regret bounds are the tightest possible due to the introduction of $\phi$-convex. As instantiations, regret bounds of normalized exponentiated subgradient and greedy/lazy projection are better than the currently known optimal results. By replacing losses of online game with monotone operators, and extending the definition of regret, namely regret$^n$, we extend online convex optimization to online monotone optimization.
Application of Machine Learning Methods in Inferring Surface Water Groundwater Exchanges using High Temporal Resolution Temperature Measurements
Authors: Mohammad A. Moghaddam, Ty P. A. Ferre, Xingyuan Chen, Kewei Chen, Mohammad Reza Ehsani
Date: 2022-01-03 00:00:00

We examine the ability of machine learning (ML) and deep learning (DL) algorithms to infer surface/ground exchange flux based on subsurface temperature observations. The observations and fluxes are produced from a high-resolution numerical model representing conditions in the Columbia River near the Department of Energy Hanford site located in southeastern Washington State. Random measurement error, of varying magnitude, is added to the synthetic temperature observations. The results indicate that both ML and DL methods can be used to infer the surface/ground exchange flux. DL methods, especially convolutional neural networks, outperform the ML methods when used to interpret noisy temperature data with a smoothing filter applied. However, the ML methods also performed well and they are can better identify a reduced number of important observations, which could be useful for measurement network optimization. Surprisingly, the ML and DL methods better inferred upward flux than downward flux. This is in direct contrast to previous findings using numerical models to infer flux from temperature observations and it may suggest that combined use of ML or DL inference with numerical inference could improve flux estimation beneath river systems.
A Mixed-Integer Programming Approach to Training Dense Neural Networks
Authors: Vrishabh Patil, Yonatan Mintz
Date: 2022-01-03 00:00:00

Artificial Neural Networks (ANNs) are prevalent machine learning models that are applied across various real-world classification tasks. However, training ANNs is time-consuming and the resulting models take a lot of memory to deploy. In order to train more parsimonious ANNs, we propose a novel mixed-integer programming (MIP) formulation for training fully-connected ANNs. Our formulations can account for both binary and rectified linear unit (ReLU) activations, and for the use of a log-likelihood loss. We present numerical experiments comparing our MIP-based methods against existing approaches and show that we are able to achieve competitive out-of-sample performance with more parsimonious models.
A Cluster-Based Trip Prediction Graph Neural Network Model for Bike Sharing Systems
Authors: Bárbara Tavares, Cláudia Soares, Manuel Marques
Date: 2022-01-03 00:00:00
Tasks: Clustering, Graph Neural Network, Link Prediction, Traffic Prediction

Bike Sharing Systems (BSSs) are emerging as an innovative transportation service. Ensuring the proper functioning of a BSS is crucial given that these systems are committed to eradicating many of the current global concerns, by promoting environmental and economic sustainability and contributing to improving the life quality of the population. Good knowledge of users' transition patterns is a decisive contribution to the quality and operability of the service. The analogous and unbalanced users' transition patterns cause these systems to suffer from bicycle imbalance, leading to a drastic customer loss in the long term. Strategies for bicycle rebalancing become important to tackle this problem and for this, bicycle traffic prediction is essential, as it allows to operate more efficiently and to react in advance. In this work, we propose a bicycle trips predictor based on Graph Neural Network embeddings, taking into consideration station groupings, meteorology conditions, geographical distances, and trip patterns. We evaluated our approach in the New York City BSS (CitiBike) data and compared it with four baselines, including the non-clustered approach. To address our problem's specificities, we developed the Adaptive Transition Constraint Clustering Plus (AdaTC+) algorithm, eliminating shortcomings of previous work. Our experiments evidence the clustering pertinence (88% accuracy compared with 83% without clustering) and which clustering technique best suits this problem. Accuracy on the Link Prediction task is always higher for AdaTC+ than benchmark clustering methods when the stations are the same, while not degrading performance when the network is upgraded, in a mismatch with the trained model.