pwc-archive/papers-with-abstracts huggingface dataset:
Gradient flows and randomised thresholding: sparse inversion and classification
Authors: Jonas Latz
Date: 2022-03-22 00:00:00
Tasks: Classification

Sparse inversion and classification problems are ubiquitous in modern data science and imaging. They are often formulated as non-smooth minimisation problems. In sparse inversion, we minimise, e.g., the sum of a data fidelity term and an L1/LASSO regulariser. In classification, we consider, e.g., the sum of a data fidelity term and a non-smooth Ginzburg--Landau energy. Standard (sub)gradient descent methods have shown to be inefficient when approaching such problems. Splitting techniques are much more useful: here, the target function is partitioned into a sum of two subtarget functions -- each of which can be efficiently optimised. Splitting proceeds by performing optimisation steps alternately with respect to each of the two subtarget functions. In this work, we study splitting from a stochastic continuous-time perspective. Indeed, we define a differential inclusion that follows one of the two subtarget function's negative subdifferential at each point in time. The choice of the subtarget function is controlled by a binary continuous-time Markov process. The resulting dynamical system is a stochastic approximation of the underlying subgradient flow. We investigate this stochastic approximation for an L1-regularised sparse inversion flow and for a discrete Allen-Cahn equation minimising a Ginzburg--Landau energy. In both cases, we study the longtime behaviour of the stochastic dynamical system and its ability to approximate the underlying subgradient flow at any accuracy. We illustrate our theoretical findings in a simple sparse estimation problem and also in low- and high-dimensional classification problems.
Hindsight is 20/20: Leveraging Past Traversals to Aid 3D Perception
Authors: Yurong You, Katie Z Luo, Xiangyu Chen, Junan Chen, Wei-Lun Chao, Wen Sun, Bharath Hariharan, Mark Campbell, Kilian Q. Weinberger
Date: 2022-03-22 00:00:00
Proceeding: ICLR 2022 4
Tasks: 3D Object Detection, Autonomous Driving, object-detection, Object Detection, Self-Driving Cars

Self-driving cars must detect vehicles, pedestrians, and other traffic participants accurately to operate safely. Small, far-away, or highly occluded objects are particularly challenging because there is limited information in the LiDAR point clouds for detecting them. To address this challenge, we leverage valuable information from the past: in particular, data collected in past traversals of the same scene. We posit that these past data, which are typically discarded, provide rich contextual information for disambiguating the above-mentioned challenging cases. To this end, we propose a novel, end-to-end trainable Hindsight framework to extract this contextual information from past traversals and store it in an easy-to-query data structure, which can then be leveraged to aid future 3D object detection of the same scene. We show that this framework is compatible with most modern 3D detection architectures and can substantially improve their average precision on multiple autonomous driving datasets, most notably by more than 300% on the challenging cases.
A Binary Characterization Method for Shape Convexity and Applications
Authors: Shousheng Luo, Jinfeng Chen, Yunhai Xiao, Xue-Cheng Tai
Date: 2022-03-22 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

Convexity prior is one of the main cue for human vision and shape completion with important applications in image processing, computer vision. This paper focuses on characterization methods for convex objects and applications in image processing. We present a new method for convex objects representations using binary functions, that is, the convexity of a region is equivalent to a simple quadratic inequality constraint on its indicator function. Models are proposed firstly by incorporating this result for image segmentation with convexity prior and convex hull computation of a given set with and without noises. Then, these models are summarized to a general optimization problem on binary function(s) with the quadratic inequality. Numerical algorithm is proposed based on linearization technique, where the linearized problem is solved by a proximal alternating direction method of multipliers with guaranteed convergent. Numerical experiments demonstrate the efficiency and effectiveness of the proposed methods for image segmentation and convex hull computation in accuracy and computing time.
Audio visual character profiles for detecting background characters in entertainment media
Authors: Rahul Sharma, Shrikanth Narayanan
Date: 2022-03-21 00:00:00
Tasks: Active Speaker Localization, Face Verification

An essential goal of computational media intelligence is to support understanding how media stories -- be it news, commercial or entertainment media -- represent and reflect society and these portrayals are perceived. People are a central element of media stories. This paper focuses on understanding the representation and depiction of background characters in media depictions, primarily movies and TV shows. We define the background characters as those who do not participate vocally in any scene throughout the movie and address the problem of localizing background characters in videos. We use an active speaker localization system to extract high-confidence face-speech associations and generate audio-visual profiles for talking characters in a movie by automatically clustering them. Using a face verification system, we then prune all the face-tracks which match any of the generated character profiles and obtain the background character face-tracks. We curate a background character dataset which provides annotations for background character for a set of TV shows, and use it to evaluate the performance of the background character detection framework.
Segmenting Medical Instruments in Minimally Invasive Surgeries using AttentionMask
Authors: Christian Wilms, Alexander Michael Gerlach, Rüdiger Schmitz, Simone Frintrop
Date: 2022-03-21 00:00:00
Tasks: Object, Object Proposal Generation, Segmentation

Precisely locating and segmenting medical instruments in images of minimally invasive surgeries, medical instrument segmentation, is an essential first step for several tasks in medical image processing. However, image degradations, small instruments, and the generalization between different surgery types make medical instrument segmentation challenging. To cope with these challenges, we adapt the object proposal generation system AttentionMask and propose a dedicated post-processing to select promising proposals. The results on the ROBUST-MIS Challenge 2019 show that our adapted AttentionMask system is a strong foundation for generating state-of-the-art performance. Our evaluation in an object proposal generation framework shows that our adapted AttentionMask system is robust to image degradations, generalizes well to unseen types of surgeries, and copes well with small instruments.
Global Matching with Overlapping Attention for Optical Flow Estimation
Authors: Shiyu Zhao, Long Zhao, Zhixing Zhang, Enyu Zhou, Dimitris Metaxas
Date: 2022-03-21 00:00:00
Proceeding: CVPR 2022 1
Tasks: Optical Flow Estimation, regression

Optical flow estimation is a fundamental task in computer vision. Recent direct-regression methods using deep neural networks achieve remarkable performance improvement. However, they do not explicitly capture long-term motion correspondences and thus cannot handle large motions effectively. In this paper, inspired by the traditional matching-optimization methods where matching is introduced to handle large displacements before energy-based optimizations, we introduce a simple but effective global matching step before the direct regression and develop a learning-based matching-optimization framework, namely GMFlowNet. In GMFlowNet, global matching is efficiently calculated by applying argmax on 4D cost volumes. Additionally, to improve the matching quality, we propose patch-based overlapping attention to extract large context features. Extensive experiments demonstrate that GMFlowNet outperforms RAFT, the most popular optimization-only method, by a large margin and achieves state-of-the-art performance on standard benchmarks. Thanks to the matching and overlapping attention, GMFlowNet obtains major improvements on the predictions for textureless regions and large motions. Our code is made publicly available at https://github.com/xiaofeng94/GMFlowNet
Training Quantised Neural Networks with STE Variants: the Additive Noise Annealing Algorithm
Authors: Matteo Spallanzani, Gian Paolo Leonardi, Luca Benini
Date: 2022-03-21 00:00:00
Proceeding: CVPR 2022 1
Tasks: image-classification, Image Classification

Training quantised neural networks (QNNs) is a non-differentiable optimisation problem since weights and features are output by piecewise constant functions. The standard solution is to apply the straight-through estimator (STE), using different functions during the inference and gradient computation steps. Several STE variants have been proposed in the literature aiming to maximise the task accuracy of the trained network. In this paper, we analyse STE variants and study their impact on QNN training. We first observe that most such variants can be modelled as stochastic regularisations of stair functions; although this intuitive interpretation is not new, our rigorous discussion generalises to further variants. Then, we analyse QNNs mixing different regularisations, finding that some suitably synchronised smoothing of each layer map is required to guarantee pointwise compositional convergence to the target discontinuous function. Based on these theoretical insights, we propose additive noise annealing (ANA), a new algorithm to train QNNs encompassing standard STE and its variants as special cases. When testing ANA on the CIFAR-10 image classification benchmark, we find that the major impact on task accuracy is not due to the qualitative shape of the regularisations but to the proper synchronisation of the different STE variants used in a network, in accordance with the theoretical results.
Generative Adversarial Network for Future Hand Segmentation from Egocentric Video
Authors: Wenqi Jia, Miao Liu, James M. Rehg
Date: 2022-03-21 00:00:00
Tasks: Generative Adversarial Network, Hand Segmentation, Image Segmentation, Semantic Segmentation, Time Series, Time Series Analysis

We introduce the novel problem of anticipating a time series of future hand masks from egocentric video. A key challenge is to model the stochasticity of future head motions, which globally impact the head-worn camera video analysis. To this end, we propose a novel deep generative model -- EgoGAN, which uses a 3D Fully Convolutional Network to learn a spatio-temporal video representation for pixel-wise visual anticipation, generates future head motion using Generative Adversarial Network (GAN), and then predicts the future hand masks based on the video representation and the generated future head motion. We evaluate our method on both the EPIC-Kitchens and the EGTEA Gaze+ datasets. We conduct detailed ablation studies to validate the design choices of our approach. Furthermore, we compare our method with previous state-of-the-art methods on future image segmentation and show that our method can more accurately predict future hand masks.
A Contrastive Objective for Learning Disentangled Representations
Authors: Jonathan Kahana, Yedid Hoshen
Date: 2022-03-21 00:00:00
Tasks: Attribute, Informativeness, Retrieval

Learning representations of images that are invariant to sensitive or unwanted attributes is important for many tasks including bias removal and cross domain retrieval. Here, our objective is to learn representations that are invariant to the domain (sensitive attribute) for which labels are provided, while being informative over all other image attributes, which are unlabeled. We present a new approach, proposing a new domain-wise contrastive objective for ensuring invariant representations. This objective crucially restricts negative image pairs to be drawn from the same domain, which enforces domain invariance whereas the standard contrastive objective does not. This domain-wise objective is insufficient on its own as it suffers from shortcut solutions resulting in feature suppression. We overcome this issue by a combination of a reconstruction constraint, image augmentations and initialization with pre-trained weights. Our analysis shows that the choice of augmentations is important, and that a misguided choice of augmentations can harm the invariance and informativeness objectives. In an extensive evaluation, our method convincingly outperforms the state-of-the-art in terms of representation invariance, representation informativeness, and training speed. Furthermore, we find that in some cases our method can achieve excellent results even without the reconstruction constraint, leading to a much faster and resource efficient training.
NeRFusion: Fusing Radiance Fields for Large-Scale Scene Reconstruction
Authors: Xiaoshuai Zhang, Sai Bi, Kalyan Sunkavalli, Hao Su, Zexiang Xu
Date: 2022-03-21 00:00:00
Proceeding: CVPR 2022 1
Tasks: 3D Reconstruction, NeRF

While NeRF has shown great success for neural reconstruction and rendering, its limited MLP capacity and long per-scene optimization times make it challenging to model large-scale indoor scenes. In contrast, classical 3D reconstruction methods can handle large-scale scenes but do not produce realistic renderings. We propose NeRFusion, a method that combines the advantages of NeRF and TSDF-based fusion techniques to achieve efficient large-scale reconstruction and photo-realistic rendering. We process the input image sequence to predict per-frame local radiance fields via direct network inference. These are then fused using a novel recurrent neural network that incrementally reconstructs a global, sparse scene representation in real-time at 22 fps. This global volume can be further fine-tuned to boost rendering quality. We demonstrate that NeRFusion achieves state-of-the-art quality on both large-scale indoor and small-scale object scenes, with substantially faster reconstruction than NeRF and other recent methods.
ImageNet Challenging Classification with the Raspberry Pi: An Incremental Local Stochastic Gradient Descent Algorithm
Authors: Thanh-Nghi Do
Date: 2022-03-21 00:00:00
Tasks: CPU, Edge-computing, Raspberry Pi 4

With rising powerful, low-cost embedded devices, the edge computing has become an increasingly popular choice. In this paper, we propose a new incremental local stochastic gradient descent (SGD) tailored on the Raspberry Pi to deal with large ImageNet ILSVRC 2010 dataset having 1,261,405 images with 1,000 classes. The local SGD splits the data block into $k$ partitions using $k$means algorithm and then it learns in the parallel way SGD models in each data partition to classify the data locally. The incremental local SGD sequentially loads small data blocks of the training dataset to learn local SGD models. The numerical test results on Imagenet dataset show that our incremental local SGD algorithm with the Raspberry Pi 4 is faster and more accurate than the state-of-the-art linear SVM run on a PC Intel(R) Core i7-4790 CPU, 3.6 GHz, 4 cores.
MetaMorph: Learning Universal Controllers with Transformers
Authors: Agrim Gupta, Linxi Fan, Surya Ganguli, Li Fei-Fei
Date: 2022-03-22 00:00:00
Proceeding: ICLR 2022 4
Tasks: Zero-shot Generalization

Multiple domains like vision, natural language, and audio are witnessing tremendous progress by leveraging Transformers for large scale pre-training followed by task specific fine tuning. In contrast, in robotics we primarily train a single robot for a single task. However, modular robot systems now allow for the flexible combination of general-purpose building blocks into task optimized morphologies. However, given the exponentially large number of possible robot morphologies, training a controller for each new design is impractical. In this work, we propose MetaMorph, a Transformer based approach to learn a universal controller over a modular robot design space. MetaMorph is based on the insight that robot morphology is just another modality on which we can condition the output of a Transformer. Through extensive experiments we demonstrate that large scale pre-training on a variety of robot morphologies results in policies with combinatorial generalization capabilities, including zero shot generalization to unseen robot morphologies. We further demonstrate that our pre-trained policy can be used for sample-efficient transfer to completely new robot morphologies and tasks.
On Supervised Feature Selection from High Dimensional Feature Spaces
Authors: Yijing Yang, Wei Wang, Hongyu Fu, C. -C. Jay Kuo
Date: 2022-03-22 00:00:00
Tasks: BIG-bench Machine Learning, feature selection, Vocal Bursts Intensity Prediction

The application of machine learning to image and video data often yields a high dimensional feature space. Effective feature selection techniques identify a discriminant feature subspace that lowers computational and modeling costs with little performance degradation. A novel supervised feature selection methodology is proposed for machine learning decisions in this work. The resulting tests are called the discriminant feature test (DFT) and the relevant feature test (RFT) for the classification and regression problems, respectively. The DFT and RFT procedures are described in detail. Furthermore, we compare the effectiveness of DFT and RFT with several classic feature selection methods. To this end, we use deep features obtained by LeNet-5 for MNIST and Fashion-MNIST datasets as illustrative examples. Other datasets with handcrafted and gene expressions features are also included for performance evaluation. It is shown by experimental results that DFT and RFT can select a lower dimensional feature subspace distinctly and robustly while maintaining high decision performance.
SPRITE: A Scalable Privacy-Preserving and Verifiable Collaborative Learning for Industrial IoT
Authors: Jayasree Sengupta, Sushmita Ruj, Sipra Das Bit
Date: 2022-03-22 00:00:00
Tasks: Privacy Preserving

Recently collaborative learning is widely applied to model sensitive data generated in Industrial IoT (IIoT). It enables a large number of devices to collectively train a global model by collaborating with a server while keeping the datasets on their respective premises. However, existing approaches are limited by high overheads and may also suffer from falsified aggregated results returned by a malicious server. Hence, we propose a Scalable, Privacy-preserving and veRIfiable collaboraTive lEarning (SPRITE) algorithm to train linear and logistic regression models for IIoT. We aim to reduce burden from resource-constrained IIoT devices and trust dependence on cloud by introducing fog as a middleware. SPRITE employs threshold secret sharing to guarantee privacy-preservation and robustness to IIoT device dropout whereas verifiable additive homomorphic secret sharing to ensure verifiability during model aggregation. We prove the security of SPRITE in an honest-but-curious setting where the cloud is untrustworthy. We validate SPRITE to be scalable and lightweight through theoretical overhead analysis and extensive testbed experimentation on an IIoT use-case with two real-world industrial datasets. For a large-scale industrial setup, SPRITE records 65% and 55% improved performance over its competitor for linear and logistic regressions respectively while reducing communication overhead for an IIoT device by 90%.
On the (Non-)Robustness of Two-Layer Neural Networks in Different Learning Regimes
Authors: Elvis Dohmatob, Alberto Bietti
Date: 2022-03-22 00:00:00
Tasks: Adversarial Robustness

Neural networks are known to be highly sensitive to adversarial examples. These may arise due to different factors, such as random initialization, or spurious correlations in the learning problem. To better understand these factors, we provide a precise study of the adversarial robustness in different scenarios, from initialization to the end of training in different regimes, as well as intermediate scenarios, where initialization still plays a role due to "lazy" training. We consider over-parameterized networks in high dimensions with quadratic targets and infinite samples. Our analysis allows us to identify new tradeoffs between approximation (as measured via test error) and robustness, whereby robustness can only get worse when test error improves, and vice versa. We also show how linearized lazy training regimes can worsen robustness, due to improperly scaled random initialization. Our theoretical results are illustrated with numerical experiments.
Practical tradeoffs between memory, compute, and performance in learned optimizers
Authors: Luke Metz, C. Daniel Freeman, James Harrison, Niru Maheswaranathan, Jascha Sohl-Dickstein
Date: 2022-03-22 00:00:00

Optimization plays a costly and crucial role in developing machine learning systems. In learned optimizers, the few hyperparameters of commonly used hand-designed optimizers, e.g. Adam or SGD, are replaced with flexible parametric functions. The parameters of these functions are then optimized so that the resulting learned optimizer minimizes a target loss on a chosen class of models. Learned optimizers can both reduce the number of required training steps and improve the final test loss. However, they can be expensive to train, and once trained can be expensive to use due to computational and memory overhead for the optimizer itself. In this work, we identify and quantify the design features governing the memory, compute, and performance trade-offs for many learned and hand-designed optimizers. We further leverage our analysis to construct a learned optimizer that is both faster and more memory efficient than previous work. Our model and training code are open source.
Representation Bias in Data: A Survey on Identification and Resolution Techniques
Authors: Nima Shahbazi, Yin Lin, Abolfazl Asudeh, H. V. Jagadish
Date: 2022-03-22 00:00:00
Tasks: Decision Making, Fairness, Survey

Data-driven algorithms are only as good as the data they work with, while data sets, especially social data, often fail to represent minorities adequately. Representation Bias in data can happen due to various reasons ranging from historical discrimination to selection and sampling biases in the data acquisition and preparation methods. Given that "bias in, bias out", one cannot expect AI-based solutions to have equitable outcomes for societal applications, without addressing issues such as representation bias. While there has been extensive study of fairness in machine learning models, including several review papers, bias in the data has been less studied. This paper reviews the literature on identifying and resolving representation bias as a feature of a data set, independent of how consumed later. The scope of this survey is bounded to structured (tabular) and unstructured (e.g., image, text, graph) data. It presents taxonomies to categorize the studied techniques based on multiple design dimensions and provides a side-by-side comparison of their properties. There is still a long way to fully address representation bias issues in data. The authors hope that this survey motivates researchers to approach these challenges in the future by observing existing work within their respective domains.