pwc-archive/papers-with-abstracts huggingface dataset:
Learning Attention Propagation for Compositional Zero-Shot Learning
Authors: Muhammad Gul Zain Ali Khan, Muhammad Ferjad Naeem, Luc van Gool, Alain Pagani, Didier Stricker, Muhammad Zeshan Afzal
Date: 2022-10-20 00:00:00
Tasks: Compositional Zero-Shot Learning, Zero-Shot Learning

Compositional zero-shot learning aims to recognize unseen compositions of seen visual primitives of object classes and their states. While all primitives (states and objects) are observable during training in some combination, their complex interaction makes this task especially hard. For example, wet changes the visual appearance of a dog very differently from a bicycle. Furthermore, we argue that relationships between compositions go beyond shared states or objects. A cluttered office can contain a busy table; even though these compositions don't share a state or object, the presence of a busy table can guide the presence of a cluttered office. We propose a novel method called Compositional Attention Propagated Embedding (CAPE) as a solution. The key intuition to our method is that a rich dependency structure exists between compositions arising from complex interactions of primitives in addition to other dependencies between compositions. CAPE learns to identify this structure and propagates knowledge between them to learn class embedding for all seen and unseen compositions. In the challenging generalized compositional zero-shot setting, we show that our method outperforms previous baselines to set a new state-of-the-art on three publicly available benchmarks.
H4VDM: H.264 Video Device Matching
Authors: Ziyue Xiang, Paolo Bestagini, Stefano Tubaro, Edward J. Delp
Date: 2022-10-20 00:00:00
Tasks: Video Forensics

Methods that can determine if two given video sequences are captured by the same device (e.g., mobile telephone or digital camera) can be used in many forensics tasks. In this paper we refer to this as "video device matching". In open-set video forensics scenarios it is easier to determine if two video sequences were captured with the same device than identifying the specific device. In this paper, we propose a technique for open-set video device matching. Given two H.264 compressed video sequences, our method can determine if they are captured by the same device, even if our method has never encountered the device in training. We denote our proposed technique as H.264 Video Device Matching (H4VDM). H4VDM uses H.264 compression information extracted from video sequences to make decisions. It is more robust against artifacts that alter camera sensor fingerprints, and it can be used to analyze relatively small fragments of the H.264 sequence. We trained and tested our method on a publicly available video forensics dataset consisting of 35 devices, where our proposed method demonstrated good performance.
Transferring learned patterns from ground-based field imagery to predict UAV-based imagery for crop and weed semantic segmentation in precision crop farming
Authors: Junfeng Gao, Wenzhi Liao, David Nuyttens, Peter Lootens, Erik Alexandersson, Jan Pieters
Date: 2022-10-20 00:00:00
Tasks: Segmentation, Semantic Segmentation

Weed and crop segmentation is becoming an increasingly integral part of precision farming that leverages the current computer vision and deep learning technologies. Research has been extensively carried out based on images captured with a camera from various platforms. Unmanned aerial vehicles (UAVs) and ground-based vehicles including agricultural robots are the two popular platforms for data collection in fields. They all contribute to site-specific weed management (SSWM) to maintain crop yield. Currently, the data from these two platforms is processed separately, though sharing the same semantic objects (weed and crop). In our paper, we have developed a deep convolutional network that enables to predict both field and aerial images from UAVs for weed segmentation and mapping with only field images provided in the training phase. The network learning process is visualized by feature maps at shallow and deep layers. The results show that the mean intersection of union (IOU) values of the segmentation for the crop (maize), weeds, and soil background in the developed model for the field dataset are 0.744, 0.577, 0.979, respectively, and the performance of aerial images from an UAV with the same model, the IOU values of the segmentation for the crop (maize), weeds and soil background are 0.596, 0.407, and 0.875, respectively. To estimate the effect on the use of plant protection agents, we quantify the relationship between herbicide spraying saving rate and grid size (spraying resolution) based on the predicted weed map. The spraying saving rate is up to 90% when the spraying resolution is at 1.78 x 1.78 cm2. The study shows that the developed deep convolutional neural network could be used to classify weeds from both field and aerial images and delivers satisfactory results.
ConfMix: Unsupervised Domain Adaptation for Object Detection via Confidence-based Mixing
Authors: Giulio Mattolin, Luca Zanella, Elisa Ricci, Yiming Wang
Date: 2022-10-20 00:00:00
Tasks: Domain Adaptation, Object Detection, Unsupervised Domain Adaptation

Unsupervised Domain Adaptation (UDA) for object detection aims to adapt a model trained on a source domain to detect instances from a new target domain for which annotations are not available. Different from traditional approaches, we propose ConfMix, the first method that introduces a sample mixing strategy based on region-level detection confidence for adaptive object detector learning. We mix the local region of the target sample that corresponds to the most confident pseudo detections with a source image, and apply an additional consistency loss term to gradually adapt towards the target data distribution. In order to robustly define a confidence score for a region, we exploit the confidence score per pseudo detection that accounts for both the detector-dependent confidence and the bounding box uncertainty. Moreover, we propose a novel pseudo labelling scheme that progressively filters the pseudo target detections using the confidence metric that varies from a loose to strict manner along the training. We perform extensive experiments with three datasets, achieving state-of-the-art performance in two of them and approaching the supervised target model performance in the other. Code is available at: https://github.com/giuliomattolin/ConfMix.
Overexposure Mask Fusion: Generalizable Reverse ISP Multi-Step Refinement
Authors: Jinha Kim, Jun Jiang, Jinwei Gu
Date: 2022-10-20 00:00:00
Tasks: Raw reconstruction

With the advent of deep learning methods replacing the ISP in transforming sensor RAW readings into RGB images, numerous methodologies solidified into real-life applications. Equally potent is the task of inverting this process which will have applications in enhancing computational photography tasks that are conducted in the RAW domain, addressing lack of available RAW data while reaping from the benefits of performing tasks directly on sensor readings. This paper's proposed methodology is a state-of-the-art solution to the task of RAW reconstruction, and the multi-step refinement process integrating an overexposure mask is novel in three ways: instead of from RGB to bayer, the pipeline trains from RGB to demosaiced RAW allowing use of perceptual loss functions; the multi-step processes has greatly enhanced the performance of the baseline U-Net from start to end; the pipeline is a generalizable process of refinement that can enhance other high performance methodologies that support end-to-end learning.
End-to-End Context-Aided Unicity Matching for Person Re-identification
Authors: Min Cao, Cong Ding, Chen Chen, Junchi Yan, Qi Tian
Date: 2022-10-20 00:00:00
Tasks: Graph Matching, Person Re-Identification

Most existing person re-identification methods compute the matching relations between person images across camera views based on the ranking of the pairwise similarities. This matching strategy with the lack of the global viewpoint and the context's consideration inevitably leads to ambiguous matching results and sub-optimal performance. Based on a natural assumption that images belonging to the same person identity should not match with images belonging to multiple different person identities across views, called the unicity of person matching on the identity level, we propose an end-to-end person unicity matching architecture for learning and refining the person matching relations. First, we adopt the image samples' contextual information in feature space to generate the initial soft matching results by using graph neural networks. Secondly, we utilize the samples' global context relationship to refine the soft matching results and reach the matching unicity through bipartite graph matching. Given full consideration to real-world person re-identification applications, we achieve the unicity matching in both one-shot and multi-shot settings of person re-identification and further develop a fast version of the unicity matching without losing the performance. The proposed method is evaluated on five public benchmarks, including four multi-shot datasets MSMT17, DukeMTMC, Market1501, CUHK03, and a one-shot dataset VIPeR. Experimental results show the superiority of the proposed method on performance and efficiency.
JRDB-Pose: A Large-scale Dataset for Multi-Person Pose Estimation and Tracking
Authors: Edward Vendrow, Duy Tho Le, Jianfei Cai, Hamid Rezatofighi
Date: 2022-10-20 00:00:00
Proceeding: CVPR 2023 1
Tasks: Diversity, Multi-Person Pose Estimation, Multi-Person Pose Estimation and Tracking, Pose Estimation, Robot Navigation, Social Navigation

Autonomous robotic systems operating in human environments must understand their surroundings to make accurate and safe decisions. In crowded human scenes with close-up human-robot interaction and robot navigation, a deep understanding requires reasoning about human motion and body dynamics over time with human body pose estimation and tracking. However, existing datasets either do not provide pose annotations or include scene types unrelated to robotic applications. Many datasets also lack the diversity of poses and occlusions found in crowded human scenes. To address this limitation we introduce JRDB-Pose, a large-scale dataset and benchmark for multi-person pose estimation and tracking using videos captured from a social navigation robot. The dataset contains challenge scenes with crowded indoor and outdoor locations and a diverse range of scales and occlusion types. JRDB-Pose provides human pose annotations with per-keypoint occlusion labels and track IDs consistent across the scene. A public evaluation server is made available for fair evaluation on a held-out test set. JRDB-Pose is available at https://jrdb.erc.monash.edu/ .
Computer-Aided Cancer Diagnosis via Machine Learning and Deep Learning: A comparative review
Authors: Solene Bechelli
Date: 2022-10-19 00:00:00

The past years have seen a considerable increase in cancer cases. However, a cancer diagnosis is often complex and depends on the types of images provided for analysis. It requires highly skilled practitioners but is often time-consuming and error-prone. If Machine Learning and deep learning algorithms have been widely used, a comprehensive review of the techniques used from the pre-processing steps to the final prediction is lacking. With this review, we aim to provide a comprehensive overview of the current steps required in building efficient and accurate machine learning algorithm for cancer prediction, detection and classification. To do so, we compile the results of cancer related study using AI over the past years. We include various cancers that encompass different types of images, and therefore different related techniques. We show that tremendous improvements have been made in the early detection of cancerous tumors and tissues. The techniques used are various and often problem-tailored and our findings is confirmed through the study of a large number of research. Moreover, we investigate the approaches best suited for different types of images such as histology, dermoscopic, MRI, etc. With this work, we summarize the main finding over the past years in cancer detection using deep learning techniques. We discuss the challenges of cancer research related to the large discrepancies in the images, and we provide some notable results in the field for lung, breast, and skin cancers.
RT-MOT: Confidence-Aware Real-Time Scheduling Framework for Multi-Object Tracking Tasks
Authors: Donghwa Kang, Seunghoon Lee, Hoon Sung Chwa, Seung-Hwan Bae, Chang Mook Kang, Jinkyu Lee, Hyeongboo Baek
Date: 2022-10-19 00:00:00
Tasks: Autonomous Vehicles, Multi-Object Tracking, Object Tracking, Scheduling

Different from existing MOT (Multi-Object Tracking) techniques that usually aim at improving tracking accuracy and average FPS, real-time systems such as autonomous vehicles necessitate new requirements of MOT under limited computing resources: (R1) guarantee of timely execution and (R2) high tracking accuracy. In this paper, we propose RT-MOT, a novel system design for multiple MOT tasks, which addresses R1 and R2. Focusing on multiple choices of a workload pair of detection and association, which are two main components of the tracking-by-detection approach for MOT, we tailor a measure of object confidence for RT-MOT and develop how to estimate the measure for the next frame of each MOT task. By utilizing the estimation, we make it possible to predict tracking accuracy variation according to different workload pairs to be applied to the next frame of an MOT task. Next, we develop a novel confidence-aware real-time scheduling framework, which offers an offline timing guarantee for a set of MOT tasks based on non-preemptive fixed-priority scheduling with the smallest workload pair. At run-time, the framework checks the feasibility of a priority-inversion associated with a larger workload pair, which does not compromise the timing guarantee of every task, and then chooses a feasible scenario that yields the largest tracking accuracy improvement based on the proposed prediction. Our experiment results demonstrate that RT-MOT significantly improves overall tracking accuracy by up to 1.5x, compared to existing popular tracking-by-detection approaches, while guaranteeing timely execution of all MOT tasks.
A Multi-Scale Deep Learning Framework for Projecting Weather Extremes
Authors: Antoine Blanchard, Nishant Parashar, Boyko Dodov, Christian Lessig, Themistoklis Sapsis
Date: 2022-10-21 00:00:00
Tasks: Deep Learning

Weather extremes are a major societal and economic hazard, claiming thousands of lives and causing billions of dollars in damage every year. Under climate change, their impact and intensity are expected to worsen significantly. Unfortunately, general circulation models (GCMs), which are currently the primary tool for climate projections, cannot characterize weather extremes accurately. To address this, we present a multi-resolution deep-learning framework that, firstly, corrects a GCM's biases by matching low-order and tail statistics of its output with observations at coarse scales; and secondly, increases the level of detail of the debiased GCM output by reconstructing the finer scales as a function of the coarse scales. We use the proposed framework to generate statistically realistic realizations of the climate over Western Europe from a simple GCM corrected using observational atmospheric reanalysis. We also discuss implications for probabilistic risk assessment of natural disasters in a changing climate.
Graph Few-shot Learning with Task-specific Structures
Authors: Song Wang, Chen Chen, Jundong Li
Date: 2022-10-21 00:00:00
Tasks: Classification, Few-Shot Learning, Graph Learning, Node Classification

Graph few-shot learning is of great importance among various graph learning tasks. Under the few-shot scenario, models are often required to conduct classification given limited labeled samples. Existing graph few-shot learning methods typically leverage Graph Neural Networks (GNNs) and perform classification across a series of meta-tasks. Nevertheless, these methods generally rely on the original graph (i.e., the graph that the meta-task is sampled from) to learn node representations. Consequently, the graph structure used in each meta-task is identical. Since the class sets are different across meta-tasks, node representations should be learned in a task-specific manner to promote classification performance. Therefore, to adaptively learn node representations across meta-tasks, we propose a novel framework that learns a task-specific structure for each meta-task. To handle the variety of nodes across meta-tasks, we extract relevant nodes and learn task-specific structures based on node influence and mutual information. In this way, we can learn node representations with the task-specific structure tailored for each meta-task. We further conduct extensive experiments on five node classification datasets under both single- and multiple-graph settings to validate the superiority of our framework over the state-of-the-art baselines. Our code is provided at https://github.com/SongW-SW/GLITTER.
Equivariant Networks for Zero-Shot Coordination
Authors: Darius Muglich, Christian Schroeder de Witt, Elise van der Pol, Shimon Whiteson, Jakob Foerster
Date: 2022-10-21 00:00:00

Successful coordination in Dec-POMDPs requires agents to adopt robust strategies and interpretable styles of play for their partner. A common failure mode is symmetry breaking, when agents arbitrarily converge on one out of many equivalent but mutually incompatible policies. Commonly these examples include partial observability, e.g. waving your right hand vs. left hand to convey a covert message. In this paper, we present a novel equivariant network architecture for use in Dec-POMDPs that effectively leverages environmental symmetry for improving zero-shot coordination, doing so more effectively than prior methods. Our method also acts as a ``coordination-improvement operator'' for generic, pre-trained policies, and thus may be applied at test-time in conjunction with any self-play algorithm. We provide theoretical guarantees of our work and test on the AI benchmark task of Hanabi, where we demonstrate our methods outperforming other symmetry-aware baselines in zero-shot coordination, as well as able to improve the coordination ability of a variety of pre-trained policies. In particular, we show our method can be used to improve on the state of the art for zero-shot coordination on the Hanabi benchmark.
Targeted active learning for probabilistic models
Authors: Christopher Tosh, Mauricio Tec, Wesley Tansey
Date: 2022-10-21 00:00:00
Tasks: Active Learning

A fundamental task in science is to design experiments that yield valuable insights about the system under study. Mathematically, these insights can be represented as a utility or risk function that shapes the value of conducting each experiment. We present PDBAL, a targeted active learning method that adaptively designs experiments to maximize scientific utility. PDBAL takes a user-specified risk function and combines it with a probabilistic model of the experimental outcomes to choose designs that rapidly converge on a high-utility model. We prove theoretical bounds on the label complexity of PDBAL and provide fast closed-form solutions for designing experiments with common exponential family likelihoods. In simulation studies, PDBAL consistently outperforms standard untargeted approaches that focus on maximizing expected information gain over the design space. Finally, we demonstrate the scientific potential of PDBAL through a study on a large cancer drug screen dataset where PDBAL quickly recovers the most efficacious drugs with a small fraction of the total number of experiments.
Adversarial Permutation Invariant Training for Universal Sound Separation
Authors: Emilian Postolache, Jordi Pons, Santiago Pascual, Joan Serrà
Date: 2022-10-21 00:00:00

Universal sound separation consists of separating mixes with arbitrary sounds of different types, and permutation invariant training (PIT) is used to train source agnostic models that do so. In this work, we complement PIT with adversarial losses but find it challenging with the standard formulation used in speech source separation. We overcome this challenge with a novel I-replacement context-based adversarial loss, and by training with multiple discriminators. Our experiments show that by simply improving the loss (keeping the same model and dataset) we obtain a non-negligible improvement of 1.4 dB SI-SNRi in the reverberant FUSS dataset. We also find adversarial PIT to be effective at reducing spectral holes, ubiquitous in mask-based separation models, which highlights the potential relevance of adversarial losses for source separation.
XAI for transparent wind turbine power curve models
Authors: Simon Letzgus
Date: 2022-10-21 00:00:00
Tasks: Explainable artificial intelligence, Explainable Artificial Intelligence (XAI), Model Selection

Accurate wind turbine power curve models, which translate ambient conditions into turbine power output, are crucial for wind energy to scale and fulfill its proposed role in the global energy transition. While machine learning (ML) methods have shown significant advantages over parametric, physics-informed approaches, they are often criticised for being opaque 'black boxes', which hinders their application in practice. We apply Shapley values, a popular explainable artificial intelligence (XAI) method, and the latest findings from XAI for regression models, to uncover the strategies ML models have learned from operational wind turbine data. Our findings reveal that the trend towards ever larger model architectures, driven by a focus on test set performance, can result in physically implausible model strategies. Therefore, we call for a more prominent role of XAI methods in model selection. Moreover, we propose a practical approach to utilize explanations for root cause analysis in the context of wind turbine performance monitoring. This can help to reduce downtime and increase the utilization of turbines in the field.
Neural Network Approximations of PDEs Beyond Linearity: A Representational Perspective
Authors: Tanya Marwah, Zachary C. Lipton, Jianfeng Lu, Andrej Risteski
Date: 2022-10-21 00:00:00

A burgeoning line of research leverages deep neural networks to approximate the solutions to high dimensional PDEs, opening lines of theoretical inquiry focused on explaining how it is that these models appear to evade the curse of dimensionality. However, most prior theoretical analyses have been limited to linear PDEs. In this work, we take a step towards studying the representational power of neural networks for approximating solutions to nonlinear PDEs. We focus on a class of PDEs known as \emph{nonlinear elliptic variational PDEs}, whose solutions minimize an \emph{Euler-Lagrange} energy functional $\mathcal{E}(u) = \int_\Omega L(x, u(x), \nabla u(x)) - f(x) u(x)dx$. We show that if composing a function with Barron norm $b$ with partial derivatives of $L$ produces a function of Barron norm at most $B_L b^p$, the solution to the PDE can be $\epsilon$-approximated in the $L^2$ sense by a function with Barron norm $O\left(\left(dB_L\right)^{\max\{p \log(1/ \epsilon), p^{\log(1/\epsilon)}\}}\right)$. By a classical result due to Barron [1993], this correspondingly bounds the size of a 2-layer neural network needed to approximate the solution. Treating $p, \epsilon, B_L$ as constants, this quantity is polynomial in dimension, thus showing neural networks can evade the curse of dimensionality. Our proof technique involves neurally simulating (preconditioned) gradient in an appropriate Hilbert space, which converges exponentially fast to the solution of the PDE, and such that we can bound the increase of the Barron norm at each iterate. Our results subsume and substantially generalize analogous prior results for linear elliptic PDEs over a unit hypercube.