pwc-archive/papers-with-abstracts huggingface dataset:
Forgery-aware Adaptive Transformer for Generalizable Synthetic Image Detection
Authors: Huan Liu, Zichang Tan, Chuangchuang Tan, Yunchao Wei, Yao Zhao, Jingdong Wang
Date: 2023-12-27 00:00:00
Proceeding: CVPR 2024 1
Tasks: Attribute, Synthetic Image Detection

In this paper, we study the problem of generalizable synthetic image detection, aiming to detect forgery images from diverse generative methods, e.g., GANs and diffusion models. Cutting-edge solutions start to explore the benefits of pre-trained models, and mainly follow the fixed paradigm of solely training an attached classifier, e.g., combining frozen CLIP-ViT with a learnable linear layer in UniFD. However, our analysis shows that such a fixed paradigm is prone to yield detectors with insufficient learning regarding forgery representations. We attribute the key challenge to the lack of forgery adaptation, and present a novel forgery-aware adaptive transformer approach, namely FatFormer. Based on the pre-trained vision-language spaces of CLIP, FatFormer introduces two core designs for the adaption to build generalized forgery representations. First, motivated by the fact that both image and frequency analysis are essential for synthetic image detection, we develop a forgery-aware adapter to adapt image features to discern and integrate local forgery traces within image and frequency domains. Second, we find that considering the contrastive objectives between adapted image features and text prompt embeddings, a previously overlooked aspect, results in a nontrivial generalization improvement. Accordingly, we introduce language-guided alignment to supervise the forgery adaptation with image and text prompts in FatFormer. Experiments show that, by coupling these two designs, our approach tuned on 4-class ProGAN data attains a remarkable detection performance, achieving an average of 98% accuracy to unseen GANs, and surprisingly generalizes to unseen diffusion models with 95% accuracy.
I2V-Adapter: A General Image-to-Video Adapter for Diffusion Models
Authors: Xun Guo, Mingwu Zheng, Liang Hou, Yuan Gao, Yufan Deng, Pengfei Wan, Di Zhang, Yufan Liu, Weiming Hu, ZhengJun Zha, Haibin Huang, Chongyang Ma
Date: 2023-12-27 00:00:00
Tasks: Video Generation

Text-guided image-to-video (I2V) generation aims to generate a coherent video that preserves the identity of the input image and semantically aligns with the input prompt. Existing methods typically augment pretrained text-to-video (T2V) models by either concatenating the image with noised video frames channel-wise before being fed into the model or injecting the image embedding produced by pretrained image encoders in cross-attention modules. However, the former approach often necessitates altering the fundamental weights of pretrained T2V models, thus restricting the model's compatibility within the open-source communities and disrupting the model's prior knowledge. Meanwhile, the latter typically fails to preserve the identity of the input image. We present I2V-Adapter to overcome such limitations. I2V-Adapter adeptly propagates the unnoised input image to subsequent noised frames through a cross-frame attention mechanism, maintaining the identity of the input image without any changes to the pretrained T2V model. Notably, I2V-Adapter only introduces a few trainable parameters, significantly alleviating the training cost and also ensures compatibility with existing community-driven personalized models and control tools. Moreover, we propose a novel Frame Similarity Prior to balance the motion amplitude and the stability of generated videos through two adjustable control coefficients. Our experimental results demonstrate that I2V-Adapter is capable of producing high-quality videos. This performance, coupled with its agility and adaptability, represents a substantial advancement in the field of I2V, particularly for personalized and controllable applications.
Prompt Expansion for Adaptive Text-to-Image Generation
Authors: Siddhartha Datta, Alexander Ku, Deepak Ramachandran, Peter Anderson
Date: 2023-12-27 00:00:00
Tasks: Image Generation, Text to Image Generation, Text-to-Image Generation

Text-to-image generation models are powerful but difficult to use. Users craft specific prompts to get better images, though the images can be repetitive. This paper proposes a Prompt Expansion framework that helps users generate high-quality, diverse images with less effort. The Prompt Expansion model takes a text query as input and outputs a set of expanded text prompts that are optimized such that when passed to a text-to-image model, generates a wider variety of appealing images. We conduct a human evaluation study that shows that images generated through Prompt Expansion are more aesthetically pleasing and diverse than those generated by baseline methods. Overall, this paper presents a novel and effective approach to improving the text-to-image generation experience.
A pipeline for multiple orange detection and tracking with 3-D fruit relocalization and neural-net based yield regression in commercial citrus orchards
Authors: Thiago T. Santos, Kleber X. S. de Souza, João Camargo Neto, Luciano V. Koenigkan, Alécio S. Moreira, Sônia Ternes
Date: 2023-12-27 00:00:00

Traditionally, sweet orange crop forecasting has involved manually counting fruits from numerous trees, which is a labor-intensive process. Automatic systems for fruit counting, based on proximal imaging, computer vision, and machine learning, have been considered a promising alternative or complement to manual counting. These systems require data association components that prevent multiple counting of the same fruit observed in different images. However, there is a lack of work evaluating the accuracy of multiple fruit counting, especially considering (i) occluded and re-entering green fruits on leafy trees, and (ii) counting ground-truth data measured in the crop field. We propose a non-invasive alternative that utilizes fruit counting from videos, implemented as a pipeline. Firstly, we employ CNNs for the detection of visible fruits. Inter-frame association techniques are then applied to track the fruits across frames. To handle occluded and re-appeared fruit, we introduce a relocalization component that employs 3-D estimation of fruit locations. Finally, a neural network regressor is utilized to estimate the total number of fruit, integrating image-based fruit counting with other tree data such as crop variety and tree size. The results demonstrate that the performance of our approach is closely tied to the quality of the field-collected videos. By ensuring that at least 30% of the fruit is accurately detected, tracked, and counted, our yield regressor achieves an impressive coefficient of determination of 0.85. To the best of our knowledge, this study represents one of the few endeavors in fruit estimation that incorporates manual fruit counting as a reference point for evaluation. We also introduce annotated datasets for multiple orange tracking (MOrangeT) and detection (OranDet), publicly available to foster the development of novel methods for image-based fruit counting.
Audio-based drone detection and identification using deep learning techniques with dataset enhancement through generative adversarial networks
Authors: Sara Al-Emadi, Abdulla Al-Ali, Abdulaziz Al-Ali
Date: 2021-07-21 00:00:00
Proceeding: journal 2021 7
Tasks: Deep Learning, Generative Adversarial Network

Drones are becoming increasingly popular not only for recreational purposes but in day-to-day applications in engineering, medicine, logistics, security and others. In addition to their useful applications, an alarming concern in regard to the physical infrastructure security, safety and privacy has arisen due to the potential of their use in malicious activities. To address this problem, we propose a novel solution that automates the drone detection and identification processes using a drone’s acoustic features with different deep learning algorithms. However, the lack of acoustic drone datasets hinders the ability to implement an effective solution. In this paper, we aim to fill this gap by introducing a hybrid drone acoustic dataset composed of recorded drone audio clips and artificially generated drone audio samples using a state-of-the-art deep learning technique known as the Generative Adversarial Network. Furthermore, we examine the effectiveness of using drone audio with different deep learning algorithms, namely, the Convolutional Neural Network, the Recurrent Neural Network and the Convolutional Recurrent Neural Network in drone detection and identification. Moreover, we investigate the impact of our proposed hybrid dataset in drone detection. Our findings prove the advantage of using deep learning techniques for drone detection and identification while confirming our hypothesis on the benefits of using the Generative Adversarial Networks to generate real-like drone audio clips with an aim of enhancing the detection of new and unfamiliar drones.
A new perspective on building efficient and expressive 3D equivariant graph neural networks
Authors: Weitao Du, Yuanqi Du, Limei Wang, Dieqiao Feng, Guifeng Wang, Shuiwang Ji, Carla Gomes, Zhi-Ming Ma
Date: 2023-04-07 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Graph Property Prediction, Molecular Property Prediction, Property Prediction

Geometric deep learning enables the encoding of physical symmetries in modeling 3D objects. Despite rapid progress in encoding 3D symmetries into Graph Neural Networks (GNNs), a comprehensive evaluation of the expressiveness of these networks through a local-to-global analysis lacks today. In this paper, we propose a local hierarchy of 3D isomorphism to evaluate the expressive power of equivariant GNNs and investigate the process of representing global geometric information from local patches. Our work leads to two crucial modules for designing expressive and efficient geometric GNNs; namely local substructure encoding (LSE) and frame transition encoding (FTE). To demonstrate the applicability of our theory, we propose LEFTNet which effectively implements these modules and achieves state-of-the-art performance on both scalar-valued and vector-valued molecular property prediction tasks. We further point out the design space for future developments of equivariant graph neural networks. Our codes are available at \url{https://github.com/yuanqidu/LeftNet}.
LagrangeBench: A Lagrangian Fluid Mechanics Benchmarking Suite
Authors: Artur P. Toshev, Gianluca Galletti, Fabian Fritz, Stefan Adami, Nikolaus A. Adams
Date: 2023-09-28 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Benchmarking

Machine learning has been successfully applied to grid-based PDE modeling in various scientific applications. However, learned PDE solvers based on Lagrangian particle discretizations, which are the preferred approach to problems with free surfaces or complex physics, remain largely unexplored. We present LagrangeBench, the first benchmarking suite for Lagrangian particle problems, focusing on temporal coarse-graining. In particular, our contribution is: (a) seven new fluid mechanics datasets (four in 2D and three in 3D) generated with the Smoothed Particle Hydrodynamics (SPH) method including the Taylor-Green vortex, lid-driven cavity, reverse Poiseuille flow, and dam break, each of which includes different physics like solid wall interactions or free surface, (b) efficient JAX-based API with various recent training strategies and three neighbor search routines, and (c) JAX implementation of established Graph Neural Networks (GNNs) like GNS and SEGNN with baseline results. Finally, to measure the performance of learned surrogates we go beyond established position errors and introduce physical metrics like kinetic energy MSE and Sinkhorn distance for the particle distribution. Our codebase is available at https://github.com/tumaer/lagrangebench .
New probabilistic interest measures for association rules
Authors: Michael Hahsler, Kurt Hornik
Date: 2008-03-06 00:00:00

Mining association rules is an important technique for discovering meaningful patterns in transaction databases. Many different measures of interestingness have been proposed for association rules. However, these measures fail to take the probabilistic properties of the mined data into account. In this paper, we start with presenting a simple probabilistic framework for transaction data which can be used to simulate transaction data when no associations are present. We use such data and a real-world database from a grocery outlet to explore the behavior of confidence and lift, two popular interest measures used for rule mining. The results show that confidence is systematically influenced by the frequency of the items in the left hand side of rules and that lift performs poorly to filter random noise in transaction data. Based on the probabilistic framework we develop two new interest measures, hyper-lift and hyper-confidence, which can be used to filter or order mined association rules. The new measures show significantly better performance than lift for applications where spurious rules are problematic.
MemoryInception: Predicting Neurological Recovery from EEG Using Recurrent Inceptions
Authors: Bjørn-Jostein Singstad, Jesper Ravn, Arian Ranjbar
Date: 2023-12-26 00:00:00
Proceeding: Computing in Cardiology 2023 12
Tasks: EEG, Time Series

Cardiac arrest (CA) may cause severe brain damage, cognitive impairments and death. Monitoring neurological recovery after hospitalization is critical to provide suitable treatment. In this study, we aim to develop an algorithm to aid in neurological recovery classification. The dataset used for this purpose includes 1020 patients and contains both continuous sensor measurements, taken from 0 to 72 hours after the CA, and structured data. More specifically the dataset is split into a training set (60%), validation set (10%) and undisclosed test set (30%). The developed model uses a one-dimensional convolutional neural network to extract features from 5-minute time series segments, fed into a recurrent neural network, to capture temporal information and provide adaptability in recording length. The output features are then merged with embedded patient metadata in a fully connected layer, for the final classification of neurological outcome. The project is part of the George B. Moody PhysioNet Challenge 2023, where our team (EEG-Attackers) achieved a challenge score and rank on the undisclosed test dataset of 0.16 (30), 0.13 (33), 0.11 (34) and 0.12 (35) using recordings from the first 12, 24, 48 and 72 hours after CA.
Towards Invariant Soft Biometrics from Electrocardiograms
Authors: Arian Ranjbar, Bjørn-Jostein Singstad, Jesper Ravn, Henrik Schirmer
Date: 2023-12-26 00:00:00
Proceeding: Computing in Cardiology 2023 12
Tasks: Age and Gender Estimation, Multi-Task Learning

Medical data such as the electrocardiogram (ECG) has received an increased interest within biometric settings. One of the main benefits is the difficulty in counterfeiting the information due to its hidden nature. However, medical information may be exposed to intra-subject variability. This is evident in extraction of soft biometric traits from ECG, where results can vary widely depending on cardiac condition and status. This work investigates methods of lowering the variability, by employing multi-task learning on a shared feature extractor. Three different architectures are suggested and benchmarked. Specifically, experiments are carried out on age and gender estimation showing state of the art results on two public ECG datasets, while lowering estimation variance among instances.
Normalization of vibration signals generated under highly varying speed and load with application to signal separation
Authors: Jacek Urbanek, Tomasz Barszcz, Marcin Strączkiewicz, Adam Jablonski
Date: 2016-07-08 00:00:00
Proceeding: Mechanical Systems and Signal Processing 2016 7

The paper presents a normalization dedicated to transform non-stationary vibration sig-
nals into signals characterized by purely stationary properties. For this purpose, a novel
class of generalized periodic signals is defined followed by a proposition of a normal-
ization technique, which takes advantage of available, instantaneous values of operational
parameters. Within the paper, a well-known discrete-random separation (DRS) technique
is recalled as an exemplary technique, which has been restricted to stationary signals so
far. The authors present a step-by-step adoption of the DRS to non-stationary signals. The
method is applied to simulated signal, test rig signal, and a vibration signal from industrial
object. Additionally, for the purpose of synthesis of simulated signal, a new model of
multi-component vibrations generated under varying regime is proposed. The presented
method aims to expand existing solutions dealing with varying frequency to a more
general solution dealing with independent, simultaneous varying frequency and ampli-
tude of signal components.
Understanding and Improving Feature Learning for Out-of-Distribution Generalization
Authors: Yongqiang Chen, Wei Huang, Kaiwen Zhou, Yatao Bian, Bo Han, James Cheng
Date: 2023-04-22 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Out-of-Distribution Generalization

A common explanation for the failure of out-of-distribution (OOD) generalization is that the model trained with empirical risk minimization (ERM) learns spurious features instead of invariant features. However, several recent studies challenged this explanation and found that deep networks may have already learned sufficiently good features for OOD generalization. Despite the contradictions at first glance, we theoretically show that ERM essentially learns both spurious and invariant features, while ERM tends to learn spurious features faster if the spurious correlation is stronger. Moreover, when fed the ERM learned features to the OOD objectives, the invariant feature learning quality significantly affects the final OOD performance, as OOD objectives rarely learn new features. Therefore, ERM feature learning can be a bottleneck to OOD generalization. To alleviate the reliance, we propose Feature Augmented Training (FeAT), to enforce the model to learn richer features ready for OOD generalization. FeAT iteratively augments the model to learn new features while retaining the already learned features. In each round, the retention and augmentation operations are performed on different subsets of the training data that capture distinct features. Extensive experiments show that FeAT effectively learns richer features thus boosting the performance of various OOD objectives.
ASUR3D: Arbitrary Scale Upsampling and Refinement of 3D Point Clouds using Local Occupancy Fields
Authors: Akash Kumbar, Tejas Anvekar, Ramesh Ashok Tabib, Uma Mudenagudi
Date: 2023-10-02 00:00:00
Proceeding: IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) 2023 10
Tasks: 3D Reconstruction, 3D Shape Generation, 3D Shape Representation, Point Cloud Super Resolution, Point Set Upsampling

In this paper, we introduce ASUR3D, a novel methodology for the arbitrary-scale upsampling of 3D point clouds employing Local Occupancy Representation. Our proposed implicit occupancy representation enables efficient point classification, effectively discerning points belonging to the surface from non-surface points. Learning an implicit representation of open surfaces, enables one to capture the better local neighbourhood representation, leading to finer refinement and reconstruction with enhanced preservation of intricate geometric details. Leveraging this capability, we can accurately sample an arbitrary number of points on the surface, facilitating precise and flexible upsampling. We demonstrate the effectiveness of ASUR3D on PUGAN and PU1K benchmark datasets. Our proposed method achieves state-of-the-art results on all benchmarks and for all evaluation metrics. Additionally, we demonstrate the efficacy of our methodology on self-proposed heritage data generated through photogrammetry, further confirming its effectiveness in diverse scenarios. The code is publicly available at https://github.com/Akash-Kumbar/ASUR3D.
TP-NoDe: Topology-aware Progressive Noising and Denoising of Point Clouds towards Upsampling
Authors: Akash Kumbar, Tejas Anvekar, Tulasi Amitha Vikrama, Ramesh Ashok Tabib, Uma Mudenagudi
Date: 2023-10-02 00:00:00
Proceeding: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops 2023 10
Tasks: Denoising, Point Cloud Super Resolution, point cloud upsampling

In this paper, we propose TP-NoDe, a novel Topology-aware Progressive Noising and Denoising technique for 3D point cloud upsampling. TP-NoDe revisits the traditional method of upsampling of the point cloud by introducing a novel perspective of adding local topological noise by incorporating a novel algorithm Density-Aware k nearest neighbour (DA-kNN) followed by denoising to map noisy perturbations to the topology of the point cloud. Unlike previous methods, we progressively upsample the point cloud, starting at a 2 x upsampling ratio and advancing to a desired ratio. TP-NoDe generates intermediate upsampling resolutions for free, obviating the need to train different models for varying upsampling ratios. TP-NoDe mitigates the need for task-specific training of upsampling networks for a specific upsampling ratio by reusing a point cloud denoising framework. We demonstrate the supremacy of our method TP-NoDe on the PU-GAN dataset and compare it with state-of-the-art upsampling methods. The code is publicly available at https://github.com/Akash-Kumbar/TPNoDe.
μ-Net: ConvNext-Based U-Nets for Cosmic Muon Tomography
Authors: Lim Li Xin Jed, Qiu Ziming
Date: 2023-12-25 00:00:00

Muon scattering tomography utilises muons, typically originating from cosmic rays to image the interiors of dense objects. However, due to the low flux of cosmic ray muons at sea-level and the highly complex interactions that muons display when travelling through matter, existing reconstruction algorithms often suffer from low resolution and high noise. In this work, we develop a novel two-stage deep learning algorithm, μ-Net, consisting of an MLP to predict the muon trajectory and a ConvNeXt-based U-Net to convert the scattering points into voxels. μ-Net achieves a state-of-the-art performance of 17.14 PSNR at the dosage of 1024 muons, outperforming traditional reconstruction algorithms such as the point of closest approach algorithm and maximum likelihood and expectation maximisation algorithm. Furthermore, we find that our method is robust to various corruptions such as inaccuracies in the muon momentum or a limited detector resolution. We also generate and publicly release the first large-scale dataset that maps muon detections to voxels. We hope that our research will spark further investigations into the potential of deep learning to revolutionise this field.
$μ$-Net: ConvNext-Based U-Nets for Cosmic Muon Tomography
Authors: Li Xin Jed Lim, Ziming Qiu
Date: 2023-12-25 00:00:00

Muon scattering tomography utilises muons, typically originating from cosmic rays to image the interiors of dense objects. However, due to the low flux of cosmic ray muons at sea-level and the highly complex interactions that muons display when travelling through matter, existing reconstruction algorithms often suffer from low resolution and high noise. In this work, we develop a novel two-stage deep learning algorithm, $\mu$-Net, consisting of an MLP to predict the muon trajectory and a ConvNeXt-based U-Net to convert the scattering points into voxels. $\mu$-Net achieves a state-of-the-art performance of 17.14 PSNR at the dosage of 1024 muons, outperforming traditional reconstruction algorithms such as the point of closest approach algorithm and maximum likelihood and expectation maximisation algorithm. Furthermore, we find that our method is robust to various corruptions such as inaccuracies in the muon momentum or a limited detector resolution. We also generate and publicly release the first large-scale dataset that maps muon detections to voxels. We hope that our research will spark further investigations into the potential of deep learning to revolutionise this field.