pwc-archive/papers-with-abstracts huggingface dataset:
Incremental Generative Occlusion Adversarial Suppression Network for Person ReID
Authors: Cairong Zhao, Xinbi Lv, Shuguang Dou, Shanshan Zhang, Jun Wu, Liang Wang
Date: 2021-04-06 00:00:00
Proceeding: IEEE Transactions on Image Processing 2021 4
Tasks: Data Augmentation, Person Re-Identification

Person re-identification (re-id) suffers from the significant challenge of occlusion, where an image contains occlusions and less discriminative pedestrian information. However, certain work consistently attempts to design complex modules to capture implicit information (including human pose landmarks, mask maps, and spatial information). The network, consequently, focuses on discriminative features learning on human nonoccluded body regions and realizes effective matching under spatial misalignment. Few studies have focused on data augmentation, given that existing single-based data augmentation methods bring limited performance improvement. To address the occlusion problem, we propose a novel Incremental Generative Occlusion Adversarial Suppression (IGOAS) network. It consists of 1) an incremental generative occlusion block, generating easy-to-hard occlusion data, that makes the network more robust to occlusion by gradually learning harder occlusion instead of hardest occlusion directly. And 2) a global-adversarial suppression (G&A) framework with a global branch and an adversarial suppression branch. The global branch extracts steady global features of the images. The adversarial suppression branch, embedded with two occlusion suppression module, minimizes the generated occlusion’s response and strengthens attentive feature representation on human non-occluded body regions. Finally, we get a more discriminative pedestrian feature descriptor by concatenating two branches’ features, which is robust to the occlusion problem. The experiments on the occluded dataset show the competitive performance of IGOAS. On Occluded-DukeMTMC, it achieves 60.1% Rank-1 accuracy and 49.4% mAP.
Human Co-Parsing Guided Alignment for Occluded Person Re-identification
Authors: Shuguang Dou, Cairong Zhao, Xinyang Jiang, Shanshan Zhang, Wei-Shi Zheng, WangMeng Zuo
Date: 2022-12-20 00:00:00
Proceeding: IEEE Transactions on Image Processing 2022 12
Tasks: Human Parsing, Occluded Person Re-Identification, Person Re-Identification

Occluded person re-identification (ReID) is a challenging task due to more background noises and incomplete foreground information. Although existing human parsing-based ReID methods can tackle this problem with semantic alignment at the finest pixel level, their performance is heavily affected by the human parsing model. Most supervised methods propose to train an extra human parsing model aside from the ReID model with cross-domain human parts annotation, suffering from expensive annotation cost and domain gap; Unsupervised methods integrate a feature clustering-based human parsing process into the ReID model, but lacking supervision signals brings less satisfactory segmentation results. In this paper, we argue that the pre-existing information in the ReID training dataset can be directly used as supervision signals to train the human parsing model without any extra annotation. By integrating a weakly supervised human co-parsing network into the ReID network, we propose a novel framework that exploits shared information across different images of the same pedestrian, called the Human Co-parsing Guided Alignment (HCGA) framework. Specifically, the human co-parsing network is weakly supervised by three consistency criteria, namely global semantics, local space, and background. By feeding the semantic information and deep features from the person ReID network into the guided alignment module, features of the foreground and human parts can then be obtained for effective occluded person ReID. Experiment results on two occluded and two holistic datasets demonstrate the superiority of our method. Especially on Occluded-DukeMTMC, it achieves 70.2% Rank-1 accuracy and 57.5% mAP.
EA-HAS-Bench:Energy-Aware Hyperparameter and Architecture Search Benchmark
Authors: Shuguang Dou, Xinyang Jiang, Cai Rong Zhao, Dongsheng Li
Date: 2023-05-01 00:00:00
Proceeding: The Eleventh International Conference on Learning Representations 2023 5
Tasks: AutoML

The energy consumption for training deep learning models is increasing at an alarming rate due to the growth of training data and model scale, resulting in a negative impact on carbon neutrality. Energy consumption is an especially pressing issue for AutoML algorithms because it usually requires repeatedly training large numbers of computationally intensive deep models to search for optimal configurations. This paper takes one of the most essential steps in developing energy-aware (EA) NAS methods, by providing a benchmark that makes EANAS research more reproducible and accessible. Specifically, we present the first large-scale energy-aware benchmark that allows studying AutoML methods to achieve better trade-offs between performance and search energy consumption, named EA-HAS-Bench. EA-HAS-Bench provides a large-scale architecture/hyperparameter joint search space, covering diversified configurations related to energy consumption. Furthermore, we propose a novel surrogate model specially designed for large joint search space, which proposes a Bezier curve-based model to predict learning curves with unlimited shape and length. Based on the proposed dataset, we modify existing AutoML algorithms to consider the search energy consumption, and our experiments show that the modified energy-aware AutoML methods achieve a better trade-off between energy consumption and model performance.
MocFormer: A Two-Stage Pre-training-Driven Transformer for Drug-Target Interactions Prediction
Authors: Yilun Zhang, Wentao Wang, Jiahui Guan, Deepak Kumar Jain, Tianyang Wang, Swalpa Kumar Roy
Date: 2023-10-26 00:00:00
Proceeding: bioRxiv 2023 10
Tasks: Drug–drug Interaction Extraction

Drug-target interactions (DTIs) is essential for advancing pharmaceuticals. Traditional drug-target interaction studies rely on labor-intensive laboratory techniques. Still, recent advancements in computing power have elevated the importance of deep learning methods, offering faster, more precise, and cost-effective screening and prediction. Nonetheless, general deep learning methods often yield low-confidence results due to the complex nature of drugs and proteins, bias, limited labeled data, and feature extraction challenges. To address these challenges, a novel two-stage pre-trained framework is proposed for DTIs prediction. In the first stage, pre-trained molecule and protein models develop a comprehensive feature representation, enhancing the framework's ability to handle drug and protein diversity. This also reduces bias, improving prediction accuracy. In the second stage, a transformer with bilinear pooling and a fully connected layer (FCN) enables predictions based on feature vectors. Comprehensive experiments were conducted using DrugBank dataset and Epigenetic-regulators dataset to evaluate the framework's effectiveness. The results demonstrate that the proposed framework outperforms the state-of-the-art methods regarding accuracy, area under the ROC curve (AUC), recall, and the area under the precision-recall curve (AUPRC). The code will be available after being accepted: https://github.com/rickwang28574/MocFormer
Learning Probabilistic Symmetrization for Architecture Agnostic Equivariance
Authors: Jinwoo Kim, Tien Dat Nguyen, Ayhan Suleymanzade, Hyeokjun An, Seunghoon Hong
Date: 2023-06-05 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Graph Classification, Graph Regression, Link Prediction

We present a novel framework to overcome the limitations of equivariant architectures in learning functions with group symmetries. In contrary to equivariant architectures, we use an arbitrary base model such as an MLP or a transformer and symmetrize it to be equivariant to the given group by employing a small equivariant network that parameterizes the probabilistic distribution underlying the symmetrization. The distribution is end-to-end trained with the base model which can maximize performance while reducing sample complexity of symmetrization. We show that this approach ensures not only equivariance to given group but also universal approximation capability in expectation. We implement our method on various base models, including patch-based transformers that can be initialized from pretrained vision transformers, and test them for a wide range of symmetry groups including permutation and Euclidean groups and their combinations. Empirical tests show competitive results against tailored equivariant architectures, suggesting the potential for learning equivariant functions for diverse groups using a non-equivariant universal base architecture. We further show evidence of enhanced learning in symmetric modalities, like graphs, when pretrained from non-symmetric modalities, like vision. Code is available at https://github.com/jw9730/lps.
FRC-GIF: Frame Ranking-based Personalized Artistic Media Generation Method for Resource Constrained Devices
Authors: Ghulam Mujtaba, Sunder Ali Khowaja, Muhammad Aslam Jarwar, Jaehyuk Choi, Eun-Seok Ryu
Date: 2023-11-30 00:00:00
Proceeding: journal 2023 11
Tasks: Animated GIF Generation, Classification

Generating video highlights in the form of animated graphics interchange formats (GIFs) has significantly simplified the process of video browsing. Animated GIFs have paved the way for applications concerning streaming platforms and emerging technologies. Existing studies have led to large computational complexity without considering user personalization. This paper proposes lightweight method to attract users and increase views of videos through personalized artistic media, i.e., static thumbnails and animated GIF generation. The proposed method analyzes lightweight thumbnail containers (LTC) using the computational resources of the client device to recognize personalized events from feature-length sports videos. Next, the thumbnails are then ranked through the frame rank pooling method for their selection. Subsequently, the proposed method processes small video segments rather than considering the whole video for generating artistic media. This makes our approach more computationally efficient compared to existing methods that use the entire video data; thus, the proposed method complies with sustainable development goals. Furthermore, the proposed method retrieves and uses thumbnail containers and video segments, which reduces the required transmission bandwidth as well as the amount of locally stored data. Experiments reveal that the computational complexity of our method is 3.73 times lower than that of the state-of-the-art method.
Convolutional Neural Operators for robust and accurate learning of PDEs
Authors: Bogdan Raonić, Roberto Molinaro, Tim De Ryck, Tobias Rohner, Francesca Bartolucci, Rima Alaifari, Siddhartha Mishra, Emmanuel de Bézenac
Date: 2023-02-02 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Operator learning, PDE Surrogate Modeling

Although very successfully used in conventional machine learning, convolution based neural network architectures -- believed to be inconsistent in function space -- have been largely ignored in the context of learning solution operators of PDEs. Here, we present novel adaptations for convolutional neural networks to demonstrate that they are indeed able to process functions as inputs and outputs. The resulting architecture, termed as convolutional neural operators (CNOs), is designed specifically to preserve its underlying continuous nature, even when implemented in a discretized form on a computer. We prove a universality theorem to show that CNOs can approximate operators arising in PDEs to desired accuracy. CNOs are tested on a novel suite of benchmarks, encompassing a diverse set of PDEs with possibly multi-scale solutions and are observed to significantly outperform baselines, paving the way for an alternative framework for robust and accurate operator learning. Our code is publicly available at https://github.com/bogdanraonic3/ConvolutionalNeuralOperator
Hadiths Classification Using a Novel Author-Based Hadith Classification Dataset (ABCD)
Authors: Ahmed Ramzy, Marwan Torki, Mohamed Abdeen, Omar Saif, Mustafa ElNainay, AbdAllah Alshanqiti, Emad Nabil
Date: 2023-08-14 00:00:00
Proceeding: Big Data Cogn. Comput. 2023 8
Tasks: 16k, Classification

Religious studies are a rich land for Natural Language Processing (NLP). The reason is
that all religions have their instructions as written texts. In this paper, we apply NLP to Islamic
Hadiths, which are the written traditions, sayings, actions, approvals, and discussions of the Prophet
Muhammad, his companions, or his followers. A Hadith is composed of two parts: the chain of
narrators (Sanad) and the content of the Hadith (Matn). A Hadith is transmitted from its author to a
Hadith book author using a chain of narrators. The problem we solve focuses on the classification
of Hadiths based on their origin of narration. This is important for several reasons. First, it helps
determine the authenticity and reliability of the Hadiths. Second, it helps trace the chain of narration
and identify the narrators involved in transmitting Hadiths. Finally, it helps understand the historical
and cultural contexts in which Hadiths were transmitted, and the different levels of authority
attributed to the narrators. To the best of our knowledge, and based on our literature review, this
problem is not solved before using machine/deep learning approaches. To solve this classification
problem, we created a novel Author-Based Hadith Classification Dataset (ABCD) collected from
classical Hadiths’ books. The ABCD size is 29 K Hadiths and it contains unique 18 K narrators, with
all their information. We applied machine learning (ML), and deep learning (DL) approaches. ML
was applied on Sanad and Matn separately; then, we did the same with DL. The results revealed that
ML performs better than DL using the Matn input data, with a 77% F1-score. DL performed better
than ML using the Sanad input data, with a 92% F1-score. We used precision and recall alongside the
F1-score; details of the results are explained at the end of the paper. We claim that the ABCD and the
reported results will motivate the community to work in this new area. Our dataset and results will
represent a baseline for further research on the same problem.
The Paradox Of Just-in-Time Liquidity in Decentralized Exchanges: More Providers Can Sometimes Mean Less Liquidity
Authors: Agostino Capponi, Ruizhe Jia, Brian Zhu
Date: 2023-11-30 00:00:00

We study Just-in-time (JIT) liquidity provision in blockchain-based decentralized exchanges. A JIT liquidity provider (LP) monitors pending swap orders in public mempools of blockchains to sandwich orders of their choice with liquidity, depositing right before and withdrawing right after the order. Our game-theoretic model with asymmetrically informed agents reveals that a JIT LP's presence does not always enhance liquidity pool depth, as one might expect. While passive LPs face adverse selection by informed arbitrageurs, a JIT LP's ability to detect pending orders for toxic order flow prior to liquidity provision lets them avoid being adversely selected. JIT LPs thus only provide liquidity to uninformed orders and crowd out passive LPs when order volume is not sufficiently elastic to pool depth, possibly reducing overall market liquidity. We show that using a two-tiered fee structure which transfers a part of a JIT LP's fee revenue to passive LPs or allowing for JIT LPs to compete \`{a} la Cournot are potential solutions to mitigate the negative effects of JIT liquidity.
Can AI Detect Wash Trading? Evidence from NFTs
Authors: Brett Hemenway Falk, Gerry Tsoukalas, Niuniu Zhang
Date: 2023-11-30 00:00:00

Existing studies on crypto wash trading often use indirect statistical methods or leaked private data, both with inherent limitations. This paper leverages public on-chain NFT data for a more direct and granular estimation. Analyzing three major exchanges, we find that ~38% (30-40%) of trades and ~60% (25-95%) of traded value likely involve manipulation, with significant variation across exchanges. This direct evidence enables a critical reassessment of existing indirect methods, identifying roundedness-based regressions \`a la Cong et al. (2023) as most promising, though still error-prone in the NFT setting. To address this, we develop an AI-based estimator that integrates these regressions in a machine learning framework, significantly reducing both exchange- and trade-level estimation errors in NFT markets (and beyond).
Robust-to-Noise Algorithms for Distributed Resource Allocation and Scheduling
Authors: Mohammadreza Doostmohammadian, Alireza Aghasi
Date: 2023-11-30 00:00:00
Tasks: Cloud Computing, Scheduling

Efficient resource allocation and scheduling algorithms are essential for various distributed applications, ranging from wireless networks and cloud computing platforms to autonomous multi-agent systems and swarm robotic networks. However, real-world environments are often plagued by uncertainties and noise, leading to sub-optimal performance and increased vulnerability of traditional algorithms. This paper addresses the challenge of robust resource allocation and scheduling in the presence of noise and disturbances. The proposed study introduces a novel sign-based dynamics for developing robust-to-noise algorithms distributed over a multi-agent network that can adaptively handle external disturbances. Leveraging concepts from convex optimization theory, control theory, and network science the framework establishes a principled approach to design algorithms that can maintain key properties such as resource-demand balance and constraint feasibility. Meanwhile, notions of uniform-connectivity and versatile networking conditions are also addressed.
Unrolling Virtual Worlds for Immersive Experiences
Authors: Alexey Tikhonov, Anton Repushko
Date: 2023-11-14 00:00:00
Tasks: Navigate

This research pioneers a method for generating immersive worlds, drawing inspiration from elements of vintage adventure games like Myst and employing modern text-to-image models. We explore the intricate conversion of 2D panoramas into 3D scenes using equirectangular projections, addressing the distortions in perception that occur as observers navigate within the encompassing sphere. Our approach employs a technique similar to "inpainting" to rectify distorted projections, enabling the smooth construction of locally coherent worlds. This provides extensive insight into the interrelation of technology, perception, and experiential reality within human-computer interaction.
A New Old Idea: Beam-Steering Reflectarrays for Efficient Sub-THz Multiuser MIMO
Authors: Krishan Kumar Tiwari, Giuseppe Caire
Date: 2023-11-30 00:00:00
Tasks: 3D geometry, Quantization

We present a novel, power- & hardware-efficient, multiuser, multibeam RIS (Reflective Intelligent Surface) architecture for multiuser MIMO, especially for very high frequency bands (e.g., high mmWave and sub-THz), where channels are typically sparse in the beamspace and LOS is the dominant component. The key module is formed by an active multiantenna feeder (AMAF) with a small number of active antennas, placed in the near field of a RIS with a much larger number of passive controllable reflecting elements. We propose a pragmatic approach to obtain a steerable beam with high gain and very low sidelobes. Then K independently controlled beams can be achieved by closely stacking K such AMAF-RIS modules. Our analysis includes the mutual interference between the modules and the fact that, due to the delay difference of propagation through the AMAF-RIS structure, the resulting channel matrix is frequency selective even for pure LOS propagation. We consider a 3D geometry and show that "beam focusing" is in fact possible (and much more effective in terms of coverage) also in the far-field, by creating spotbeams with limited footprint both in angle and in range. Our results show that: 1) simple RF beamforming (BF) without computationally expensive baseband multiuser precoding is sufficient to practically eliminate multiuser interference when the users are chosen with sufficient angular/range separation, thanks to the extremely low sidelobe beams; 2) the impact of beam pointing errors with standard deviation as large as 2.5 deg and RIS quantized phase-shifters with quantization bits > 2 is essentially negligible; 3) The proposed architecture is more power efficient & much simpler from a hardware implementation viewpoint than standard active arrays with the same BF performance. We show also that the array gain of the proposed AMAF-RIS structure is linear with the RIS aperture.
Subspace Hybrid MVDR Beamforming for Augmented Hearing
Authors: Sina Hafezi, Alastair H. Moore, Pierre H. Guiraud, Patrick A. Naylor, Jacob Donley, Vladimir Tourbabin, Thomas Lunner
Date: 2023-11-30 00:00:00
Tasks: Computational Efficiency, Speech Enhancement

Signal-dependent beamformers are advantageous over signal-independent beamformers when the acoustic scenario - be it real-world or simulated - is straightforward in terms of the number of sound sources, the ambient sound field and their dynamics. However, in the context of augmented reality audio using head-worn microphone arrays, the acoustic scenarios encountered are often far from straightforward. The design of robust, high-performance, adaptive beamformers for such scenarios is an on-going challenge. This is due to the violation of the typically required assumptions on the noise field caused by, for example, rapid variations resulting from complex acoustic environments, and/or rotations of the listener's head. This work proposes a multi-channel speech enhancement algorithm which utilises the adaptability of signal-dependent beamformers while still benefiting from the computational efficiency and robust performance of signal-independent super-directive beamformers. The algorithm has two stages. (i) The first stage is a hybrid beamformer based on a dictionary of weights corresponding to a set of noise field models. (ii) The second stage is a wide-band subspace post-filter to remove any artifacts resulting from (i). The algorithm is evaluated using both real-world recordings and simulations of a cocktail-party scenario. Noise suppression, intelligibility and speech quality results show a significant performance improvement by the proposed algorithm compared to the baseline super-directive beamformer. A data-driven implementation of the noise field dictionary is shown to provide more noise suppression, and similar speech intelligibility and quality, compared to a parametric dictionary.
Audio Prompt Tuning for Universal Sound Separation
Authors: Yuzhuo Liu, Xubo Liu, Yan Zhao, Yuanyuan Wang, Rui Xia, Pingchuan Tain, Yuxuan Wang
Date: 2023-11-30 00:00:00

Universal sound separation (USS) is a task to separate arbitrary sounds from an audio mixture. Existing USS systems are capable of separating arbitrary sources, given a few examples of the target sources as queries. However, separating arbitrary sounds with a single system is challenging, and the robustness is not always guaranteed. In this work, we propose audio prompt tuning (APT), a simple yet effective approach to enhance existing USS systems. Specifically, APT improves the separation performance of specific sources through training a small number of prompt parameters with limited audio samples, while maintaining the generalization of the USS model by keeping its parameters frozen. We evaluate the proposed method on MUSDB18 and ESC-50 datasets. Compared with the baseline model, APT can improve the signal-to-distortion ratio performance by 0.67 dB and 2.06 dB using the full training set of two datasets. Moreover, APT with only 5 audio samples even outperforms the baseline systems utilizing full training data on the ESC-50 dataset, indicating the great potential of few-shot APT.