pwc-archive/papers-with-abstracts huggingface dataset:
Graph Koopman Autoencoder for Predictive Covert Communication Against UAV Surveillance
Authors: Sivaram Krishnan, Jihong Park, Gregory Sherman, Benjamin Campbell, Jinho Choi
Date: 2024-01-23 00:00:00

Low Probability of Detection (LPD) communication aims to obscure the very presence of radio frequency (RF) signals, going beyond just hiding the content of the communication. However, the use of Unmanned Aerial Vehicles (UAVs) introduces a challenge, as UAVs can detect RF signals from the ground by hovering over specific areas of interest. With the growing utilization of UAVs in modern surveillance, there is a crucial need for a thorough understanding of their unknown nonlinear dynamic trajectories to effectively implement LPD communication. Unfortunately, this critical information is often not readily available, posing a significant hurdle in LPD communication. To address this issue, we consider a case-study for enabling terrestrial LPD communication in the presence of multiple UAVs that are engaged in surveillance. We introduce a novel framework that combines graph neural networks (GNN) with Koopman theory to predict the trajectories of multiple fixed-wing UAVs over an extended prediction horizon. Using the predicted UAV locations, we enable LPD communication in a terrestrial ad-hoc network by controlling nodes' transmit powers to keep the received power at UAVs' predicted locations minimized. Our extensive simulations validate the efficacy of the proposed framework in accurately predicting the trajectories of multiple UAVs, thereby effectively establishing LPD communication.
DoorINet: Door Heading Prediction through Inertial Deep Learning
Authors: Aleksei Zakharchenko, Sharon Farber, Itzik Klein
Date: 2024-01-24 00:00:00
Tasks: Deep Learning, Prediction

Inertial sensors are widely used in a variety of applications. A common task is orientation estimation. To tackle such a task, attitude and heading reference system algorithms are applied. Relying on the gyroscope readings, the accelerometer measurements are used to update the attitude angles, and magnetometer measurements are utilized to update the heading angle. In indoor environments, magnetometers suffer from interference that degrades their performance resulting in poor heading angle estimation. Therefore, applications that estimate the heading angle of moving objects, such as walking pedestrians, closets, and refrigerators, are prone to error. To circumvent such situations, we propose DoorINet, an end-to-end deep-learning framework to calculate the heading angle from door-mounted, low-cost inertial sensors without using magnetometers. To evaluate our approach, we record a unique dataset containing 391 minutes of accelerometer and gyroscope measurements and corresponding ground-truth heading angle. We show that our proposed approach outperforms commonly used, model based approaches and data-driven methods.
ML-ASPA: A Contemplation of Machine Learning-based Acoustic Signal Processing Analysis for Sounds, & Strains Emerging Technology
Authors: Ratul Ali, Aktarul Islam, Md. Shohel Rana, Saila Nasrin, Sohel Afzal Shajol, Professor Dr. A. H. M. Saifullah Sadi
Date: 2023-12-18 00:00:00

Acoustic data serves as a fundamental cornerstone in advancing scientific and engineering understanding across diverse disciplines, spanning biology, communications, and ocean and Earth science. This inquiry meticulously explores recent advancements and transformative potential within the domain of acoustics, specifically focusing on machine learning (ML) and deep learning. ML, comprising an extensive array of statistical techniques, proves indispensable for autonomously discerning and leveraging patterns within data. In contrast to traditional acoustics and signal processing, ML adopts a data-driven approach, unveiling intricate relationships between features and desired labels or actions, as well as among features themselves, given ample training data. The application of ML to expansive sets of training data facilitates the discovery of models elucidating complex acoustic phenomena such as human speech and reverberation. The dynamic evolution of ML in acoustics yields compelling results and holds substantial promise for the future. The advent of electronic stethoscopes and analogous recording and data logging devices has expanded the application of acoustic signal processing concepts to the analysis of bowel sounds. This paper critically reviews existing literature on acoustic signal processing for bowel sound analysis, outlining fundamental approaches and applicable machine learning principles. It chronicles historical progress in signal processing techniques that have facilitated the extraction of valuable information from bowel sounds, emphasizing advancements in noise reduction, segmentation, signal enhancement, feature extraction, sound localization, and machine learning techniques...
Short-Form Videos and Mental Health: A Knowledge-Guided Neural Topic Model
Authors: Jiaheng Xie, Ruicheng Liang, Yidong Chai, Yang Liu, Daniel Zeng
Date: 2024-01-11 00:00:00
Tasks: Form, Topic Models, Video Classification

Along with the rise of short-form videos, their mental impacts on viewers have led to widespread consequences, prompting platforms to predict videos' impact on viewers' mental health. Subsequently, they can take intervention measures according to their community guidelines. Nevertheless, applicable predictive methods lack relevance to well-established medical knowledge, which outlines clinically proven external and environmental factors of mental disorders. To account for such medical knowledge, we resort to an emergent methodological discipline, seeded Neural Topic Models (NTMs). However, existing seeded NTMs suffer from the limitations of single-origin topics, unknown topic sources, unclear seed supervision, and suboptimal convergence. To address those challenges, we develop a novel Knowledge-Guided NTM to predict a short-form video's suicidal thought impact on viewers. Extensive empirical analyses using TikTok and Douyin datasets prove that our method outperforms state-of-the-art benchmarks. Our method also discovers medically relevant topics from videos that are linked to suicidal thought impact. We contribute to IS with a novel video analytics method that is generalizable to other video classification problems. Practically, our method can help platforms understand videos' suicidal thought impacts, thus moderating videos that violate their community guidelines.
Towards a large-scale fused and labeled dataset of human pose while interacting with robots in shared urban areas
Authors: E. Sherafat, B. Farooq
Date: 2024-01-28 00:00:00
Tasks: Navigate, Pose Estimation, Pose Prediction, Simultaneous Localization and Mapping

Over the last decade, Autonomous Delivery Robots (ADRs) have transformed conventional delivery methods, responding to the growing e-commerce demand. However, the readiness of ADRs to navigate safely among pedestrians in shared urban areas remains an open question. We contend that there are crucial research gaps in understanding their interactions with pedestrians in such environments. Human Pose Estimation is a vital stepping stone for various downstream applications, including pose prediction and socially aware robot path-planning. Yet, the absence of an enriched and pose-labeled dataset capturing human-robot interactions in shared urban areas hinders this objective. In this paper, we bridge this gap by repurposing, fusing, and labeling two datasets, MOT17 and NCLT, focused on pedestrian tracking and Simultaneous Localization and Mapping (SLAM), respectively. The resulting unique dataset represents thousands of real-world indoor and outdoor human-robot interaction scenarios. Leveraging YOLOv7, we obtained human pose visual and numeric outputs and provided ground truth poses using manual annotation. To overcome the distance bias present in the traditional MPJPE metric, this study introduces a novel human pose estimation error metric called Mean Scaled Joint Error (MSJE) by incorporating bounding box dimensions into it. Findings demonstrate that YOLOv7 effectively estimates human pose in both datasets. However, it exhibits weaker performance in specific scenarios, like indoor, crowded scenes with a focused light source, where both MPJPE and MSJE are recorded as 10.89 and 25.3, respectively. In contrast, YOLOv7 performs better in single-person estimation (NCLT seq 2) and outdoor scenarios (MOT17 seq1), achieving MSJE values of 5.29 and 3.38, respectively.
Deep hybrid models: infer and plan in a dynamic world
Authors: Matteo Priorelli, Ivilin Peev Stoianov
Date: 2024-02-01 00:00:00
Tasks: Decision Making, reinforcement-learning, Reinforcement Learning

To determine an optimal plan for complex tasks, one often deals with dynamic and hierarchical relationships between several entities. Traditionally, such problems are tackled with optimal control, which relies on the optimization of cost functions; instead, a recent biologically-motivated proposal casts planning and control as an inference process. Active inference assumes that action and perception are two complementary aspects of life whereby the role of the former is to fulfill the predictions inferred by the latter. Here, we present an active inference approach that exploits discrete and continuous processing, based on three features: the representation of potential body configurations in relation to the objects of interest; the use of hierarchical relationships that enable the agent to easily interpret and flexibly expand its body schema for tool use; the definition of potential trajectories related to the agent's intentions, used to infer and plan with dynamic elements at different temporal scales. We evaluate this deep hybrid model on a habitual task: reaching a moving object after having picked a moving tool. We show that the model can tackle the presented task under different conditions. This study extends past work on planning as inference and advances an alternative direction to optimal control.
Mathematical Explanations
Authors: Joseph Y. Halpern
Date: 2023-12-31 00:00:00
Tasks: All

A definition of what counts as an explanation of mathematical statement, and when one explanation is better than another, is given. Since all mathematical facts must be true in all causal models, and hence known by an agent, mathematical facts cannot be part of an explanation (under the standard notion of explanation). This problem is solved using impossible possible worlds.
LSRFormer: Efficient Transformer Supply Convolutional Neural Networks with Global Information for Aerial Image Segmentation
Authors: Renhe Zhang, Qian Zhang, Guixu Zhang
Date: 2024-02-16 00:00:00
Proceeding: IEEE Transactions on Geoscience and Remote Sensing 2024 2
Tasks: Image Segmentation, Semantic Segmentation

Both local context and global context information are essential for the semantic segmentation of aerial images. Convolutional Neural Networks (CNNs) can capture local context information well but cannot model the global dependencies. Vision transformers (ViTs) are good at extracting global information but cannot retain the spatial details well. In order to leverage the advantages of these two paradigms, we integration them in one model in this study. However, global token interaction of ViT brings high computational cost, which makes it difficult to apply to large-sized aerial images. To handle this problem, we propose a novel efficient ViT block named long-short-range transformer (LSRFormer). Instead of mainstream ViTs designed as backbones, LSRFormer is a pre-training-free and plug-and-play module to be appended after CNN stages to supplement the global information. It is composed of long-range self-attention (LR-SA), short-range self-attention (SR-SA), and multi-scale-convolutional feed-forward-network (MSC-FFN). LR-SA establishes long-range dependencies at the junction of the windows and SR-SA diffuses the long-range information from window boundary to internal. MSC-FFN can capture multi-scale information inside the ViT block. We append LSRFormer block after each CNN stage of a pure convolutional network to build a model named ConvLSR-Net. Compared with existing models which combining CNN and ViTs, our model can learn both local and global representation at all stages of the model. In particular, ConvLSR-Net achieves state-of-the-art (SOTA) results on four challenging aerial image segmentation benchmarks, including iSAID, LoveDA, ISPRS Potsdam and Vaihingen. Code has been released at https://github.com/stdcoutzrh/ConvLSR-Net.
ESA: External Space Attention Aggregation for Image-Text Retrieval
Authors: Hongguang Zhu; Chunjie Zhang; Yunchao Wei; Shujuan Huang; Yao Zhao
Date: 2023-10-10 00:00:00
Proceeding: journal 2023 10
Tasks: Image-text Retrieval, Retrieval, Text Retrieval, Triplet

Due to the large gap between vision and language modalities, effective and efficient image-text retrieval is still an unsolved problem. Recent progress devotes to unilaterally pursuing retrieval accuracy by either entangled image-text interaction or large-scale vision-language pre-training in a brute force way. However, the former often leads to unacceptable retrieval time explosion when deploying on large-scale databases. The latter heavily relies on the extra corpus to learn better alignment in the feature space while obscuring the contribution of the network architecture. In this work, we aim to investigate a trade-off to balance effectiveness and efficiency. To this end, on the premise of efficient retrieval, we propose the plug-and-play External Space attention Aggregation (ESA) module to enable element-wise fusion of modal features under spatial dimensional attention. Based on flexible spatial awareness, we further propose the Self-Expanding triplet Loss (SEL) to expand the representation space of samples and optimize the alignment of embedding space. The extensive experiments demonstrate the effectiveness of our method on two benchmark datasets. With identical visual and textual backbones, our single model has outperformed the ensemble modal of similar methods, and our ensemble model can further expand the advantage. Meanwhile, compared with the vision-language pre-training embedding-base method that used 83× image-text pairs than ours, our approach not only surpasses in performance but also accelerates 3× on retrieval time. Codes and pre-trained models are available at https://github.com/KevinLight831/ESA
Accelerate adversarial training with loss guided propagation for robust image classification
Authors: Changkai Xu, Chunjie Zhang, Yanwu Yang, Huaizhi Yang, Yijun Bo, Danyong Li, Riquan Zhang
Date: 2023-01-01 00:00:00
Proceeding: journal 2023 1
Tasks: image-classification, Image Classification

Adversarial training is effective to train robust image classification models. To improve the robustness, existing approaches often use many propagations to generate adversarial examples, which have high time consumption. In this work, we propose an efficient adversarial training method with loss guided propagation (ATLGP) to accelerate the adversarial training process. ATLGP takes the loss value of generated adversarial examples as guidance to control the number of propagations for each training instance at different training stages, which decreases the computation while keeping the strength of generated adversarial examples. In this way, our method can achieve comparable robustness with less time than traditional training methods. It also has good generalization ability and can be easily combined with other efficient training methods. We conduct comprehensive experiments on CIFAR10 and MNIST, the standard datasets for several benchmarks. The experimental results show that ATLGP reduces 30% to 60% training time compared with other baseline methods while achieving similar robustness against various adversarial attacks. The combination of ATLGP and ATTA (an efficient adversarial training method) achieves superior acceleration potential when robustness meets high requirements. The statistical propagation in different training stages and ablation studies prove the effectiveness of applying loss guided propagation for each training instance. The acceleration technique can more easily extend adversarial training methods to large-scale datasets and more diverse model architectures such as vision transformers.
Anchor-free temporal action localization via Progressive Boundary-aware Boosting
Authors: Yepeng Tang, Weining Wang, Yanwu Yang, Chunjie Zhang, Jing Liu
Date: 2023-01-01 00:00:00
Proceeding: journal 2023 1
Tasks: Action Localization, Temporal Action Localization

Enormous untrimmed videos from the real world are difficult to analyze and manage. Temporal action localization algorithms can help us to locate and recognize human activity clips in untrimmed videos. Recently, anchor-free temporal action localization methods have gained increasing attention due to small computational costs and no complex hyperparameters of pre-set anchors. Although the performance has been significantly improved, most existing anchor-free temporal action localization methods still suffer from inaccurate action boundary predictions. In this paper, we want to alleviate the above problem through boundary refinement and temporal context aggregation. To this end, a novel Progressive Boundary-aware Boosting Network (PBBNet) is proposed for anchor-free temporal action localization. The PBBNet consists of three main modules: Temporal Context-aware Module (TCM), Instance-wise Boundary-aware Module (IBM), and Frame-wise Progressive Boundary-aware Module (FPBM). The TCM aggregates the temporal context information and provides features for the IBM and the FPBM. The IBM generates multi-scale video features to predict action results coarsely. Compared with IBM, the FPBM focuses on instance features corresponding to action predictions and uses more supervision information for boundary regression. Given action results from IBM, the FPBM uses a progressive boosting strategy to refine the boundary predictions multiple times with supervision from weak to strong. Extensive experiments on three benchmark datasets THUMOS14, ActivityNet-v1.3 and HACS show our PPBNet outperforms all existing anchor-free methods. Further, our PPBNet achieves state-of-the-art performance (72.5% mAP at tIoU 
 0.5) on THUMOS14 dataset.
Temporal Action Proposal Generation With Action Frequency Adaptive Network
Authors: Yepeng Tang; Weining Wang; Chunjie Zhang; Jing Liu; Yao Zhao
Date: 2023-06-23 00:00:00
Proceeding: journal 2023 6
Tasks: Knowledge Distillation, Temporal Action Proposal Generation, Video Understanding

As the cornerstone of human-behavior analysis in video understanding, temporal action proposal generation aims to predict the starting and ending time of human action instances in untrimmed videos. Although large achievements in temporal action proposal generation have been achieved, most previous studies ignore the variability of action frequency in raw videos, leading to unsatisfying performances on high-action-frequency videos. In fact, there exists two main issues which should be well addressed: data imbalance between high and low action-frequency videos, and inferior detection of short actions in high-action-frequency videos. To address the above issues, we propose an effective framework by adapting to the variability of action frequency, namely Action Frequency Adaptive Network (AFAN), which can be flexibly built upon any temporal action proposal generation method. AFAN consists of two modules: Learning From Experts (LFE) and Fine-Grained Processing (FGP). The LFE first trains a series of action proposal generators on different subsets of imbalanced data as experts and then teaches a unified student model via knowledge distillation. To better detect short actions, FGP first finds out high-action-frequency videos and then performs fine-grained detection. Extensive experimental results on four benchmark datasets (ActivityNet-1.3, HACS, THUMOS14 and FineAction) demonstrate the effectiveness and generalizability of the proposed AFAN, especially for high-action-frequency videos.
Dynamic Weighted Adversarial Learning for Semi-Supervised Classification under Intersectional Class Mismatch
Authors: Mingyu Li, Tao Zhou, Zhuo Huang, Jian Yang, Jie Yang, Chen Gong
Date: 2024-01-11 00:00:00
Proceeding: ACM Transactions on Multimedia Computing, Communications, and Applications 2024 1
Tasks: Domain Adaptation

Nowadays, class-mismatch problem has drawn intensive attention in Semi-Supervised Learning (SSL), where the classes of labeled data are assumed to be only a subset of the classes of unlabeled data. However, in a more realistic scenario, the labeled data and unlabeled data often share some common classes while they also have their individual classes, which leads to an “intersectional class-mismatch” problem. As a result, existing SSL methods are often confused by these individual classes and suffer from performance degradation. To address this problem, we propose a novel Dynamic Weighted Adversarial Learning (DWAL) framework to properly utilize unlabeled data for boosting the SSL performance. Specifically, to handle the influence of the individual classes in unlabeled data (i.e., Out-Of-Distribution classes), we propose an enhanced adversarial domain adaptation to dynamically assign weight for each unlabeled example from the perspectives of domain adaptation and a class-wise weighting mechanism, which consists of transferability score and prediction confidence value. Besides, to handle the influence of the individual classes in labeled data (i.e., private classes), we propose a dissimilarity maximization strategy to suppress the inaccurate correlations caused by the examples of individual classes within labeled data. Therefore, our DWAL can properly make use of unlabeled data to acquire an accurate SSL classifier under intersectional class-mismatch setting, and extensive experimental results on five public datasets demonstrate the effectiveness of the proposed model over other state-of-the-art SSL methods.
Mispronunciation detection and diagnosis using deep neural networks: a systematic review
Authors: Meriem Lounis, Bilal Dendani, Halima Bahi
Date: 2024-01-09 00:00:00
Proceeding: journal 2024 1
Tasks: Deep Learning, Systematic Literature Review

The increased need for foreign language learning, along with advances in speech technology have heightened interest in computer-assisted pronunciation teaching (CAPT) applications. Herein, the automatic diagnosis of pronunciation errors is essential, it allows language learners to identify their mispronunciations and thus improve their oral skills. Meanwhile, the emergence of deep learning algorithms for speech processing led to the use of deep neural networks at several stages of the mispronunciation detection and diagnosis process. Therefore, an overview of the state-of-the-art in deep learning algorithms for mispronunciation diagnosis is needed, for which we performed a systematic literature review. This study aims to provide an overview of the recent use of deep neural networks for mispronunciation detection and diagnosis (MDD). A thorough statistical analysis is provided in this review which was conducted by extracting specific information from 53 papers published between the years 2015 and 2023. This review indicates that the diagnosis of pronunciation errors is a highly active area of research. Quite a few deep learning models and approaches have been proposed in this area, but there are still some important open issues and limitations to be addressed in future works.