pwc-archive/papers-with-abstracts huggingface dataset:
Effective Document Image Enhancement Using tokens-to-token Transformer Network
Authors: Risab Biswas, Swalpa Kumar Roy, Umapada Pal
Date: 2023-02-10 00:00:00
Proceeding: Preprint 2023 2
Tasks: Binarization, Image Enhancement

Document image enhancement is a fundamental and important stage for attaining the best performance in any document analysis assignment because there are many degradation situations that could harm document images, making it more difficult to recognize and analyze them. In this paper, we propose to employ a Tokens-to-Token Transformer network for document image enhancement, a novel encoder-decoder architecture based on a tokens-to-token vision transformer. The proposed architecture uses a tokens-to-token architecture in the encoder section. Each image is divided into a set of tokens with a defined length using the ViT model, which is then applied several times to model the global relationship between the tokens. However, the conventional tokenization of input data does not adequately reflect the crucial local structure between adjacent pixels of the input image, which results in low efficiency. Instead of using a simple ViT and hard splitting of images for the document image enhancement task, we employed a progressive tokeniza-tion technique to capture this local information from an image for achieving more effective results. Experiments on various DIBCO and H-DIBCO benchmarks demonstrate that the proposed model outperforms the existing CNN and ViT-based state-of-the-art methods. In this research, the primary area of examination is the application of the proposed architecture to the task of document binarization. The source code will be made available at https://github.com/RisabBiswas/T2T-BinFormer.
Local Window Attention Transformer for Polarimetric SAR Image Classification
Authors: Ali Jamali, Swalpa Kumar Roy, Avik Bhattacharya, Pedram Ghamisi
Date: 2023-01-23 00:00:00
Proceeding: IEEE Geoscience and Remote Sensing Letters 2023 1
Tasks: Classification, Earth Observation, image-classification, Image Classification

Convolutional neural networks (CNNs) have recently found great attention in image classification since deep CNNs have exhibited excellent performance in computer vision. Owing to their immense success, of late, scientists are exploring the functionality of transformers in Earth observation applications. Nevertheless, the primary issue with transformers is that they demand significantly more training data than CNN classifiers. Thus, the use of these transformers in remote sensing is considered challenging, notably in utilizing polarimetric SAR (PolSAR) data, due to the insufficient number of existing labeled data. In this letter, we develop and propose a vision transformer-based framework that utilizes 3D and 2D CNNs as feature extractors and, in addition, local window attention for the effective classification of PolSAR data. Extensive experimental results demonstrated that the developed model PolSARFormer obtained better classification accuracy than the state-of-the-art vision Swin Transformer and FNet algorithms. The PolSARFormer outperformed the Swin Transformer and FNet by the margin of 5.86% and 17.63%, in terms of average accuracy in the San Francisco data benchmark. Moreover , the results over the Flevoland dataset illustrated that the PolSARFormer exceeds several other algorithms, including the ResNet (97.49%), Swin Transformer (96.54%), FNet (95.28%), 2D CNN (94.57%), and AlexNet (91.83%), with a kappa index of 99.30%. The code will be made available publicly at https://github.com/aj1365/PolSARFormer.
From Node to Graph: Joint Reasoning on Visual-Semantic Relational Graph for Zero-Shot Detection
Authors: Hui Nie, Ruiping Wang, Xilin Chen
Date: 2022-02-15 00:00:00
Proceeding: Winter Conference on Applications of Computer Vision (WACV) 2022 2
Tasks: Generalized Zero-Shot Object Detection, Scene Understanding, Zero-Shot Object Detection

Zero-Shot Detection (ZSD), which aims at localizing andrecognizing unseen objects in a complicated scene, usuallyleverages  the  visual  and  semantic  information  of  individ-ual  objects  alone.   However,  scene  understanding  of  hu-man exceeds recognizing individual objects separately: thecontextual information among multiple objects such as vi-sual  relational information  (e.g.   visually  similar  objects)and semantic relational information (e.g.  co-occurrences)is  helpful  for  understanding  of  visual  scene.   In  this  pa-per, we verify that contextual information plays a more im-portant  role  in  ZSD  than  in  traditional  object  detection.To  make  full  use  of  such  information,  we  propose  a  newend-to-end ZSD methodGRaphAligningNetwork (GRAN)based on graph modeling and reasoning which simultane-ously considers visual and semantic information of multipleobjects instead of individual objects.  Specifically, we for-mulate a Visual Relational Graph (VRG) and a SemanticRelational Graph (SRG), where the nodes are the objectsin the image and the semantic representations of classes re-spectively and the edges are the relevance between nodesin each graph.  To characterize mutual effect between twomodalities, the two graphs are further merged into a hetero-geneous Visual-Semantic Relational Graph (VSRG), wheremodal translators are designed for the two subgraphs to en-able modal information to transform into a common spacefor communication,  and message passing among nodes isenforced to refine their representations. Comprehensive ex-periments on MSCOCO dataset demonstrate the advantageof our method over state-of-the-arts, and qualitative anal-ysis suggests the validity of using contextual information.
Configurable EBEN: Extreme Bandwidth Extension Network to enhance body-conducted speech capture
Authors: Julien Hauret, Thomas Joubaud, Véronique Zimpfer, Éric Bavu
Date: 2023-03-17 00:00:00
Tasks: Bandwidth Extension, Generative Adversarial Network

This paper presents a configurable version of Extreme Bandwidth Extension Network (EBEN), a Generative Adversarial Network (GAN) designed to improve audio captured with body-conduction microphones. We show that although these microphones significantly reduce environmental noise, this insensitivity to ambient noise happens at the expense of the bandwidth of the speech signal acquired by the wearer of the devices. The obtained captured signals therefore require the use of signal enhancement techniques to recover the full-bandwidth speech. EBEN leverages a configurable multiband decomposition of the raw captured signal. This decomposition allows the data time domain dimensions to be reduced and the full band signal to be better controlled. The multiband representation of the captured signal is processed through a U-Net-like model, which combines feature and adversarial losses to generate an enhanced speech signal. We also benefit from this original representation in the proposed configurable discriminators architecture. The configurable EBEN approach can achieve state-of-the-art enhancement results on synthetic data with a lightweight generator that allows real-time processing.
Feature Selection With Maximal Relevance and Minimal Supervised Redundancy
Authors: Yadi Wang, Xiaoping Li, and Rubén Ruiz
Date: 2022-02-07 00:00:00
Proceeding: IEEE Transactions on Cybernetics 2022 2
Tasks: Classification, feature selection

Feature selection (FS) for classification is crucial for large-scale images and bio-microarray data using machine learning. It is challenging to select informative features from high-dimensional data which generally contains many irrelevant and redundant features. These features often impede classifier performance and misdirect classification tasks. In this article, we
present an efficient FS algorithm to improve classification accuracy by taking into account both the relevance of the features and the pairwise features correlation in regard to class labels. Based on conditional mutual information and entropy, a new supervised similarity measure is proposed. The supervised similarity measure is connected with feature redundancy inimization evaluation and then combined with feature relevance maximization evaluation. A new criterion max-relevance and min-supervisedredundancy (MRMSR) is introduced and theoretically proved for FS. The proposed MRMSR-based ethod is compared to seven existing FS approaches on several frequently studied public benchmark datasets. Experimental results demonstrate that the proposal is more effective at selecting informative features and results in better competitive classification performance.
Discrete Wavelet Transform for Generative Adversarial Network to Identify Drivers Using Gyroscope and Accelerometer Sensors
Authors: Rouhollah Ahmadian, Mehdi Ghatee, Johan Wahlström
Date: 2022-04-01 00:00:00
Proceeding: IEEE Sensors Journal 2022 4
Tasks: Classification, Data Augmentation, Driver Authentication, Driver Identification, Generative Adversarial Network, Image Augmentation

Driver identification is a central research area in intelligent transportation systems, with applications in commercial freight transport and usage-based insurance. One way to perform the identification is to use smartphones as the main sensor devices. After extracting features from smartphone-embedded sensors, various machine learning methods can be used to identify the driver. However, the accuracy often degrades as the number of drivers increases. This paper uses a Generative Adversarial Network (GAN) for data augmentation to obtain a driver identification algorithm that maintains excellent performance also when the number of drivers increases. Since GAN diversifies the drivers’ data, it makes it possible to apply the identification algorithm on a larger number of drivers without overfitting. Although GANs are commonly used in image processing for image augmentation, their use for driving signal augmentation is novel. However, GAN’s training on raw driving signals diverges. This challenge is solved by getting the Discrete Wavelet Transform (DWT) on driving signals before feeding to GAN. Our experiments prove the usefulness of GAN model for generating driving signals emanating from DWT on smartphones’ accelerometer and gyroscope signals. After collecting the augmented data, their histograms along the overlapped windows are fed to machine learning methods covered by a Stacked Generalization Method (SGM). The presented hybrid GAN-SGM approach identifies drivers with 97% accuracy, 98% precision, 97% recall, and 97% F1-measure that outperforms standard machine learning methods that utilize features extracted by the statistical, spectral, and temporal approaches.
A Reinforcement Learning Neural Network for Robotic Manipulator Control
Authors: Yazhou Hu, Bailu Si
Date: 2018-07-01 00:00:00
Proceeding: Conference 2018 7
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We propose a neural network model for reinforcement learning to control a robotic manipulator with unknown parameters and dead zones. The model is composed of three networks. The state of the robotic manipulator is predicted by the state network of the model, the action policy is learned by the action network, and the performance index of the action policy is estimated by a critic network. The three networks work together to optimize the performance index based on the reinforcement learning control scheme. The convergence of the learning methods is analyzed. Application of the proposed model on a simulated two-link robotic manipulator demonstrates the effectiveness and the stability of the model.
A predictive estimator of finite population mean using nonparametric regression
Authors: M. Rueda, R. Sánchez-Borrego
Date: 2008-11-20 00:00:00
Proceeding: Computational Statistics (CompStat) 2008 11
Tasks: regression

The paper considers the problem of estimating the population mean using
auxiliary information. We propose a new model-based estimator of the population
mean, based on local polynomial regression. This estimator exhibits several attractive
properties under the model-based approach. The estimator is compared to a number
of methods which have been proposed in the literature via a simulation study based
on several populations.
Less is More: Reducing Task and Model Complexity for 3D Point Cloud Semantic Segmentation
Authors: Li Li, Hubert P. H. Shum, Toby P. Breckon
Date: 2023-03-20 00:00:00
Proceeding: CVPR 2023 1
Tasks: 3D Semantic Segmentation, Autonomous Driving, Pseudo Label, Segmentation, Semantic Segmentation, Semi-Supervised Semantic Segmentation

Whilst the availability of 3D LiDAR point cloud data has significantly grown in recent years, annotation remains expensive and time-consuming, leading to a demand for semi-supervised semantic segmentation methods with application domains such as autonomous driving. Existing work very often employs relatively large segmentation backbone networks to improve segmentation accuracy, at the expense of computational costs. In addition, many use uniform sampling to reduce ground truth data requirements for learning needed, often resulting in sub-optimal performance. To address these issues, we propose a new pipeline that employs a smaller architecture, requiring fewer ground-truth annotations to achieve superior segmentation accuracy compared to contemporary approaches. This is facilitated via a novel Sparse Depthwise Separable Convolution module that significantly reduces the network parameter count while retaining overall task performance. To effectively sub-sample our training data, we propose a new Spatio-Temporal Redundant Frame Downsampling (ST-RFD) method that leverages knowledge of sensor motion within the environment to extract a more diverse subset of training data frame samples. To leverage the use of limited annotated data samples, we further propose a soft pseudo-label method informed by LiDAR reflectivity. Our method outperforms contemporary semi-supervised work in terms of mIoU, using less labeled data, on the SemanticKITTI (59.5@5%) and ScribbleKITTI (58.1@5%) benchmark datasets, based on a 2.3x reduction in model parameters and 641x fewer multiply-add operations whilst also demonstrating significant performance improvement on limited training data (i.e., Less is More).
Monocular Cyclist Detection with Convolutional Neural Networks
Authors: Charles Tang
Date: 2023-01-16 00:00:00
Tasks: 2D Cyclist Detection, object-detection, Object Detection, Transfer Learning

Cycling is an increasingly popular method of transportation for sustainability and health benefits. However, cyclists face growing risks, especially when encountering large vehicles on the road. This study aims to reduce the number of vehicle-cyclist collisions, which are often caused by poor driver attention to blind spots. To achieve this, we designed a state-of-the-art real-time monocular cyclist detection that can detect cyclists with object detection convolutional neural networks, such as EfficientDet Lite and SSD MobileNetV2. First, our proposed cyclist detection models achieve greater than 0.900 mAP (IoU: 0.5), fine-tuned on a newly proposed cyclist image dataset comprising over 20,000 images. Next, the models were deployed onto a Google Coral Dev Board mini-computer with a camera module and analyzed for speed, reaching inference times as low as 15 milliseconds. Lastly, the end-to-end cyclist detection device was tested in real-time to model traffic scenarios and analyzed further for performance and feasibility. We concluded that this cyclist detection device can accurately and quickly detect cyclists and has the potential to improve cyclist safety significantly. Future studies could determine the feasibility of the proposed device in the vehicle industry and improvements to cyclist safety over time.
On the Convergence of Decentralized Federated Learning Under Imperfect Information Sharing
Authors: Vishnu Pandi Chellapandi, Antesh Upadhyay, Abolfazl Hashemi, Stanislaw H /. Zak
Date: 2023-03-19 00:00:00
Tasks: Distributed Optimization, Federated Learning

Decentralized learning and optimization is a central problem in control that encompasses several existing and emerging applications, such as federated learning. While there exists a vast literature on this topic and most methods centered around the celebrated average-consensus paradigm, less attention has been devoted to scenarios where the communication between the agents may be imperfect. To this end, this paper presents three different algorithms of Decentralized Federated Learning (DFL) in the presence of imperfect information sharing modeled as noisy communication channels. The first algorithm, Federated Noisy Decentralized Learning (FedNDL1), comes from the literature, where the noise is added to their parameters to simulate the scenario of the presence of noisy communication channels. This algorithm shares parameters to form a consensus with the clients based on a communication graph topology through a noisy communication channel. The proposed second algorithm (FedNDL2) is similar to the first algorithm but with added noise to the parameters, and it performs the gossip averaging before the gradient optimization. The proposed third algorithm (FedNDL3), on the other hand, shares the gradients through noisy communication channels instead of the parameters. Theoretical and experimental results demonstrate that under imperfect information sharing, the third scheme that mixes gradients is more robust in the presence of a noisy channel compared with the algorithms from the literature that mix the parameters.
CCTV-Gun: Benchmarking Handgun Detection in CCTV Images
Authors: Srikar Yellapragada, Zhenghong Li, Kevin Bhadresh Doshi, Purva Makarand Mhasakar, Heng Fan, Jie Wei, Erik Blasch, Bin Zhang, Haibin Ling
Date: 2023-03-19 00:00:00
Tasks: Benchmarking, object-detection, Object Detection

Gun violence is a critical security problem, and it is imperative for the computer vision community to develop effective gun detection algorithms for real-world scenarios, particularly in Closed Circuit Television (CCTV) surveillance data. Despite significant progress in visual object detection, detecting guns in real-world CCTV images remains a challenging and under-explored task. Firearms, especially handguns, are typically very small in size, non-salient in appearance, and often severely occluded or indistinguishable from other small objects. Additionally, the lack of principled benchmarks and difficulty collecting relevant datasets further hinder algorithmic development. In this paper, we present a meticulously crafted and annotated benchmark, called \textbf{CCTV-Gun}, which addresses the challenges of detecting handguns in real-world CCTV images. Our contribution is three-fold. Firstly, we carefully select and analyze real-world CCTV images from three datasets, manually annotate handguns and their holders, and assign each image with relevant challenge factors such as blur and occlusion. Secondly, we propose a new cross-dataset evaluation protocol in addition to the standard intra-dataset protocol, which is vital for gun detection in practical settings. Finally, we comprehensively evaluate both classical and state-of-the-art object detection algorithms, providing an in-depth analysis of their generalizing abilities. The benchmark will facilitate further research and development on this topic and ultimately enhance security. Code, annotations, and trained models are available at https://github.com/srikarym/CCTV-Gun.
DataLight: Offline Data-Driven Traffic Signal Control
Authors: Liang Zhang, Yutong Zhang, Jianming Deng, Chen Li
Date: 2023-03-20 00:00:00
Tasks: Offline RL, Reinforcement Learning (RL), Traffic Signal Control

Reinforcement learning (RL) has emerged as a promising solution for addressing traffic signal control (TSC) challenges. While most RL-based TSC systems typically employ an online approach, facilitating frequent active interaction with the environment, learning such strategies in the real world is impractical due to safety and risk concerns. To tackle these challenges, this study introduces an innovative offline data-driven approach, called DataLight. DataLight employs effective state representations and reward function by capturing vehicular speed information within the environment. It then segments roads to capture spatial information and further enhances the spatially segmented state representations with sequential modeling. The experimental results demonstrate the effectiveness of DataLight, showcasing superior performance compared to both state-of-the-art online and offline TSC methods. Additionally, DataLight exhibits robust learning capabilities concerning real-world deployment issues. The code is available at https://github.com/LiangZhang1996/DataLight.
Metric Search for Rank List Compatibility Matching with Applications
Authors: Wenqi Guo, Jeffrey Uhlmann
Date: 2023-03-14 00:00:00

As online dating has become more popular in the past few years, an efficient and effective algorithm to match users is needed. In this project, we proposed a new dating matching algorithm that uses Kendall-Tau distance to measure the similarity between users based on their ranking for items in a list. (e.g., their favourite sports, music, etc.) To increase the performance of the search process, we applied a tree-based searching structure, Cascading Metric Tree (CMT), on this metric. The tree is built on ranked lists from all the users; when a query target and a radius are provided, our algorithm can return users within the radius of the target. We tested the scaling of this searching method on a synthetic dataset by varying list length, population size, and query radius. We observed that the algorithm is able to query the best matching people for the user in a practical time, given reasonable parameters. We also provided potential future improvements that can be made to this algorithm based on the limitations. Finally, we offered more use cases of this search structure on Kendall-Tau distance and new insight into real-world applications of distance search structures.
Self-Supervised Learning for Multimodal Non-Rigid 3D Shape Matching
Authors: Dongliang Cao, Florian Bernard
Date: 2023-03-20 00:00:00
Proceeding: CVPR 2023 1
Tasks: Self-Supervised Learning

The matching of 3D shapes has been extensively studied for shapes represented as surface meshes, as well as for shapes represented as point clouds. While point clouds are a common representation of raw real-world 3D data (e.g. from laser scanners), meshes encode rich and expressive topological information, but their creation typically requires some form of (often manual) curation. In turn, methods that purely rely on point clouds are unable to meet the matching quality of mesh-based methods that utilise the additional topological structure. In this work we close this gap by introducing a self-supervised multimodal learning strategy that combines mesh-based functional map regularisation with a contrastive loss that couples mesh and point cloud data. Our shape matching approach allows to obtain intramodal correspondences for triangle meshes, complete point clouds, and partially observed point clouds, as well as correspondences across these data modalities. We demonstrate that our method achieves state-of-the-art results on several challenging benchmark datasets even in comparison to recent supervised methods, and that our method reaches previously unseen cross-dataset generalisation ability.