pwc-archive/papers-with-abstracts huggingface dataset:
An Omnidirectional Approach to Touch-based Continuous Authentication
Authors: Peter Aaby, Mario Valerio Giuffrida, William J Buchanan, Zhiyuan Tan
Date: 2023-01-13 00:00:00

This paper focuses on how touch interactions on smartphones can provide a continuous user authentication service through behaviour captured by a touchscreen. While efforts are made to advance touch-based behavioural authentication, researchers often focus on gathering data, tuning classifiers, and enhancing performance by evaluating touch interactions in a sequence rather than independently. However, such systems only work by providing data representing distinct behavioural traits. The typical approach separates behaviour into touch directions and creates multiple user profiles. This work presents an omnidirectional approach which outperforms the traditional method independent of the touch direction - depending on optimal behavioural features and a balanced training set. Thus, we evaluate five behavioural feature sets using the conventional approach against our direction-agnostic method while testing several classifiers, including an Extra-Tree and Gradient Boosting Classifier, which is often overlooked. Results show that in comparison with the traditional, an Extra-Trees classifier and the proposed approach are superior when combining strokes. However, the performance depends on the applied feature set. We find that the TouchAlytics feature set outperforms others when using our approach when combining three or more strokes. Finally, we highlight the importance of reporting the mean area under the curve and equal error rate for single-stroke performance and varying the sequence of strokes separately.
Sanity checks and improvements for patch visualisation in prototype-based image classification
Authors: Romain Xu-Darme, Georges Quénot, Zakaria Chihani, Marie-Christine Rousset
Date: 2023-01-20 00:00:00
Tasks: image-classification, Image Classification

In this work, we perform an in-depth analysis of the visualisation methods implemented in two popular self-explaining models for visual classification based on prototypes - ProtoPNet and ProtoTree. Using two fine-grained datasets (CUB-200-2011 and Stanford Cars), we first show that such methods do not correctly identify the regions of interest inside of the images, and therefore do not reflect the model behaviour. Secondly, using a deletion metric, we demonstrate quantitatively that saliency methods such as Smoothgrads or PRP provide more faithful image patches. We also propose a new relevance metric based on the segmentation of the object provided in some datasets (e.g. CUB-200-2011) and show that the imprecise patch visualisations generated by ProtoPNet and ProtoTree can create a false sense of bias that can be mitigated by the use of more faithful methods. Finally, we discuss the implications of our findings for other prototype-based models sharing the same visualisation method.
Effective Data Augmentation With Diffusion Models
Authors: Brandon Trabucco, Kyle Doherty, Max Gurinas, Ruslan Salakhutdinov
Date: 2023-02-07 00:00:00
Tasks: Data Augmentation, Diversity, Few-Shot Image Classification, image-classification, Image Classification, Representation Learning

Data augmentation is one of the most prevalent tools in deep learning, underpinning many recent advances, including those from classification, generative models, and representation learning. The standard approach to data augmentation combines simple transformations like rotations and flips to generate new images from existing ones. However, these new images lack diversity along key semantic axes present in the data. Current augmentations cannot alter the high-level semantic attributes, such as animal species present in a scene, to enhance the diversity of data. We address the lack of diversity in data augmentation with image-to-image transformations parameterized by pre-trained text-to-image diffusion models. Our method edits images to change their semantics using an off-the-shelf diffusion model, and generalizes to novel visual concepts from a few labelled examples. We evaluate our approach on few-shot image classification tasks, and on a real-world weed recognition task, and observe an improvement in accuracy in tested domains.
Deep Multi-stream Network for Video-based Calving Sign Detection
Authors: Ryosuke Hyodo, Teppei Nakano, Tetsuji Ogawa
Date: 2023-01-10 00:00:00
Tasks: Attribute, Management

We have designed a deep multi-stream network for automatically detecting calving signs from video. Calving sign detection from a camera, which is a non-contact sensor, is expected to enable more efficient livestock management. As large-scale, well-developed data cannot generally be assumed when establishing calving detection systems, the basis for making the prediction needs to be presented to farmers during operation, so black-box modeling (also known as end-to-end modeling) is not appropriate. For practical operation of calving detection systems, the present study aims to incorporate expert knowledge into a deep neural network. To this end, we propose a multi-stream calving sign detection network in which multiple calving-related features are extracted from the corresponding feature extraction networks designed for each attribute with different characteristics, such as a cow's posture, rotation, and movement, known as calving signs, and are then integrated appropriately depending on the cow's situation. Experimental comparisons conducted using videos of 15 cows demonstrated that our multi-stream system yielded a significant improvement over the end-to-end system, and the multi-stream architecture significantly contributed to a reduction in detection errors. In addition, the distinctive mixture weights we observed helped provide interpretability of the system's behavior.
Unpaired Image-to-Image Translation with Limited Data to Reveal Subtle Phenotypes
Authors: Anis Bourou, Auguste Genovesio
Date: 2023-01-21 00:00:00
Tasks: Image-to-Image Translation, Translation

Unpaired image-to-image translation methods aim at learning a mapping of images from a source domain to a target domain. Recently, these methods proved to be very useful in biological applications to display subtle phenotypic cell variations otherwise invisible to the human eye. However, current models require a large number of images to be trained, while mostmicroscopy experiments remain limited in the number of images they can produce. In this work, we present an improved CycleGAN architecture that employs self-supervised discriminators to alleviate the need for numerous images. We demonstrate quantitatively and qualitatively that the proposed approach outperforms the CycleGAN baseline, including when it is combined with differentiable augmentations. We also provide results obtained with small biological datasets on obvious and non-obvious cell phenotype variations, demonstrating a straightforward application of this method.
Rapid-Motion-Track: Markerless Tracking of Fast Human Motion with Deeper Learning
Authors: Renjie Li, Chun Yu Lao, Rebecca St. George, Katherine Lawler, Saurabh Garg, Son N. Tran, Quan Bai, Jane Alty
Date: 2023-01-18 00:00:00
Tasks: Rhythm

Objective The coordination of human movement directly reflects function of the central nervous system. Small deficits in movement are often the first sign of an underlying neurological problem. The objective of this research is to develop a new end-to-end, deep learning-based system, Rapid-Motion-Track (RMT) that can track the fastest human movement accurately when webcams or laptop cameras are used. Materials and Methods We applied RMT to finger tapping, a well-validated test of motor control that is one of the most challenging human motions to track with computer vision due to the small keypoints of digits and the high velocities that are generated. We recorded 160 finger tapping assessments simultaneously with a standard 2D laptop camera (30 frames/sec) and a high-speed wearable sensor-based 3D motion tracking system (250 frames/sec). RMT and a range of DLC models were applied to the video data with tapping frequencies up to 8Hz to extract movement features. Results The movement features (e.g. speed, rhythm, variance) identified with the new RMT system exhibited very high concurrent validity with the gold-standard measurements (97.3\% of RMT measures were within +/-0.5Hz of the Optotrak measures), and outperformed DLC and other advanced computer vision tools (around 88.2\% of DLC measures were within +/-0.5Hz of the Optotrak measures). RMT also accurately tracked a range of other rapid human movements such as foot tapping, head turning and sit-to -stand movements. Conclusion: With the ubiquity of video technology in smart devices, the RMT method holds potential to transform access and accuracy of human movement assessment.
Bangla sign digits recognition using depth information
Authors: S. M. Rayeed, Gazi Wasif Akram, Sidratul Tamzida Tuba, Golam Sadman Zilani, Hasan Mahmud, Md. Kamrul Hasan
Date: 2022-03-04 00:00:00
Proceeding: International Conference on Machine Vision 2022 3
Tasks: Sign Language Recognition

Sign Language Recognition (SLR) targets interpreting sign language into text or speech, so as to facilitate communication between deaf-mute people and ordinary people. The task has a broad social impact but is still very challenging due to the complexity and large variations in hand actions. The existing dataset for Sign Language Recognition (SLR) in Bangla Sign Language (BdSL) is based on RGB images. Recent research on sign language recognition has shown better recognition accuracy using depth-based features. In this paper, we present a complete dataset for Bangla sign digits from Zero (Shunno in Bangla) to Nine (Noy in Bangla) using MediaPipe, a cross-platform depth-map estimation framework. The proposed method can utilize hand skeleton joint points containing depth information in addition to x, and y coordinates from RGB images only. To validate the effectiveness of our proposed approach, we have run MediaPipe on a benchmark American Sign Language (ASL) dataset. Running different classifiers in our proposed dataset we got 98.65% using Support Vector Machine (SVM). Moreover, we compared our dataset with the existing Bangla digit dataset Ishara Bochon using a deep learning-based approach and achieved significantly higher accuracy.
Fine-Grained Visual Classification via Internal Ensemble Learning Transformer
Authors: Qin Xu, Jiahui Wang, Bo Jiang, Bin Luo
Date: 2023-02-13 00:00:00
Proceeding: IEEE Transactions on Multimedia 2023 2
Tasks: Classification, Ensemble Learning, Fine-Grained Image Classification, Fine-Grained Visual Recognition

Recently, vision transformers (ViTs) have been investigated in fine-grained visual recognition (FGVC) and are now
considered state of the art. However, most ViT-based works ignore
the different learning performances of the heads in the multihead self-attention (MHSA) mechanism and its layers. To address
these issues, in this paper, we propose a novel internal ensemble
learning transformer (IELT) for FGVC. The proposed IELT
involves three main modules: multi-head voting (MHV) module,
cross-layer refinement (CLR) module, and dynamic selection (DS)
module. To solve the problem of the inconsistent performances of
multiple heads, we propose the MHV module, which considers
all of the heads in each layer as weak learners and votes for
tokens of discriminative regions as cross-layer feature based on
the attention maps and spatial relationships. To effectively mine
the cross-layer feature and suppress the noise, the CLR module
is proposed, where the refined feature is extracted and the assist
logits operation is developed for the final prediction. In addition,
a newly designed DS module adjusts the token selection number
at each layer by weighting their contributions of the refined
feature. In this way, the idea of ensemble learning is combined
with the ViT to improve fine-grained feature representation. The
experiments demonstrate that our method achieves competitive
results compared with the state of the art on five popular FGVC
datasets. Source code has been released and can be found at
https://github.com/mobulan/IELT.
Neutrosophic set based local binary pattern for texture classification
Authors: Nuh Alpaslan
Date: 2022-12-15 00:00:00
Proceeding: Expert Systems with Applications 2022 12
Tasks: Classification, Texture Classification

This paper presents novel neutrosophic set-based Completed Local Binary Pattern (CLBP) hybrid methods. These methods first transform the input image into a neutrosophic domain, and the image's texture is characterized by truth and false membership components. Grayscale images are more robust to noise in the neutrosophic domain. The neutrosophic components suppress noise, by this way edges in the images may be detected more precisely. As a result, using neutrosophic truth and false sets with a grayscale input image has resulted in more robust features. Besides, these discriminative features are combined with rotation invariant LBP features with a cross-scale joint coding strategy. The proposed methods can contribute to the classification performance with a reasonable computational cost. When compared to state-of-the-art hand-crafted approaches, experimental findings on state-of-the-art texture databases show that the proposed methods can enhance classification accuracy by roughly 24%. Moreover, the proposed method achieves up to 34% better results than the state-of-the-art deep learning-based methods.
Learning Pseudo Labels for Semi-and-Weakly Supervised Semantic Segmentation
Authors: Yude Wang, Jie Zhang, Meina Kan, Shiguang Shan
Date: 2022-08-02 00:00:00
Proceeding: Pattern Recognition 2022 8
Tasks: Pseudo Label, Semantic Segmentation, Semi-Supervised Semantic Segmentation, Weakly supervised Semantic Segmentation, Weakly-Supervised Semantic Segmentation

In this paper, we aim to tackle semi-and-weakly supervised semantic segmentation (SWSSS), where many image-level classification labels and a few pixel-level annotations are available. We believe the most crucial point for solving SWSSS is to produce high-quality pseudo labels, and our method deals with it from two perspectives. Firstly, we introduce a class-aware cross entropy (CCE) loss for network training. Compared to conventional cross entropy loss, CCE loss encourages the model to distinguish concurrent classes only and simplifies the learning target of pseudo label generation. Secondly, we propose a progressive cross training (PCT) method to build cross supervision between two networks with a dynamic evaluation mechanism, which progressively introduces high-quality predictions as additional supervision for network training. Our method significantly improves the quality of generated pseudo labels in the regime with extremely limited annotations. Extensive experiments demonstrate that our approach outperforms state-of-the-art methods significantly.
Self-FuseNet: Data Free Unsupervised Remote Sensing Image Super-Resolution
Authors: Divya Mishra, Ofer Hadar
Date: 2023-01-23 00:00:00
Proceeding: IEEE Journal 2023 1
Tasks: Blind Super-Resolution, Feature Engineering, Feature Importance, Image Enhancement, Image Reconstruction, Image Super-Resolution, Multi-Exposure Image Fusion, satellite image super-resolution, Super-Resolution, Unsupervised Image-To-Image Translation, Unsupervised Pre-training

Real-world degradations deviate from ideal degradations, as most deep learning-based scenarios involve the ideal synthesis of low-resolution (LR) counterpart images by popularly used bicubic interpolation. Moreover, supervised learning approaches rely on many high-resolution (HR) and LR image pairings to reconstruct missing information based on their association, developed by complex long hours of deep neural network training. Additionally, the trained model's generalizability on various image datasets with various distributions is not guaranteed. To overcome this challenge, we proposed our novel Self-FuseNet, particularly for extremely poor-resolution satellite images. Also, the network exhibits strong generalization performance on additional datasets (both “ideal” and “nonideal” scenarios). The network is especially for those image datasets suffering from the following two significant limitations: 1) nonavailability of ground truth HR images; 2) limitation of a large count of the unpaired dataset for deep neural network training. The benefit of the proposed model is threefold: 1) it does not require any significant extensive training data, either paired or unpaired but only a single LR image without prior knowledge of its distribution; 2) it is a simple and effective model for super-resolving very poor-resolution images, saving computational resources and time; 3) using UNet, the processing of data are accelerated by the network's wide skip connections, allowing image reconstruction with fewer parameters. Rather than using an inverse approach, as common in most deep learning scenarios, we introduced a forward approach to super-resolve exceptionally LR remote sensing images. This demonstrates its supremacy over recently proposed state-of-the-art methods for unsupervised single real-world image blind super-resolution.
On the adequacy of static analysis warnings with respect to code smell prediction
Authors: Fabiano Pecorelli, Savanna Lujan, Valentina Lenarduzzi, Fabio Palomba, Andrea De Lucia
Date: 2022-03-17 00:00:00
Proceeding: Empirical Software Engineering 2022 3

Code smells are poor implementation choices that developers apply while evolving source code and that affect program maintainability. Multiple automated code smell detectors have been proposed: while most of them relied on heuristics applied over software metrics, a recent trend concerns the definition of machine learning techniques. However, machine learning-based code smell detectors still suffer from low accuracy: one of the causes is the lack of adequate features to feed machine learners. In this paper, we face this issue by inves- tigating the role of static analysis warnings generated by three state-of-the-art tools to be used as features of machine learning models for the detection of seven code smell types. We conduct a three-step study in which we (1) verify the relation between static analysis warnings and code smells and the potential predictive power of these warnings; (2) build code smell prediction models exploiting and combining the most relevant features coming from the first analysis; (3) compare and combine the performance of the best code smell prediction model with the one achieved by a state of the art approach. The results reveal the low performance of the models exploiting static analysis warnings alone, while we observe significant improvements when combining the warnings with additional code met- rics. Nonetheless, we still find that the best model does not perform better than a random model, hence leaving open the challenges related to the definition of ad-hoc features for code smell prediction.
Evaluation of Deep Learning Models for Kannada Handwritten Digit Recognition
Authors: Qisheng Hu
Date: 2020-12-09 00:00:00
Proceeding: 01-02 August 2020 12
Tasks: Deep Learning, Handwritten Digit Recognition

Handwritten digit recognition is a basic and important problem in computer vision. While it has been studied intensively, the newer datasets keep bring new challenges to existed solutions. A new Kannada-MNIST dataset is mainly digital images of the Kannada language, which contains 70,000 28×28 gray-scale sample images and 10 classes. It also contains a Dig-MNIST dataset, which is an out-of domain test sample dataset and a more challenging test dataset for recognition. In this paper, we evaluate the state-of-art deep learning models for Kannada-MNIST and improve their performance with data augmentation and transfer learning techniques. We manage to achieve a best accuracy of 90.27% for Dig-MNIST in the literature with Inception Model.
Propagation Measurements and Analyses at 28 GHz via an Autonomous Beam-Steering Platform
Authors: Bharath Keshavamurthy, Yaguang Zhang, Christopher R. Anderson, Nicolo Michelusi, James V. Krogmeier, David J. Love
Date: 2023-02-16 00:00:00

This paper details the design of an autonomous alignment and tracking platform to mechanically steer directional horn antennas in a sliding correlator channel sounder setup for 28 GHz V2X propagation modeling. A pan-and-tilt subsystem facilitates uninhibited rotational mobility along the yaw and pitch axes, driven by open-loop servo units and orchestrated via inertial motion controllers. A geo-positioning subsystem augmented in accuracy by real-time kinematics enables navigation events to be shared between a transmitter and receiver over an Apache Kafka messaging middleware framework with fault tolerance. Herein, our system demonstrates a 3D geo-positioning accuracy of 17 cm, an average principal axes positioning accuracy of 1.1 degrees, and an average tracking response time of 27.8 ms. Crucially, fully autonomous antenna alignment and tracking facilitates continuous series of measurements, a unique yet critical necessity for millimeter wave channel modeling in vehicular networks. The power-delay profiles, collected along routes spanning urban and suburban neighborhoods on the NSF POWDER testbed, are used in pathloss evaluations involving the 3GPP TR38.901 and ITU-R M.2135 standards. Empirically, we demonstrate that these models fail to accurately capture the 28 GHz pathloss behavior in urban foliage and suburban radio environments. In addition to RMS direction-spread analyses for angles-of-arrival via the SAGE algorithm, we perform signal decoherence studies wherein we derive exponential models for the spatial/angular autocorrelation coefficient under distance and alignment effects.
Generative Causal Representation Learning for Out-of-Distribution Motion Forecasting
Authors: Shayan Shirahmad Gale Bagi, Zahra Gharaee, Oliver Schulte, Mark Crowley
Date: 2023-02-17 00:00:00
Tasks: Motion Forecasting, Representation Learning, Trajectory Prediction, Transfer Learning

Conventional supervised learning methods typically assume i.i.d samples and are found to be sensitive to out-of-distribution (OOD) data. We propose Generative Causal Representation Learning (GCRL) which leverages causality to facilitate knowledge transfer under distribution shifts. While we evaluate the effectiveness of our proposed method in human trajectory prediction models, GCRL can be applied to other domains as well. First, we propose a novel causal model that explains the generative factors in motion forecasting datasets using features that are common across all environments and with features that are specific to each environment. Selection variables are used to determine which parts of the model can be directly transferred to a new environment without fine-tuning. Second, we propose an end-to-end variational learning paradigm to learn the causal mechanisms that generate observations from features. GCRL is supported by strong theoretical results that imply identifiability of the causal model under certain assumptions. Experimental results on synthetic and real-world motion forecasting datasets show the robustness and effectiveness of our proposed method for knowledge transfer under zero-shot and low-shot settings by substantially outperforming the prior motion forecasting models on out-of-distribution prediction. Our code is available at https://github.com/sshirahmad/GCRL.