Reviewerly/exHarmony huggingface dataset:
Design of a Solar System UAV

The objective of the project was to design a solar-powered unmanned aerial vehicle (SPUAV) system. The concept of SPUAVs simply stands for Unmanned Aerial Vehicle (UAV) covered with solar cells and uses solar energy as the only power source. During the daytime flight, the cells get solar energy from the sun and then convert it into electrical energy that is used to power the engine, and the excess electrical energy is stored in the battery for the night flight. The solar system was installed on the UAV airframe, which had a specially designed flying wing. The goal was to double the endurance of any UAV after adding solar cells (SCs) and flying at an altitude of 100 m and 35 km/h. This study was conducted using the eCalc website and mathematical equations as well as MATLAB and Multisim software to evaluate the system’s performance. By providing the SPUAV with a camera, this project can be used to monitor illegal immigration attempts across the Omani borders, and to track missing persons at sea or in the mountains, since these tasks require continuous flight operations.
RV-VAE: Integrating Random Variable Algebra into Variational Autoencoders

Among deep generative models, variational autoencoders (VAEs) are a central approach in generating new samples from a learned, latent space while effectively reconstructing input data. The original formulation requires a stochastic sampling operation, implemented via the reparameterization trick, to approximate a posterior latent distribution. In this paper, we introduce a novel approach that leverages the full distributions of encoded input to optimize the model over the entire range of the data, instead of discrete samples. We treat the encoded distributions as continuous random variables and use operations defined by the algebra of random variables during decoding. This approach integrates an innate mathematical prior into the model, helping to improve data efficiency and reduce computational load. Experimental results across different datasets and architectures confirm that this modification enhances VAE-based architectures’ performance. Specifically, our approach improves the reconstruction error and generative capabilities of several VAE architectures, as measured by the Fréchet Inception Distance (FID) metric, while exhibiting similar or better training convergence behavior. Our method exemplifies the power of combining deep learning with inductive priors, promoting data efficiency and less reliance on brute-force learning. Code available at https://github.com/VassilisCN/RV-VAE.
Securing NextG Systems against Poisoning Attacks on Federated Learning: A Game-Theoretic Solution

This paper studies the poisoning attack and defense interactions in a federated learning (FL) system, specifically in the context of wireless signal classification using deep learning for next-generation (NextG) communications. FL collectively trains a global model without the need for clients to exchange their data samples. By leveraging geographically dispersed clients, the trained global model can be used for incumbent user identification, facilitating spectrum sharing. However, in this distributed learning system, the presence of malicious clients introduces the risk of poisoning the training data to manipulate the global model through falsified local model exchanges. To address this challenge, a proactive defense mechanism is employed in this paper to make informed decisions regarding the admission or rejection of clients participating in FL systems. Consequently, the attack-defense interactions are modeled as a game, centered around the underlying admission and poisoning decisions. First, performance bounds are established, encompassing the best and worst strategies for attackers and defenders. Subsequently, the attack and defense utilities are characterized within the Nash equilibrium, where no player can unilaterally improve its performance given the fixed strategies of others. The results offer insights into novel operational modes that safeguard FL systems against poisoning attacks by quantifying the performance of both attacks and defenses in the context of NextG communications.
Adversarial Attacks on LoRa Device Identification and Rogue Signal Detection with Deep Learning

Low-Power Wide-Area Network (LPWAN) technologies, such as LoRa, have gained significant attention for their ability to enable long-range, low-power communication for Internet of Things (IoT) applications. However, the security of LoRa networks remains a major concern, particularly in scenarios where device identification and classification of legitimate and spoofed signals are crucial. This paper studies a deep learning framework to address these challenges, considering LoRa device identification and legitimate vs. rogue LoRa device classification tasks. A deep neural network (DNN), either a convolutional neural network (CNN) or feedforward neural network (FNN), is trained for each task by utilizing real experimental I/Q data for LoRa signals, while rogue signals are generated by using kernel density estimation (KDE) of received signals by rogue devices. Fast Gradient Sign Method (FGSM)-based adversarial attacks are considered for LoRa signal classification tasks using deep learning models. The impact of these attacks is assessed on the performance of two tasks, namely device identification and legitimate vs. rogue device classification, by utilizing separate or common perturbations against these signal classification tasks. Results presented in this paper quantify the level of transferability of adversarial attacks on different LoRa signal classification tasks as a major vulnerability and highlight the need to make IoT applications robust to adversarial attacks.
FArMARe: a Furniture-Aware Multi-task methodology for Recommending Apartments based on the user interests

Nowadays, many people frequently have to search for new accommodation options. Searching for a suitable apartment is a time-consuming process, especially because visiting them is often mandatory to assess the truthfulness of the advertisements found on the Web. While this process could be alleviated by visiting the apartments in the metaverse, the Web-based recommendation platforms are not suitable for the task. To address this shortcoming, in this paper, we define a new problem called text-to-apartment recommendation, which requires ranking the apartments based on their relevance to a textual query expressing the user’s interests. To tackle this problem, we introduce FArMARe, a multi-task approach that supports cross-modal contrastive training with a furniture-aware objective. Since public datasets related to indoor scenes do not contain detailed descriptions of the furniture, we collect and annotate a dataset comprising more than 6000 apartments. A thorough experimentation with three different methods and two raw feature extraction procedures reveals the effectiveness of FArMARe in dealing with the problem at hand.
A Simple and Explainable Method for Uncertainty Estimation using Attribute Prototype Networks

Deep learning’s utility in applications like medical diagnosis, autonomous driving, and natural language processing often hinges on the accurate estimation of uncertainty. Yet, conventional methods for uncertainty estimation face challenges, including high computational cost, difficulties with scalability, or poor interpretability. This paper presents a novel approach to uncertainty estimation using Attribute Prototype Networks (APNs), a method designed for learning robust and interpretable data representations. By leveraging prototype similarity scores, we propose a straightforward way to quantify the uncertainty of predictions, providing explainability and introducing a new technique for detecting out-of-distribution samples based on the distance to the nearest prototype. Our experiments demonstrate that this method offers valuable uncertainty information across several datasets. Our research opens up a new avenue for uncertainty estimation in deep learning, providing a simpler and more explainable solution.
Distilling Part-whole Hierarchical Knowledge from a Huge Pretrained Class Agnostic Segmentation Framework

We propose a novel approach for distilling visual knowledge from a large-scale pre-trained segmentation model, namely, the Segment Anything Model (SAM). Our goal is to pre-train the Agglomerator, a recently introduced column-style network architecture inspired by the organization of neurons in the Neocortex, to learn part-whole hierarchies in images. Despite its biological plausibility, we find that the original pre-training strategy of the Agglomerator, using supervised contrastive loss, fails to work effectively with natural images. To address this, we introduce a new pre-training strategy that aims to instill the model with prior knowledge of the compositional nature of our world. Our approach involves dividing the input image into patches and using the center point of each patch to generate segmentation masks through SAM. SAM produces three results per point to handle ambiguity at the whole, part, and sub-part levels. We then train a simple encoder to utilize the intermediate feature maps of the Agglomerator and reconstruct the embeddings of the masks. This forces the network’s intermediate features to learn objects and their constituent parts. By employing our pre-training strategy, we significantly enhance the classification performance on Imagenette, achieving an accuracy improvement from 58.6% to 91.2% without relying on any augmentation. Remarkably, we achieve this with a minimal parameter count of only 3.2 million, which is approximately 54 times smaller than the originally proposed Agglomerator. These results demonstrate both exceptional data and resource efficiency. Our code is available at: https://github.com/AhmedMostafaSoliman/distill-part-whole
Diff3DHPE: A Diffusion Model for 3D Human Pose Estimation

Accurately estimating 3D human pose (3D HPE) and joint locations using only 2D keypoints is challenging. The noise in the predictions produced by conventional 2D human pose estimators often impeded the accuracy. In this paper, we present a diffusion-based model for 3D pose estimation, named Diff3DHPE, inspired by diffusion models’ noise distillation abilities. The proposed model takes a temporal sequence of 2D keypoints as the input of a GNN backbone model to extract the 3D pose from Gaussian noise using a diffusion process during training. The model then refines it using a reverse diffusion process. To overcome over-smoothing issues in GNNs, Diff3DHPE is integrated with a discretized partial differential equation, which makes it a particular form of Graph Neural Diffusion (GRAND). Extensive experiments show that our model outperforms current state-of-the-art methods on two benchmark datasets, Human3.6M and MPI-INF-3DHP, achieving up to 39.1% improvement in MPJPE on MPI-INF-3DHP. The code is available at https://github.com/socoolzjm/Diff3DHPE.
MCRFF: A Meta-Contrastive Learning-Based RF Fingerprinting Method

Radio frequency fingerprinting (RFF) is used for wireless device identification based on the hardware impairments of the transmitter. Recent works extensively adopt deep learning models to perform RFF, which train the model with data collected from some devices (seen devices), and the trained model can then perform classification. The performance of these methods needs to be improved. More importantly, new unseen devices may often be added to the network. Unfortunately, the performance of existing methods for detecting and classifying unseen devices is often unsatisfactory. In this paper, we propose a meta-contrastive learning-based RFF method (MCRFF). As a model-agnostic training procedure, MCRFF can be applied to almost any RFF model, which can improve the performance of the model for seen devices while especially improving the performance of the model for unseen devices. Inspired by recent advances in contrastive learning and meta-learning, MCRFF enables models to learn to extract similarity-based discriminative representations and to coordinately improve the model’s performance for both seen and unseen devices. We conduct extensive experiments on LoRa and WiFi devices to demonstrate the effectiveness of MCRFF. Moreover, we explore replacing existing models with a lightweight model that has an order of magnitude less computational cost, yet achieves better performance with the help of MCRFF.
Logarithm-transform aided Gaussian Sampling for Few-Shot Learning

Few-shot image classification has recently witnessed the rise of representation learning being utilised for models to adapt to new classes using only a few training examples. Therefore, the properties of the representations, such as their underlying probability distributions, assume vital importance. Representations sampled from Gaussian distributions have been used in recent works, [19] to train classifiers for few-shot classification. These methods rely on transforming the distributions of experimental data to approximate Gaussian distributions for their functioning. In this paper, I propose a novel Gaussian transform, that outperforms existing methods on transforming experimental data into Gaussian-like distributions. I then utilise this novel transformation for few-shot image classification and show significant gains in performance, while sampling lesser data.
Is context all you need? Scaling Neural Sign Language Translation to Large Domains of Discourse

Sign Language Translation (SLT) is a challenging task that aims to generate spoken language sentences from sign language videos, both of which have different grammar and word/gloss order. From a Neural Machine Translation (NMT) perspective, the straightforward way of training translation models is to use sign language phrase-spoken language sentence pairs. However, human interpreters heavily rely on the context to understand the conveyed information, especially for sign language interpretation, where the vocabulary size may be significantly smaller than their spoken language equivalent.Taking direct inspiration from how humans translate, we propose a novel multi-modal transformer architecture that tackles the translation task in a context-aware manner, as a human would. We use the context from previous sequences and confident predictions to disambiguate weaker visual cues. To achieve this we use complementary transformer encoders, namely: (1) A Video Encoder, that captures the low-level video features at the frame-level, (2) A Spotting Encoder, that models the recognized sign glosses in the video, and (3) A Context Encoder, which captures the context of the preceding sign sequences. We combine the information coming from these encoders in a final transformer decoder to generate spoken language translations.We evaluate our approach on the recently published large-scale BOBSL dataset, which contains ~1.2M sequences, and on the SRF dataset, which was part of the WMT-SLT 2022 challenge. We report significant improvements on state-of-the-art translation performance using contextual information, nearly doubling the reported BLEU-4 scores of baseline approaches.
Transformer-Based Sensor Fusion for Autonomous Driving: A Survey

Sensor fusion is an essential topic in many perception systems, such as autonomous driving and robotics. According to the dataset leaderboards, the transformers-based detection head and CNN-based feature encoder to extract features from raw sensor data has emerged as one of the top performing sensor-fusion 3D-detection-framework. In this work, we provide an in-depth literature survey of transformer-based 3D-object detection tasks in the recent past, primarily focusing on sensor fusion. We also briefly review the Vision Transformers (ViT) basics so readers can easily follow through with the paper. Moreover, we also briefly go through a few non-transformer-based, less-dominant methods for sensor fusion for autonomous driving. In conclusion, we summarize the role that transformers play in the domain of sensor fusion and also provoke future research in the field.
Are current long-term video understanding datasets long-term?

Many real-world applications, from sport analysis to surveillance, benefit from automatic long-term action recognition. In the current deep learning paradigm for automatic action recognition, it is imperative that models are trained and tested on datasets and tasks that evaluate if such models actually learn and reason over long-term information. In this work, we propose a method to evaluate how suitable a video dataset is to evaluate models for long-term action recognition. To this end, we define a long-term action as excluding all the videos that can be correctly recognized using solely short-term information. We test this definition on existing long-term classification tasks on three popular real-world datasets, namely Breakfast, CrossTask and LVU, to determine if these datasets are truly evaluating long-term recognition. Our study reveals that these datasets can be effectively solved using shortcuts based on short-term information. Following this finding, we encourage long-term action recognition researchers to make use of datasets that need long-term information to be solved.
Novel Nonlinear Neural-Network Layers for High Performance and Generalization in Modulation-Recognition Applications

The paper presents a novel type of capsule network (CAP) that uses custom-defined neural network (NN) layers for blind classification of digitally modulated signals using their in-phase/quadrature (I/Q) components. The custom NN layers of the CAP are inspired by cyclostationary signal processing (CSP) techniques and implement feature extraction capabilities that are akin to the calculation of the cyclic cumulant (CC) features employed in conventional CSP-based approaches to blind modulation classification and signal identification. The classification performance and the generalization abilities of the proposed CAP are tested using two distinct datasets that contain similar classes of digitally modulated signals but that have been generated independently, and numerical results obtained reveal that the proposed CAP with novel NN feature extraction layers achieves high classification accuracy while also outperforming alternative deep learning (DL)-based approaches for signal classification in terms of both classification accuracy and generalization abilities.
Zero-Shot Dynamic Neural Network Adaptation in Tactical Wireless Systems

Models based on deep neural networks have attracted interest in a myriad of applications in the wireless landscape, including spectrum intelligence, modulation recognition, and radio fingerprinting, among others. One key challenge that is currently inhibiting the application of such models in real-world tactical scenarios is that their performance radically changes with continuously changing dynamic wireless channel conditions. Existing work tackle this by performing efficient fine-tuning or adaptation using as few samples as possible. However, some prior knowledge about the new conditions is involved in the process. In this paper, we propose for the first time zero-shot dynamic neural network adaptation (zDNA) (i.e., without any additional training samples) of wireless classification models after deployment in unseen conditions. Specifically, we show that by changing only the affine transformation parameters and normalizing the learned features in different layers of the neural network in an online manner without any labeled samples, we can achieve superior performance in dynamic conditions. Our proposed approach is evaluated on the publicly available RadioML 2018.01A dataset, to test its adaptability to dynamically changing signal-to-noise ratio (SNR) conditions. Performance improvement consistency in all unseen test scenarios (up to 24% in low SNR regime) demonstrates the applicability of our framework in real-world contexts.
Denoising Diffusion for 3D Hand Pose Estimation from Images

Hand pose estimation from a single image has many applications. However, approaches to full 3D body pose estimation are typically trained on day-to-day activities or actions. As such, detailed hand-to-hand interactions are poorly represented, especially during motion. We see this in the failure cases of techniques such as OpenPose [6] or MediaPipe[30]. However, accurate hand pose estimation is crucial for many applications where the global body motion is less important than accurate hand pose estimation.This paper addresses the problem of 3D hand pose estimation from monocular images or sequences. We present a novel end-to-end framework for 3D hand regression that employs diffusion models that have shown excellent ability to capture the distribution of data for generative purposes. Moreover, we enforce kinematic constraints to ensure realistic poses are generated by incorporating an explicit forward kinematic layer as part of the network. The proposed model provides state-of-the-art performance when lifting a 2D single-hand image to 3D. However, when sequence data is available, we add a Transformer module over a temporal window of consecutive frames to refine the results, overcoming jittering and further increasing accuracy.The method is quantitatively and qualitatively evaluated showing state-of-the-art robustness, generalization, and accuracy on several different datasets.
Self-training and multi-task learning for limited data: evaluation study on object detection

Self-training allows a network to learn from the predictions of a more complicated model, thus often requires well-trained teacher models and mixture of teacher-student data while multi-task learning jointly optimizes different targets to learn salient interrelationship and requires multi-task annotations for each training example. These frameworks, despite being particularly data demanding have potentials for data exploitation if such assumptions can be relaxed. In this paper, we compare self-training object detection under the deficiency of teacher training data where students are trained on unseen examples by the teacher, and multi-task learning with partially annotated data, i.e. single-task annotation per training example. Both scenarios have their own limitation but potentially helpful with limited annotated data. Experimental results show the improvement of performance when using a weak teacher with unseen data for training a multi-task student. Despite the limited setup we believe the experimental results show the potential of multitask knowledge distillation and self-training, which could be beneficial for future study. Source code and data splits are at https://lhoangan.github.io/multas
Progressive Feature Adjustment for Semi-supervised Learning from Pretrained Models

As an effective way to alleviate the burden of data annotation, semi-supervised learning (SSL) provides an attractive solution due to its ability to leverage both labeled and unlabeled data to build a predictive model. While significant progress has been made recently, SSL algorithms are often evaluated and developed under the assumption that the network is randomly initialized. This is in sharp contrast to most vision recognition systems that are built from fine-tuning a pretrained network for better performance. While the marriage of SSL and a pretrained model seems to be straightforward, recent literature suggests that naively applying state-of-the-art SSL with a pretrained model fails to unleash the full potential of training data. In this paper, we postulate the underlying reason is that the pretrained feature representation could bring a bias inherited from the source data, and the bias tends to be magnified through the self-training process in a typical SSL algorithm. To overcome this issue, we propose to use pseudo-labels from the unlabelled data to update the feature extractor that is less sensitive to incorrect labels and only allow the classifier to be trained from the labeled data. More specifically, we progressively adjust the feature extractor to ensure its induced feature distribution maintains a good class separability even under strong input perturbation. Through extensive experimental studies, we show that the proposed approach achieves superior performance over existing solutions.