Reviewerly/exHarmony huggingface dataset:
Fast converging Federated Learning with Non-IID Data

With the advancement of device capabilities, Internet of Things (IoT) devices can employ built-in hardware to perform machine learning (ML) tasks, extending their horizons in many promising directions. In traditional ML, data are sent to a server for training. However, this approach raises user privacy concerns. On the other hand, transferring user data to a cloud-centric environment results in increased latency. A decentralized ML technique, Federated learning (FL), has been proposed to enable devices to train locally on personal data and then send the data to a server for model aggregation. In these models, malicious devices, or devices with a minor contribution to a global model, increase communication rounds and resource usage. Likewise, heterogeneous data, such as non-independent and identically distributed (Non-IID), may decrease accuracy of the FL model. This paper proposes a mechanism to quantify device contributions based on weight divergence. We propose an outlier-removal approach which identifies irrelevant device updates. Client selection probabilities are computed using a Bayesian model. To obtain a global model, we employ a novel merging algorithm utilizing weight shifting values to ensure convergence towards more accurate predictions. A simulation using the MNIST dataset employing both non-iid and iid devices, distributed on 10 Jetson Nano devices, shows that our approach converges faster, significantly reduces communication cost, and improves accuracy.
Shape and Intensity Analysis of Glioblastoma Multiforme Tumors

We use a geometric approach to characterize tumor shape and intensity along the tumor contour in the context of Glioblastoma Multiforme. Properties of the proposed shape+intensity representation include invariance to translation, scale, rotation and reparameterization, which allow for objective comparison of tumor features. Controlling for the weight of intensity information in the shape+intensity representation results in improved comparisons between tumor features of different patients who have been diagnosed with Glioblastoma Multiforme; further, it allows for identification of different partitions of the data associated with different median survival among such patients. Our findings suggest that integrating and appropriately balancing information regarding GBM tumor shape and intensity can be beneficial for disease prognosis. We evaluate the proposed statistical framework using simulated examples as well as a real dataset of Glioblastoma Multiforme tumors.
Multi-scale Local Implicit Keypoint Descriptor for Keypoint Matching

We investigate the potential of multi-scale descriptors which has been under-explored in the existing literature. At the pixel level, we propose utilizing both coarse and fine-grained descriptors and present a scale-aware method of negative sampling, which trains descriptors at different scales in a complementary manner, thereby improving their discriminative power. For sub-pixel level descriptors, we also propose adopting coordinate-based implicit modeling and learning the non-linearity of local descriptors on continuous-domain coordinates. Our experiments show that the proposed method achieves state-of-the-art performance on various tasks, i.e., image matching, relative pose estimation, and visual localization.
Heterogeneous Generative Dataset for UASes

In this poster, we present the construction of HGDAVE (Heterogeneous Generative Dataset for Unmanned Autonomous Systems), a new dataset for Connected and Autonomous Vehicles (CAVs) and Unmanned Aerial Vehicles (UAVs), namely Unmanned Autonomous Systems (UASes). The dataset will be used to train artificial intelligence (AI) models to detect cybersecurity and safety-related risks, malfunctions, and crashes. The dataset was collected from three sources: 1) script-generated flying or driving missions, 2) software fuzzer-generated crashes instances, and 3) cybersecurity exploits generated by ethical hackers. To collect the data, we utilized the Digital Twin (DT) to replicate the behavior of UASes, which provides data that can be used to analyze, develop, and detect new anomaly detection algorithms.
PIC-Score: Probabilistic Interpretable Comparison Score for Optimal Matching Confidence in Single- and Multi-Biometric Face Recognition

In the context of biometrics, matching confidence refers to the confidence that a given matching decision is correct. Since many biometric systems operate in critical decision-making processes, such as in forensics investigations, accurately and reliably stating the matching confidence becomes of high importance. Previous works on biometric confidence estimation can well differentiate between high and low confidence, but lack interpretability. Therefore, they do not provide accurate probabilistic estimates of the correctness of a decision. In this work, we propose a probabilistic interpretable comparison (PIC) score that accurately reflects the probability that the score originates from samples of the same identity. We prove that the proposed approach provides optimal matching confidence. Contrary to other approaches, it can also optimally combine multiple samples in a joint PIC score which further increases the recognition and confidence estimation performance. In the experiments, the proposed PIC approach is compared against all biometric confidence estimation methods available on four publicly available databases and five state-of-the-art face recognition systems. The results demonstrate that PIC has a significantly more accurate probabilistic interpretation than similar approaches and is highly effective for multi-biometric recognition. The code is publicly-available <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Leveraging Multi-view Data for Improved Detection Performance: An Industrial Use Case

Printed circuit boards (PCBs) are essential components of electronic devices, and ensuring their quality is crucial in their production. However, the vast variety of components and PCBs manufactured by different companies makes it challenging to adapt to production lines with speed demands. To address this challenge, we present a multi-view object detection framework that offers a fast and precise solution. We introduce a novel multi-view dataset with semiautomatic ground-truth data, which results in significant labeling resource savings. Labeling PCB boards for object detection is a challenging task due to the high density of components and the small size of the objects, which makes it difficult to identify and label them accurately. By training an object detector model with multi-view data, we achieve improved performance over single-view images. To further enhance the accuracy, we develop a multi-view inference method that aggregates results from different viewpoints. Our experiments demonstrate a 15% improvement in mAP for detecting components that range in size from 0.5 to 27.0 mm.
Predictive World Models from Real-World Partial Observations

Cognitive scientists believe adaptable intelligent agents like humans perform reasoning through learned causal mental simulations of agents and environments. The problem of learning such simulations is called predictive world modeling. Recently, reinforcement learning (RL) agents leveraging world models have achieved SOTA performance in game environments. However, understanding how to apply the world modeling approach in complex real-world environments relevant to mobile robots remains an open question. In this paper, we present a framework for learning a probabilistic predictive world model for real-world road environments. We implement the model using a hierarchical VAE (HVAE) capable of predicting a diverse set of fully observed plausible worlds from accumulated sensor observations. While prior HVAE methods require complete states as ground truth for learning, we present a novel sequential training method to allow HVAEs to learn to predict complete states from partially observed states only. We experimentally demonstrate accurate spatial structure prediction of deterministic regions achieving 96.21 IoU, and close the gap to perfect prediction by 62 % for stochastic regions using the best prediction. By extending HVAEs to cases where complete ground truth states do not exist, we facilitate continual learning of spatial prediction as a step towards realizing explainable and comprehensive predictive world models for real-world mobile robotics applications. Code is available at https://github.com/robin-karlsson0/predictive-world-models.
A multi-UAV fast search path planning algorithm research

This study provides a multi-UAV fast search path planning (MFSPP) algorithm to address the problems of unbalanced task allocation, low efficiency, and time-consuming when several unmanned aerial vehicles (UAVs) conduct search tasks. First, several places with a high probability of target occurrence are inferred based on the location data reported before the target's disappearance. Second, the K-means clustering method is used to initialize the sub-task area. Then, the conflict units of the boundary are divided according to the initial state auction of UAVs, and the search tasks of each UAV are balanced. Finally, each UAV combines the probability of target occurrence with the heuristic greedy search algorithm to design its path in the sub-region. The simulation results reveal that the proposed algorithm is highly scalable and can find the target at a smaller area coverage compared to other search algorithms, which greatly reduces the search time and improves the search efficiency.
Memory-efficient and GPU-oriented visual anomaly detection with incremental dimension reduction

Recent studies show that the image features from pre-trained convolution neural network (CNN) can be used for anomaly detection, even without fine-tuning. A common type of methods divides the image space into patches, and estimates the distribution of CNN-based features per patch of all training data. While this types of methods can achieve high accuracies, the high dimensionality of CNN features causes overhead to both computing and storage. In this paper, we present an incremental algorithm to reduce the dimensionality of CNN features during the training. As our algorithm ultimately computes the Truncated PCA of the features, it only maintains the truncated singular values and vectors during the training. Besides, to efficiently update the truncated singular values/vectors of all patches, we further optimize the algorithm in order to fully utilize GPUs for parallel execution. We show that with our approach, we can achieve high accuracies on the texture classes of MVTec AD with small memory footprint and extreme high speed (around 200FPS) on a single GPU.
Intelligent Recognition for Fast Access to Machine to Machine

A key technology of Machine to Machine (M2M) communication system is link adaptation. The identification of modulation mode is of great significance to link adaptation. The traditional automatic modulation classification (AMC) method uses support vector machine (SVM) to classify signals, which has the following problems. The SVM classifier is a supervised method and the computation is huge. Aiming at the problems of traditional AMC methods, this paper creatively proposes a modulation classification method based on Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm. The method uses machine learning, and the computational load is far less than the deep learning used by the traditional AMC algorithm. In this paper, 5 feature selection methods are used to extract 12 parameters and classify signals. 5 feature selection methods are compared and their contributions to classification are analyzed. The experimental results show the excellent accuracy of modulation classification when the SNR is 5dB.
Deep Learning Based Efficient Pedestrian Age Group Estimation and Tracking Techniques for Autonomous Mover

For the application of smart autonomous movers, based on the information of body proportion, in this study a simple and effective processing scheme is developed for age group estimation, distance prediction, and object tracking for pedestrians. By the YOLO-based CNN model for head and body objects detections, the recognition accuracies of age group classification for children and adults are up to 90%. By the age group identification algorithm, the average error rate of the head-to-body ratio and height is estimated to be 3% based on the bounding box information given by YOLO. The predicted error rate of pedestrian's distance by the proposed design is about 3%. For real-time embedded applications, the pedestrian's tracking design is implemented on GPU-based platform, and the frame per second (FPS) rate can be up to 20.
Real Time-Based Face Recognition, Tracking, Counting, and Calculation of Spent Time of Person Using OpenCV and Centroid Tracker Algorithms

The study of human vision is one of the current controversial subjects in the computer vision consortium. Real-time tracking, face recognition, and counting of persons from video footage and CCTV or Webcam are almost new in flexible activities. Face recognition is often the first impression when used in video tracking, interfaces, and facial recognition. In this paper, we resolve to implement human face recognition, tracking, and counting, and spent time calculating in real-time using OpenCV and Python programming. The pilot method was implemented by using different types of OpenCV libraries like face recognition, Imutils, DateTime, and Centroid Tracker algorithm which we acquired the exact and absolute outcome for face recognition, tracking, counting, and spent time calculation depending on real-time. Face recognition algorithms is recognized faces in video files or webcams by showing people's identities. People are tracked and assigned an Individual ID using OpenCV's centroid tracking algorithm. Then the system calculates spent time for indicates the person was in the frame. After that, counting the number of people with the help of different personal IDs of other people which were provided by the tracker. The live frame captures the number of people (Live person count) and the total number of people (Total people count). Real-time and date are also displayed for security reasons. Through this research, we have been able to identify person through face detection, track and count all individuals with spent time calculating in real-time, which plays an important role in the security of important public and private institutions, especially banks, shopping malls, universities, etc.
Language Models are Causal Knowledge Extractors for Zero-shot Video Question Answering

Causal Video Question Answering (CVidQA) queries not only association or temporal relations but also causal relations in a video. Existing question synthesis methods pretrained question generation (QG) systems on reading comprehension datasets with text descriptions as inputs. However, QG models only learn to ask association questions (e.g., "what is someone doing…") and result in inferior performance due to the poor transfer of association knowledge to CVidQA, which focuses on causal questions like "why is someone doing …". Observing this, we proposed to exploit causal knowledge to generate question-answer pairs, and proposed a novel framework, Causal Knowledge Extraction from Language Models (CaKE-LM), leveraging causal commonsense knowledge from language models to tackle CVidQA. To extract knowledge from LMs, CaKE-LM generates causal questions containing two events with one triggering another (e.g., "score a goal" triggers "soccer player kicking ball") by prompting LM with the action (soccer player kicking ball) to retrieve the intention (to score a goal). CaKE-LM significantly outperforms conventional methods by 4% to 6% of zero-shot CVidQA accuracy on NExT-QA and Causal-VidQA datasets. We also conduct comprehensive analyses and provide key findings for future research.
CAVLI - Using image associations to produce local concept-based explanations

While explainability is becoming increasingly crucial in computer vision and machine learning, producing explanations that can link decisions made by deep neural networks to concepts that are easily understood by humans still remains a challenge. To address this challenge, we propose a framework that produces local concept-based explanations for the classification decisions made by a deep neural network. Our framework is based on the intuition that if there is a high overlap between the regions of the image that are associated with a human-defined concept and regions of the image that are useful for decision-making, then the decision is highly dependent on the concept. Our proposed CAVLI framework combines a global approach (TCAV) with a local approach (LIME). To test the effectiveness of the approach, we conducted experiments on both the ImageNet and CelebA datasets. These experiments validate the ability of our framework to quantify the dependence of individual decisions on predefined concepts. By providing local concept-based explanations, our framework has the potential to improve the transparency and interpretability of deep neural networks in a variety of applications.
QRADCOM: Quantum Assisted Framework for Joint Detection and Estimation in Radar Communications

For next-generation vehicle-to-everything (V2X) communications, there has been a growing emerging technology at the intersection of Radar and Communications, namely RadCom. RadCom provides many benefits such as efficient usage of bandwidth, joint detection and estimation, and improved tracking performance for future intelligent transportation. However, there is a limitation in the computation of parameters for large dimensions (antennas, users, or targets). To provide, a computational complexity advantage for the target detection and parameters, we introduce Quantum-assisted Radar Communication systems, namely QRADCOM. In this work, we have proposed algorithms for the multi-target detection hypothesis based on quantum phase estimation (QPE), and also the quantum inversion method for communication signal detection in presence of observation noise, interference and multi-tap wireless channel. We show numerical results and analysis as well for the QRADCOM framework on both classical and IBM-Quantum computers.
Systematic Architectural Design of Scale Transformed Attention Condenser DNNs via Multi-Scale Class Representational Response Similarity Analysis

Self-attention mechanisms are commonly included in a convolutional neural networks to achieve an improved efficiency performance balance. However, adding self-attention mechanisms adds additional hyperparameters to tune for the application at hand. In this work we propose a novel type of DNN analysis called Multi-Scale Class Representational Response Similarity Analysis (ClassRep-Sim) which can be used to identify specific design interventions that lead to more efficient self-attention convolutional neural network architectures. Using insights grained from ClassRepSim we propose the Spatial Transformed Attention Condenser (STAC) module, a novel attention-condenser based self-attention module. We show that adding STAC modules to ResNet style architectures can result in up to a 1.6% increase in top-1 accuracy compared to vanilla ResNet models and up to a 0.5% increase in top-1 accuracy compared to SENet models on the ImageNet64x64 dataset, at the cost of up to 1.7% increase in FLOPs and 2x the number of parameters. In addition, we demonstrate that results from ClassRepSim analysis can be used to select an effective parameterization of the STAC module resulting in competitive performance compared to an extensive parameter search.
Grover Adaptive Search for Joint Maximum-Likelihood Detection of Power-Domain Non-Orthogonal Multiple Access

In this paper, we address the joint maximum-likelihood detection of power-domain non-orthogonal multiple access and transform the detection problem into a binary optimization problem. Then, we solve it by a quantum algorithm, Grover adaptive search (GAS), and further accelerate the algorithm. Assuming fault-tolerant quantum computing, GAS provides a quadratic speedup for binary optimization problems in terms of query complexity. The proposed method designs the number of Grover operators using a time-invariant distribution of the number of solutions, which can be obtained in advance. Simulation results demonstrate that the proposed method is capable of reducing the query complexity in the classical domain while achieving the optimal performance.
t-RAIN: Robust generalization under weather-aliasing label shift attacks

In the classical supervised learning settings, classifiers are fit with the assumption of balanced label distributions and produce remarkable results on the same. In the real world, however, these assumptions often bend and in turn adversely impact model performance. Identifying bad learners in skewed target distributions is even more challenging. Thus achieving model robustness under these "label shift" settings is an important task in autonomous perception. In this paper, we analyze the impact of label shift on the task of multi-weather classification for autonomous vehicles. We use this information as a prior to better assess pedestrian detection in adverse weather. We model the classification performance as an indicator of robustness under 4 label shift scenarios and study the behavior of multiple classes of models. We propose t-RAIN a similarity mapping technique for synthetic data augmentation using large scale generative models and evaluate the performance on DAWN dataset. This mapping boosts model test accuracy by 2.1, 4.4, 1.9, 2.7 % in no-shift, fog, snow, dust shifts respectively. We present state-of-the-art pedestrian detection results on real and synthetic weather domains with best performing 82.69 AP (snow) and 62.31 AP (fog) respectively.
Exploring the Importance of Pretrained Feature Extractors for Unsupervised Anomaly Detection and Localization

Modeling the distribution of descriptors obtained by pretrained feature extractors is a popular approach for unsupervised visual anomaly detection. While recent work primarily focuses on the development of new methods that build on such extractors, the importance of the selected feature space itself has not been sufficiently studied. We therefore conduct a systematic analysis of current anomaly detection methods with respect to different feature extractors, their intermediate layers, and pretraining protocols. We show that the investigated methods are highly sensitive to the particular choice of feature space. We further demonstrate that using an optimal feature selection strategy can significantly improve the anomaly detection performance, up to a point where selecting a single feature layer outperforms computationally expensive ensembling approaches.