Reviewerly/exHarmony huggingface dataset:
Real-Time Multi-Modal Active Vision for Object Detection on UAVs Equipped With Limited Field of View LiDAR and Camera

This letter aims to solve the challenging problems in multi-modal active vision for object detection on unmanned aerial vehicles (UAVs) with a monocular camera and a limited Field of View (FoV) LiDAR. The point cloud acquired from the low-cost LiDAR is firstly converted into a 3-channel tensor via motion compensation, accumulation, projection, and up-sampling processes. The generated 3-channel point cloud tensor and RGB image are fused into a 6-channel tensor using an early fusion strategy for object detection based on a Gaussian YOLO network structure. To solve the low computational resource problem and improve the real-time performance, the velocity information of the UAV is further fused with the detection results based on an extended Kalman Filter (EKF). A perception-aware model predictive control (MPC) is designed to achieve active vision on our UAV. According to our performance evaluation, our pre-processing step improves other literature methods running time by a factor of 10 while maintaining acceptable detection performance. Furthermore, our fusion architecture reaches 94.6 mAP on the test set, outperforming the individual sensor networks by roughly 5%. We also described an implementation of the overall algorithm on a UAV platform and validated it in real-world experiments.
Using Large Language Models for Interpreting Autonomous Robots Behaviors

The deployment of autonomous robots in various domains has raised significant concerns about their trustworthiness and accountability. This study explores the potential of Large Language Models (LLMs) in analyzing ROS 2 logs generated by autonomous robots and proposes a framework for log analysis that categorizes log files into different aspects. The study evaluates the performance of three different language models in answering questions related to StartUp, Warning, and PDDL logs. The results suggest that GPT 4, a transformer-based model, outperforms other models, however, their verbosity is not enough to answer why or how questions for all kinds of actors involved in the interaction.
Content-Aware Network Traffic Prediction Framework for Quality of Service-Aware Dynamic Network Resource Management

Next-generation mobile networks (such as Fifth-Generation (5G), and Sixth-Generation (6G)) are envisioned to undergo an unprecedented transformation from connected things to connected intelligence with more stringent characteristics, i.e., low end-to-end latency, high bandwidth, reliable connectivity, etc. Such networks will significantly increase network traffic in the distribution networks, causing the need for real-time automated decision-making, such as automated network resource allocation. The network resources must be allocated dynamically based on the Quality of Service (QoS) requirements. However, the primary concern is that the distribution networks may get congested soon after allocating network resources. Thus, a QoS-aware prediction framework can be used proactively to predict the future trend of network traffic in each QoS class. In this paper, we propose a framework for predicting the heterogeneous multivariate QoS-aware network traffic to make the best use of network resources dynamically. Specifically, we use a Recurrent Neural Network (RNN) to integrate a Bidirectional Long Short-Term Memory (BLSTM) neural network. The results show that the fusion of RNN-BLSTM can predict the QoS-aware network traffic for over 13 hours with an average accuracy of 97.68%. Moreover, the proposed model is trained and tested over limited data, collected and identified through Deep Packet Inspection (DPI) over an operational network. In addition, we compared the RNN-BLSTM with other prediction algorithms (i.e., LSTM, ARIMA, SVM) in terms of precision, execution time, and energy consumption. Lastly, the proposed framework is used to assign the network resources to each QoS class based on the QoS requirements of that class and its pre-defined priority.
Adapting YOLOv8 as a Vision-Based Animal Detection System to Facilitate Herding

In this work, the YOLOv8 model is adapted to a specific problem in order to increase its performance. Thus, a vision-based system is developed to provide perceptual information to a robot to detect animals in the environment and to be able to perform herding tasks. For this purpose, a dataset is created by selecting animal images from the public AP10K dataset, as well as sheep images acquired by a camera attached to a 4-legged robot. Three different configurations of YOLOv8 are considered: nano, medium and extra-large, trained on the COCO dataset. Its fine-tuning with the animal image dataset shows an improvement in performance achieved not only from the point of view of the robot, but also from the point of view of a drone or a person. The best results are obtained with the YOLOv8 medium configuration when it is trained with the dataset that includes images of the robot’s view.
Deploying Electronic Voting System Use-case on Ethereum Public Blockchain

Electronic voting system has great potential to decrease organizational cost of traditional voting system and also it has ability to maximize the voters turnout. When real time voting system is used for election, it must be convenient, secure, accurate, safe and legitimate. This paper evaluates blockchain framework and implement real time voting system by using blockchain technology which increase the security and decrease the total organizational cost for election. Proposed model is divided into three parts i.e., admin, voter and registration phase. Here admin responsible for managing the life cycle of each and every election. It has not allowed to cast a ballot. Registration phase checks voters’ recognizable proof and register represents them in the framework. In this, accounts are held by authorities in surveying station found close by local locations. We used RPC server for network and solidity as a programming language for smart contract. Here protocol execution uses consensus mechanism which is responsible to secure the Ethereum blockchain. We tested proposed model by using Rinke by test network for ensuring the feasibility.
Multi-NeuS: 3D Head Portraits From Single Image With Neural Implicit Functions

We present an approach for the reconstruction of textured 3D meshes of human heads from one or few views.Since such few-shot reconstruction is underconstrained, it requires prior knowledge which is hard to impose on traditional 3D reconstruction algorithms.In this work, we rely on the recently introduced 3D representation -neural implicit functions -which, being based on neural networks, allows to naturally learn priors about human heads from data, and is directly convertible to textured mesh.Namely, we extend NeuS, a state-of-the-art neural implicit function formulation, to represent multiple objects of a class (human heads in our case) simultaneously.The underlying neural net architecture is designed to learn the commonalities among these objects and to generalize to unseen ones.Our model is trained on just a hundred smartphone videos and does not require any scanned 3D data.Afterwards, the model can fit novel heads in the few-shot or one-shot modes with good results.
Semi-Supervised ML-Based Joint Channel Estimation and Decoding for m-MIMO With Gaussian Inference Learning

This letter proposes the use of quasi-orthogonal space-time block codes (QOSTBC) to enhance link quality and reliability in massive multiple-input multiple-output (m-MIMO) systems subject to independent fading in dynamic channels. It has been shown, however, that the computational complexity of classical decoding algorithms, such as maximum likelihood, can hinder the adoption of QOSTBC codes in systems with many antennas and high-order modulation schemes. Complex-valued neural networks (CVNNs) offer a promising alternative for joint decoding and channel estimation with competitive computational complexity. This letter presents an extension of our previously proposed CVNN with supervised training, which incorporates two semi-supervised learning techniques: hard inference learning (HIL) and Gaussian inference learning (GIL). By leveraging non-pilot-aided data, HIL and GIL enable the CVNNs to self-learn from useful information, increasing their tracking ability and robustness in dynamic channels.
Investigating UAV Propellers Performances Near Moving Obstacles: CFD Study, Thrust Control, and Battery Energy Management

Quadcopters are widely used in military and civilian applications, but when flying near moving objects, they may encounter flight instability. This study investigates the flow aerodynamic interference of two drones' propellers when they hover near moving obstacles using CFD simulations. An unstructured mesh is used with frame motion to model the propeller's rotation. Different altitudes, rotational speeds, and obstacle velocities are tested to understand their aerodynamic effects on the propeller's performance. The results indicate that hovering 0.1 m above a fixed obstacle at 9550 rpm increased lift by 9.28%. This decreased by half when propellers were positioned adjacent to each other. Besides, a moving wall below the propellers at various velocities significantly affects the thrust and power variation of the propellers at low hovering speeds. For example, when hovering at 3000 rpm over a moving wall at 10 m/s, the thrust on one side of the UAV decreases by 8.51%, leading to flight instability. Different thrust control strategies and flight scenarios are performed to ensure flight stability and energy management. Finally, thrust control strategies prove crucial in stabilizing thrust and altitude, but power consumption increases by 12.8%, requiring 30% more energy required under specific flight scenarios near moving obstacles.
Design and Implementation of Side Channel Attack Based on Deep Learning LSTM

Encryption algorithms and encryption devices both play a key role in ensuring the safety of data that has been encrypted. Various types of attacks, such as energy analysis, can be used to assess the reliability of the encryption devices. Since it was originally introduced, side channel attacks' deep learning-based methodology has drawn plenty of attention. This is one of several different attack strategies. In this paper, a side channel attack method based on the LSTM deep learning network is suggested. The method use Correlation Power Analysis (CPA) to find the relevant information in the side channel power consumption data. The choice of a suitable interest interval to utilize as the feature vector in the creation of the neural network model is then guided by the position of the interest points. The trials' findings show that the LSTM model outperforms both MLP and CNN in terms of how well it executes side channel attacks.
Probability Density Function for Clustering Validation

The choice of the number of clusters is a leading problem in Machine Learning. Validation methods provide solutions, with the drawback that inference is not possible. In this manuscript, we derive a distribution for the number of clusters for clustering validation. The starting point of our approach is the data transformation to the probabilistic space. Then, the dependence of the non-negative factorization to the dimensionality of the space span provides a sequence of the traces when the dimensionality varies. Its limit is a gamma. This result allows a non-excluding discussion when interpreting probabilities as credibility levels, and we open the door to inference for clustering validation.
Breast Cancer Detection Using Hybrid Segmentation Using FOA and FCM Clustering

Medical image processing has recently been widely applied in a variety of fields. Finding the anomaly problems in that image is highly beneficial for the early diagnosis of these ailments. There are several techniques available for segmenting MRI images to find breast cancer. Breast cancer is the second greatest cause of death in women. Early detection of breast cancer reduces the number of women who die from cancer. If caught in time, breast cancer is among the forms of cancer that can be cured. In this study, breast cancer is identified in medical photos using a unique hybrid segmentation technique. Fruitfy optimization technique (FOA) and FCM clustering are both used in hybrid segmentation. To get a more accurate value of the clustering centers in FCM Clustering, a Fruitfy optimization algorithm (FOA) approach was applied. The MRI images' features are extracted using the Extended Gabor wavelet transform (IGWT). When compared to other approaches, the result demonstrates that the hybrids segment performs with great performance and good accuracy of 96.50%.
A Hybrid Artistic Model Using Deepy-Dream Model and Multiple Convolutional Neural Networks Architectures

The significant increase in drug abuse cases prompts developers to investigate techniques that mimic the hallucinations imagined by addicts and abusers, in addition to the increasing demand for the use of decorative images resulting from the use of computer technologies. This research uses Deep Dream and Neural Style Transfer technologies to solve this problem. Despite the significance researches on Deep Dream technology, there are several limitations in existing studies, including image quality and evaluation metrics. We have successfully addressed these issues by improving image quality and diversifying the types of generated images. This enhancement allows for more effective use of Deep Dream in simulating hallucinated images. Moreover, the high-quality generated images can be saved for dataset enlargement, like the augmentation process. Our proposed deepy-dream model combines features from five convolutional neural network architectures: VGG16, VGG19, Inception v3, Inception-ResNet-v2, and Xception. Additionally, we generate Deep Dream images by implementing each architecture as a separate Deep Dream model. We have employed autoencoder Deep Dream model as another method. To evaluate the performance of our models, we utilize normalized cross-correlation and structural similarity indexes as metrics. The values obtained for those two quality measures for our proposed deepy-dream model are 0.1863 and 0.0856, respectively, indicating effective performance. When considering the content image, the metrics yield values of 0.8119 and 0.3097, respectively. Whiefor the style image, the corresponding quality measure values are 0.0007 and 0.0073, respectively.
Multi-Scale Aggregation Transformers for Multispectral Object Detection

Multispectral object detection for autonomous driving is multi-object localization and classification task on visible and thermal modalities. In this scenario, modality differences lead to the lack of object information in a single modality and the misalignment of cross-modality information. To alleviate these problems, most existing methods extract information based on a single scale ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g</i> ., these methods mainly focus on detecting significant cars or pedestrians), which leads to insufficient performance in capturing multi-scale discriminative information ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g</i> ., small bicycles and blurred pedestrians) and safety hazards in the driving process. In this paper, we propose a Multi-Scale Aggregation Network (MSANet) consisting of two parts Multi-Scale Aggregation Transformer (MSAT) and the Cross-modal Merging Fusion Mechanism (CMFM), which combined with the advantages of Transformer and CNN to extract rich image information from two modalities by mining both local and global context dependencies. Firstly, to reduce the lack of information in a single modality, we design a novel MSAT module to extract rich details and texture from multi-scale. Secondly, to alleviate feature misalignment caused by modality differences, the CMFM is utilized to aggregate complementary information on multiple levels. Comprehensive experiments on two benchmarks demonstrate that our approach shows better results than several state-of-the-art methods. The code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/ysh-strive/MSANet</uri> .
User Authentication by Eye Movement Features Employing SVM and XGBoost Classifiers

Devices capable of tracking the user’s gaze have become significantly more affordable over the past few years, thus broadening their application, including in-home and office computers and various customer service equipment. Although such devices have comparatively low operating frequencies and limited resolution, they are sufficient to supplement or replace classic input interfaces, such as the keyboard and mouse. The biometric application we researched verifies a user’s identity based on parameters acquired with a low-cost eye tracker. The use of the eye-tracking device in bank booths has many advantages, including the fact that eye trackers are contactless devices, which, especially in the light of the Covid pandemic, has increasing importance, in addition to providing a solution for confirming the liveness of the user. This paper describes an experiment in which 20 features extracted from eye movement data related mainly to saccades and fixations are used as a complementary biometric modality to authenticate clients at banking kiosks. Data were collected from 39 subjects while operating a banking system using engineered biometric kiosk prototypes. Authentication performance employing eye-movement tracking and parameterizing was compared for two classifiers: Support Vector Machine (SVM) and extreme Gradient Boosting (XGBoost). The results showed that the XGBoost-based classifier outperformed the SVM-based one regarding equal error rates (6.76% to 8% vs. 16.21 to 18.78%). Similar differences were obtained for true acceptance rates at different false acceptance rates (0.1 and 0.01), where the SVM-based classifier achieved a maximum of 81.08% and the XGBoost-based achieved 98.65%. Finally, prospects for the broader application of eye movement tracking as a biometric modality are discussed.
Privacy-Encoded Federated Learning Against Gradient-Based Data Reconstruction Attacks

Federated learning (FL) enables multiple local clients to collaboratively train a global model, which can reduce privacy leakage by sharing model parameters instead of private datasets. However, recent works have revealed that gradient-based data reconstruction attacks, e.g., deep leakage from gradients (DLG), improved DLG (iDLG), and inverting gradients (IG), may still reveal private information by exploiting model parameters from a local client. Current privacy-preserving FL strategies, e.g., differential privacy or gradient compression, can handle such attacks to some extent, but seriously sacrifice their model accuracy. In this work, we propose a novel privacy-preserving FL method, named privacy-encoded FL (PEFL), to combat such data reconstruction attacks without degrading the model performance. The key concept of PEFL is that each large weight matrix of the neural network model is decomposed into multiple cascading sub-matrices, which thus establishes a novel privacy-encoded mechanism by introducing an entangling nonlinear mapping between model gradients and raw data. As such, multiple sub-matrices are directly trained in parallel at the local clients, but only partial sub-matrices are reported to a global server, which suffices to reconstruct the global model whilst increasing the complexity of the coupling between the model parameters and raw data. We provide a detailed analysis of the accuracy, security, and complexity of our method. As shown, it breaks the limit of classical defensive methods, by significantly reducing the risk of data reconstruction attacks yet not degrading the model performance. Compared to classical defenses, the proposed PEFL decreases the peak-signal-to-noise ratio (PSNR) between the reconstructed data and the raw data by ~20dB, without sacrificing the test accuracy. As a new paradigm for privacy-preserving FL, our proposed method has great potential in privacy-demanding learning applications.
EMED-UNet: An Efficient Multi-Encoder-Decoder Based UNet for Medical Image Segmentation

Many current and state-of-the-art deep learning models for accurate image segmentation are based on the U-Net architecture, a convolutional neural network designed for biomedical applications. Despite its widespread adoption in the medical imaging community, U-Net has two major limitations. First, due to its deep structure and the large number of filters used, the number of parameters and Floating-Point Operations Per Second (FLOPS) are high. This results in high computational complexity and demands a large memory size, making real-time implementation and deployment of U-Net models challenging. Second, the base U-Net model only uses a single type of kernel (i.e., 3×3) throughout the network for all convolution operations. Feature extraction using a single spatial extent throughout the network is suitable if the size, location, and shape of the salient regions remain static in all the images of a dataset, which is not necessarily the case in medical imaging. To address these two limitations, we propose an Efficient Multi-Encoder-Decoder based UNet (EMED-UNet), a novel architecture for efficient medical image segmentation. We evaluated our network on four medical imaging datasets: Montgomery County, Shenzhen CXR, COVID-19 CT LS, and BraTS (Brain Tumor Segmentation) dataset. EMED-UNet outperforms the U-Net and its variants in terms of accuracy, with around 77% reduction in parameters, a 60% reduction in FLOPS, and a 79.2% reduction in memory usage (all as compared to U-Net). The results demonstrate that EMED-UNet is a lightweight and accurate model for image segmentation that substantially improves upon the U-Net base model and is more feasible to deploy, given its decreased computational cost.
Multimodal Deep Learning Crime Prediction Using Tweets

Crime prevention relies on crime prediction as a crucial method to determine the most effective patrol strategy for law enforcement agencies. Various approaches and solutions have been utilized to predict criminal activity. Nonetheless the environment and nature of information for crime prediction is constantly changing. Although, potentially a useful source for gathering sentiments, social media content has been ignored by the prediction models. The utilization of social media for sharing information and ideas has experienced a significant surge. Twitter, in particular, is regarded as a valuable platform for gathering public sentiments, emotions, perspectives, and feedback. In this regard, techniques for analyzing the sentiment of tweets on Twitter have been developed to ascertain whether the textual content conveys a positive or negative viewpoint on crime incident. Therefore, our interest lies in investigating the potential and advantages of fusing the information of sentiment and crime modalities. In this paper, ConvBiLSTM is applied to train the model, features of both tweet and crime modalities were extracted independently at vector level and fused into a single representation that captures the information from all modalities. This study involved collecting and conducting experiments using two datasets. The first dataset consisted of crime incident data obtained from the Chicago police department, specifically covering the period between September 1 and September 30, 2019. The second dataset comprised tweets containing crime-related terminology specific to Chicago. The crime prediction using multimodal data fusion on ConvBiLSTM outperform against other models with 97.75% of accuracy.