Reviewerly/exHarmony huggingface dataset:
Patient Monitoring System

With the need to monitor the status of individuals, many researchers and scientists have used pose estimation techniques to record and analyze human poses. The research proposes a method to monitor patients based on human detection and human pose estimation and has proposed a methodology for it. The main purpose of this paper is to detect if a person in a given frame is a patient or visitor and then detect various poses of the patient only. For human detection, YOLOv4-tiny is trained on 2 classes: Patients and Visitors using the Google Colab platform, and the dataset was collected from open-source resources like Google images. Pose is estimated only for patients after training using the Mediapipe framework, which detects 33 keypoints. The model will classify a pose into 4 different classes: Sleeping, Sitting, Walking, and Standing. The keypoints from Mediapipe are then used to classify the pose of the patient, using PCA and Xgboost to monitor and export the data in a CSV file. After the training, the pretrained model is used to apply it in a real-time video application developed using the Tkinter GUI library in Python. The accuracy for the YOLOv4-tiny model is 91.09% and for the Xgboost model it is 98.82%.
Energy-Efficient FPGA Implementation of Power-of-2 Weights-Based Convolutional Neural Networks With Low Bit-Precision Input Images

Convolutional neural networks (CNNs) have performed exceptionally well on a variety of image classification tasks but need significant amount of memory and computational resources. In this brief, we propose a power-of-2 weights based CNN inference engine, that takes images whose pixels are either binarized or quantized to low-resolution and also has its weights at each layer reduced to powers-of-2, thus replacing the multiplication operations with a simple shift operation, thereby reducing the computations significantly. The proposed CNN architecture, designed to be used in an always-on inference engine, has been demonstrated for SVHN and CIFAR-10 datasets and achieves an accuracy of 83.23% and 87.02%, respectively, while using 3-bit quantized images. The proposed architecture, therefore, achieves a performance comparable to inference engines that used 8-bit quantized images while achieving a frame-rate of 380 FPS by consuming 0.789 W of dynamic power, a very competitive energy efficiency of 892.45 GOPs/J. The system requires ≈1 Mb of memory for storing weights, making it suitable for memory constrained edge devices and ASIC implementations.
Differential Cryptanalysis of Salsa20 Based on Comprehensive Analysis of PNBs

This paper focuses on the differential cryptanalysis of the Salsa20 stream cipher. The existing differential cryptanalysis approaches first study the differential bias of the Salsa20 stream cipher and then search for probabilistic neutral bits (PNBs). However, the differential bias and the set of PNBs obtained in this method are not always the optimal solution. To figure out a better solution, we apply the differential cryptanalysis based on the comprehensive analysis of PNBs on the reduced-round Salsa20 introduced in [19]. At first, we comprehensively analyze the neutrality measure of all keybits concerning all output differential $$\mathcal{O}\mathcal{D}$$ bits. Afterward, we select the $$\mathcal{O}\mathcal{D}$$ bit position with the best neutrality measure and look for the corresponding input differential $$\mathcal{I}\mathcal{D}$$ with the best differential bias. Taking everything into account, the proposed approach could be used to attack Salsa20/8 with a time complexity of $$2^{144.75}$$ and a data complexity of $$2^{55.74}$$ .
NIFL: A Statistical Measures-Based Method for Client Selection in Federated Learning

Federated learning (FL) has been proposed as a machine learning approach to collaboratively learn a shared prediction model.Although, during FL training, only a subset of workers participate in each round, existing approaches introduce model bias when considering the average of local model parameters of heterogeneous workers, which degrades the accuracy of the learned global model.In this paper, we introduce NIFL, a new strategy for worker selection that handles the statistical challenges of FL when local data is Non-Independent and Identically Distributed (N-IID).In NIFL, the server starts sending the signal to the workers that react by sending the number of their samples.The server then selects a percentage of workers with the highest number of samples and requests data statistics such as mean and standard deviation.After that, the server calculates our proposed N-IID index, based on the statistical information collected from the workers without having access to their data, and uses this index as a criterion for worker selection.Finally, the server broadcasts the global model to the selected workers.NIFL takes into account the disparity in the distribution of workers' data in order to improve the performance of the model in heterogeneous data environment.We have performed several experiments with N-IID data.The obtained results show that both the convergence of our method and the test accuracy increased considerably comparing to the other techniques while keeping a reasonable computation and communication costs.
CARM: CUDA-Accelerated RNS Multiplication in Word-Wise Homomorphic Encryption Schemes for Internet of Things

Homomorphic encryption (HE), which allows computation over encrypted data, has often been used to preserve privacy. However, the computationally heavy nature and complexity of network topologies make the deployment of HE schemes in the Internet of Things (IoT) scenario difficult. In this work, we propose CARM, the first optimized GPU implementation that covers BGV, BFV and CKKS, targeting for accelerating homomorphic multiplication using GPU in heterogeneous IoT systems. Our solution is suitable for accelerating RNS homomorphic multiplication on both high-performance and embedded GPUs, as it is a parametric and generic design and offers various trade-offs between resource and efficiency. We offer constant-time low-level arithmetic with minimum instructions and memory usage, as well as performance- and memory-prior configurations. Through this, we can provide more real-time evaluation results and relieve the computational pressure on cloud devices. We deploy our implementations on two GPUs. Compared to the CPU implementation, we achieve up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$378.4\times$</tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$234.5\times$</tex-math></inline-formula> , and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$287.2\times$</tex-math></inline-formula> speedup for homomorphic multiplication of BGV, BFV, and CKKS on Tesla V100S, and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$8.8\times$</tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$9.2\times$</tex-math></inline-formula> , and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$10.3\times$</tex-math></inline-formula> on Jetson AGX Xavier, respectively.
A Spectral View of Randomized Smoothing Under Common Corruptions: Benchmarking and Improving Certified Robustness

Certified robustness guarantee gauges a model's resistance to test-time attacks and can assess the model's readiness for deployment in the real world. In this work, we explore a new problem setting to critically examine how the adversarial robustness guarantees change when state-of-the-art randomized smoothing-based certifications encounter common corruptions of the test data. Our analysis demonstrates a previously unknown vulnerability of these certifiably robust models to low-frequency corruptions such as weather changes, rendering these models unfit for deployment in the wild. To alleviate this issue, we propose a novel data augmentation scheme, FourierMix, that produces augmentations to improve the spectral coverage of the training data. Furthermore, we propose a new regularizer that encourages consistent predictions on noise perturbations of the augmented data to improve the quality of the smoothed models. We show that FourierMix helps eliminate the spectral bias of certifiably robust models, enabling them to achieve significantly better certified robustness on a range of corruption benchmarks. Our evaluation also uncovers the inability of current corruption benchmarks to highlight the spectral biases of the models. To this end, we propose a comprehensive benchmarking suite that contains corruptions from different regions in the spectral domain. Evaluation of models trained with popular augmentation methods on the proposed suite unveils their spectral biases. It also establishes the superiority of FourierMix trained models in achieving stronger certified robustness guarantees under corruptions over the entire frequency spectrum.
Helmet Detection System for Motorcycle Riders with Explainable Artificial Intelligence Using Convolutional Neural Network and Grad-CAM

Motorcycle accidents are one of the most common causes of injury and death in road users. This research has applied convolutional neural network (CNN) and explainable AI to detect motorcyclist without helmet and explain why CNN made that decision. The concept is based on deep learning and CNN principles applied to automatically detect motorcycle riders without helmet from images using three baseline classifiers: support vector machine, random forest and logistic regression to compare with deep convolutional neural network and measure accuracy with accuracy and F1-Score. The results revealed that CNN's F1-Score was 0.8326, which was the highest among all predictive models.
An Application of the IoVs Information Sharing Scheme Based on Blockchain Technology

Data generated in the internet of vehicles (IoVs) system is always considered to improve the driving performance of vehicle manufacturers. Aiming at the issues of security privacy and low willingness of users involved in the data sharing of IoVs, the paper proposes an IoVs information sharing scheme with double-chains parallelizing, combining asymmetric encryption technology and smart contract in blockchain. There are two chains in this novel scheme, one is data to be uploaded chain includes vehicles' data and the other is uploading data chain includes the requests of users. The scheme is secure and voluntary circulation by the automatic execution of smart contracts and this scheme can motivate vehicle users to become information production nodes and share high-quality and highly reliable IoVs information for other users in the region.
MNSRNet: Multimodal Transformer Network for 3D Surface Super-Resolution

With the rapid development of display technology, it has become an urgent need to obtain realistic 3D surfaces with as high-quality as possible. Due to the unstructured and irregular nature of 3D object data, it is usually difficult to obtain high-quality surface details and geometry textures at a low cost. In this article, we propose an effective multimodal-driven deep neural network to perform 3D surface super-resolution in 2D normal domain, which is simple, accurate, and robust to the above difficulty. To leverage the multimodal information from different perspectives, we jointly consider the texture, depth, and normal modalities to simultaneously restore fine-grained surface details as well as preserve geometry structures. To better utilize the cross-modality information, we explore a two-bridge normal method with a transformer structure for feature alignment, and investigate an affine transform module for fusing multimodal features. Extensive experimental results on public and our newly constructed photometric stereo dataset demonstrate that the proposed method delivers promising surface geometry details compared with nine competitive schemes.
Sparse Attack on Skeleton-Based Human Action Recognition for Internet of Video Things Systems

Action recognition is widely used in industry like human-computer interaction and intelligent monitoring. But in the circumstance of deep model being threatened by adversarial attack, the robustness of skeleton base action recognition, which is widely concerned for its better performance, has not been widely discussed. Recently, the robustness of existing HAR models have been questioned for their vulnerability to adversarial attacks. However, Previous attacks introduce unnecessary perturbations which heavily increase the deviation between adversarial sample and origin sample. Base on solving this problem, this article proposes a sparse attack method with dynamic attention which generates adversarial samples with lower deviation and without a drop of attack success rate. Firstly, we develop spatial temporal consistency loss to preserve not only spatial integrity between reference sample and adversarial sample, but also temporal coherence between consecutive frames across adversarial sample. Then, we develop an interaction-based perturbation contribution analyze method to discard unnecessary perturbations. Finally, we design a dynamic attention approach to tilt the perturbations towards some joints that have higher dynamics which can decrease overall deviations. By attacks, our method contributes to discover potential causes of model fragility and provides material in adversarial training which will increase the robustness of HAR models.
RECALL: Rehearsal-free Continual Learning for Object Classification

Convolutional neural networks show remarkable results in classification but struggle with learning new things on the fly. We present a novel rehearsal-free approach, where a deep neural network is continually learning new unseen object categories without saving any data of prior sequences. Our approach is called RECALL, as the network recalls categories by calculating logits for old categories before training new ones. These are then used during training to avoid changing the old categories. For each new sequence, a new head is added to accommodate the new categories. To mitigate forgetting, we present a regularization strategy where we replace the classification with a regression. Moreover, for the known categories, we propose a Mahalanobis loss that includes the variances to account for the changing densities between known and unknown categories. Finally, we present a novel dataset for continual learning (HOWS-CL-25), especially suited for object recognition on a mobile robot, including 150,795 synthetic images of 25 household object categories. Our approach RECALL outperforms the current state of the art on CORe50 and iCIFAR-100 and reaches the best performance on HOWS-CL-25.
PPIRB:Achieving an privacy-preserving image retrieval scheme Based on Blockchain

With the proliferation of cloud services, the work of large-scale image retrieval is carried out on remote cloud become a trend in order to get rid of the storage burden and computation. However, traditional retrieval work based on centralized cloud services is considered to take risk in giving away users' privacy in many researches. We need a secure way for retrieving and storing images by decentralized approach. In this paper, we propose a decentralized scheme named PPIRB(An Privacy-Preserving Image Retrieval Scheme Base on Blockchain) for retrieving images based on blockchain. We used the data structure called amplifying locality sensitive hashing (ALSH) to improve search accuracy. Searchable symmetric en-cryption(SSE) and ALSH used to realize the encryption retrieval of image index on the blockchain. Our scheme also uses IPFS to replace the cloud server to realize distributed storage, and uses secret sharing scheme to realize the access control. We implemented and evaluated the scheme in Ethereum, and the results show that our scheme is efficient and accurate in finding similar images while preserving privacy. The overall overhead is reasonable.
Class-Balanced Contrastive Learning for Fine-Grained Airplane Detection

Airplane detection and fine-grained recognition in remote sensing images are challenging due to class imbalance and high inter-class indistinction. To alleviate these issues, we propose class-balanced contrastive learning (CBCL) approach for airplane detection to exploit the correlation between samples in different images, which is rarely explored in previous research. Specifically, we first dynamically build class-balanced memory queues during training, which mitigates class imbalance by memorizing training samples. Upon class-balanced memory queues, hard triplet contrastive learning is introduced to increase the inter-class discriminability, which enforces the maximum distance of the positive sample pair to be smaller than the minimum distance of the negative sample pair. We integrate the proposed CBCL strategy into oriented object detection frameworks for fine-grained airplane detection. The experimental results on the FAIR1M dataset reveal that several state-of-the-art algorithms with CBCL achieve significantly improvements.
Comparative Analysis of Despeckle Filtering Methods for Ultrasound Images

Medical image recognition is becoming more in demand every day. For more accurate and faster diagnosis, it is necessary not only to enhance image recognition systems but also to improve the quality of images. Images taken with ultrasound suffer from speckle noise. Due to the fact that the tissue has a texture, irregularities reflect ultrasonic radiation and create a speckle pattern. Speckles give an understanding of the surface topography, but image distortion takes place. Denoising is a commonly effective way to improve image quality. This paper presents a comparative analysis of popular speckle denoising methods. An anisotropic diffusion filter was created and adapted for this task. This work showed that even such a highly specialized filter has its limitations. A filter was found that can compete with it. The results of this work can be used by scientists and developers as a reference on the main types of filters for removing speckle noise.
Comparative evaluation of path planning algorithms in a simulated disaster environment

Anticipating and managing the human flow is essential for saving human lives during a disaster. Unmanned Aerial Vehicle (UAV) helps in human detection and navigating to the location of the injured person within a short time. This paper compares Q-learning and state-action-reward-state-action (SARSA) path planning algorithms for guiding the UAV to the desired location. A virtual disaster environment is created using Gazebo and the path planning algorithms are executed in the UAV using the Robot Operating System (ROS) platform. The UAV is equipped with a SONAR sensor for obstacle avoidance. For the experiment, the UAV starts and ends at fixed locations. The experiment results show that the Q-learning algorithm performs better than the SARSA algorithm.
More Than Meets the Eye - An Anti-Phishing Learning Game with a Focus on Phishing Emails

Phishing is a constant threat to the online security of end-users. Since technical measures currently fall short of preventing phishing attacks completely, educating end-users is an important factor in reducing the risk of successful attacks. Here, game-based learning has emerged as a scalable, motivational educational approach. While learning games that focus on phishing emails have been created in the past, they mainly include simple game mechanics, which do not map to the complex decisions that are involved in recognizing malicious emails. To this end, we present a novel anti-phishing learning game, consisting of two different game modes: Either, players have to create phishing emails from given templates themselves, or they have to analyze emails for malicious cues and mark relevant parts. The game is designed for a broad target group of adult users with little to no prior knowledge about phishing. To facilitate immersion, the game content is generated automatically and allows for personalization. This paper presents the design, implementation, and a preliminary usability evaluation of the game.
Vision-Based Maritime Object Detection Covering Far and Tiny Obstacles

Though general object detection with deep learning has achieved great success in the maritime domain, recognizing far-away and tiny objects is still challenging. However, an autonomous vessel must be aware of objects even several nautical miles away for early planning. In this paper, we propose a far-away and tiny object detection framework with deep learning by utilizing a high-resolution image. The framework is two-fold: Small Object Detector (SOD) and Large Object Detector (LOD). SOD investigates a region around the horizon with a fine-resolution image aiming to recognize far-away and small objects. LOD targets large objects that are identifiable in an even coarse image. We evaluate our approach on datasets including objects of various sizes inshore or offshore. The results verify that our proposed scheme is effective against even far-away and tiny objects while keeping computational costs low.
A Full- Reference Quality Assessment Metric for Cartoon Images

Cartoon images are illustrations that are typically drawn, sometimes animated, in an unrealistic or semi-realistic style, which are widely applied in multimedia services. However, in some post-production processes as well as transmission systems, cartoon images are inevitably distorted by wrong color arrangement and compression. Therefore, it is urgent to carry out image quality assessment (IQA) metrics to automatically predict the perceptual quality levels of distorted cartoon images. Nevertheless, the existing mainstream IQA metrics are specially developed for natural scene images (NSIs). Due to the statistical difference in structure and color aspects between cartoon images and NSIs, the scores predicted by such metrics are often inconsistent with the human vision system (HVS) for cartoon images. To further improve the performance of cartoon image quality assessment (C-IQA) methods and provide guidance for practical applications, we propose a full-reference (FR) IQA method to tackle the challenge of C-IQA. Specifically, the proposed method extracts edge and texture features to analyze the structural error. Then the moment and entropy of various color spaces are computed to reflect color distortions. Then the features are regressed into quality scores with the assistance of a support vector regression (SVR) model. Experimental results show that our metric outperforms the mainstream FR-IQA metrics, which indicates that the proposed method is more capable of modeling the visual quality loss of cartoon images.
Predicting the soil organic carbon by recent machine learning algorithms

In this paper, we focus on the Soil Organic Carbon (SOC) prediction, to discuss a comparative analysis between two recently proposed techniques, namely the Adaptive Network-based Fuzzy Inference System (ANFIS) with fractional Tikhonov regularization and the Extreme Learning Machine (ELM) with the same kind of regularization. Three groups of experiments were performed using some publicly available datasets, in particular from Estonia. The results showed the good accuracy of the ANFIS-based approach.