Reviewerly/exHarmony huggingface dataset:
Parameter Optimization Clustering Federated Learning Algorithm Based on Data Heterogeneity

Federated learning, as a new privacy-preserving distributed learning paradigm, can break data silos and train better models. In the case of client heterogeneity, the significant difference in data distribution among different clients makes the model convergence speed significantly lower, which leads to data drift. To address the impact of data heterogeneity in federated learning, a density-based federated clustering method, FedDec, is pro-posed. FedDec firstly divides the hyperparameters of the overall clients into different clusters by using the extreme value clustering method to ensure that the clients within each cluster have a high degree of similarity, and at the same time safeguard the data privacy and security. The parameter optimization is performed for each client to improve the accuracy of the client model before going to participate in model aggregation. And it is compared with some classical algorithms on the public datasets MNSIT and CFAR-10, and the accuracy is improved on both the MNIST dataset and CFAR-10 dataset.
Multi-Objective Fine-Grained Neural Architecture Search

The computational complexity of modern deep learning models is extremely high. For onboard robot calculators, their use is impossible without additional manipulations. It is promising to create a neural network for a specific task with appropriate requirements for the complexity of the model. Neural architectural search methods on popular datasets demonstrate promising results in creating small-sized architectures. In practice, not all of them can be used for their own purposes due to their high demands on computing resources at the search stage. In this paper, we propose an algorithm for reducing the computational complexity of the model based on solving a multi-objective optimization problem. The result of the proposed method is a set of trained networks with a different ratio of quality and computational complexity. The obtained results of practical experiments on own data confirm the effectiveness of the proposed method. With a slight loss in quality, it was possible to reduce the computational complexity of the network several times in comparison with the original architecture.
A 5G Slice Traffic Anomaly Detection Method Based on Convolution Neural Network

As the network security threats caused by the Internet of Things (IoT) continue to increase, the attack surface continues to expand and the network heterogeneity increases, and the use of virtualization technology and distributed architecture increases. Protecting the future network will become a challenging field. This paper proposes the approach of converting 5G slice traffic into images for analysis by a CNN (Convolutional Neural Network) model, and uses NAS (Neural Architecture Search) to detect anomalous network traffic. In order to improve the accuracy and efficiency of the intrusion detection system, this paper uses SDS(Software Defined Security) as a means to design a scalable, high-efficiency 5G traffic defense system. In the multiclass classification, the accuracy of CNN model is 94.9\%. The results from this method are promising as the model has performed in multiclass classification with an accuracy of 96.4\%. Furthermore, we suggest the optimal CNN design for the better performance through numerous experiments.
Research on Gesture Recognition and Classification Algorithm in Classroom Interaction Based on Convolutional Neural Network

As a very important part of the teaching process, classroom interaction can promote students’ participation in class and improve the teaching effect. Gesture recognition has the potential to be widely used in classroom interaction. Through the gesture recognition classification algorithm based on convolutional neural network, students’ gestures in classroom interaction can be automatically recognized and classified, so as to achieve a more intelligent interactive experience, assess students’ participation and understanding more accurately, adjust teaching strategies in time and provide personalized guidance. This paper discusses the research of gesture recognition classification algorithm in classroom interaction. Firstly, the importance and application scenarios of gesture recognition in classroom interaction are analyzed, and the basic principle and characteristics of gesture recognition classification algorithm are introduced. Secondly, the network model and experimental process based on 3D-CNN and ConvLSTM in this paper are discussed in detail, including data collection and preprocessing, network structure design and parameter tuning. Finally, the effectiveness and accuracy of the proposed algorithm are verified by experiments.
A Self-Supervised Denoising Method for pCLE Images

When using probe based confocal laser endomicroscopy (pCLE) to diagnose diseases, the fiber bundle makes a honeycomb pattern on the result which causing significant impact on the clinical diagnosis. Therefore, one of the main challenges of this technique is the quality of images. Recently, deep learning technique have made significant progress on image denoising. However, acquiring noise-clean training image pairs for each scene is costly in practice. There have been a few works that allow training a denoising network on signal noisy image. Taking one step further, this paper propose a novel self-supervised denoising method by designing a image quality assessment network to cooperate with the denoising network. Compared with previous self-supervised denoising methods, our method shows better results on pCLE images and fluorescence microscope images.
High-fidelity Facial Exchange: Exploration and Application of Model Innovation and Super-resolution

Face swapping technology has specific and important application prospects in the age of artificial intelligence, and with the development and advancement of computer technology, more and more researchers are investing in this area of research. However, previous research results are difficult to synthesize quickly with high fidelity while focusing on the high quality of the face. This study proposes a high-fidelity and realism synthesis method based on traditional model innovation while combining facial super-resolution techniques. Specifically, we first optimise and improve the original algorithm, and after using the model synthesis, the model is enhanced frame by frame using a face facial image recovery model for facial resolution, after which the results are combined for video. At the same time, we propose a subjective evaluation method based on the quality of facial generation, further assessing the effect of the synthesis quality by combining objective and subjective aspects. Through experimental comparison, this implementation outperforms the more mainstream models in terms of quality and efficiency, providing new ideas and methods for the development of facial synthesis technology.
Improved Residual Cumulative Resampling PF-PHD Filter Based on Stratified Rough Sorting

Resampling is the main step that constrains the hardware implementation of Particle Filter- Probability hypothesis density(PF-Phd). This step requires particle deletion and replication, which brings huge latency and resource consumption. Based on this, a improved Residual Cumulative Resampling (RCR) method based on stratified rough sorting is proposed in this paper. The stratified rough sorting method adds the operation of stratifying the particles according to the weight on the basis of the original rough sorting method to ensure that the large weight particle set always lies above the small weight particle set in the sorting process, which improves the sorting stability and ensures the tracking accuracy. The experimental results show that the RCR method based on stratified rough sorting outperforms the original rough sorting method in both OSPA distance and target number estimation, and the specific effect is comparable to Multinomial Resampling and Systematic Resampling, which can improve the tracking accuracy while preserving the simple structure of rough sorting.
A Study of Student Action Recognition in Smart Classrooms Based on Improved SlowFast Swin Transformer

Understanding actions in videos remains a significant challenge in computer vision, and Transformer-based methods have emerged as leading approaches for image recognition. In comparison to traditional image recognition, video action recognition is more difficult due to its higher dimensionality and larger dataset. Consequently, we introduce SF-Swin based on the Transformer framework, which incorporates SlowFast’s fast and slow pathways to address spatial structure and temporal events. Additionally, we employ lateral connections to fuse the information from both pathways. We also introduce Smoothed Sample Loss (SSLoss) to handle the long-tail distribution problem. Lastly, we design and created a student action dataset utilizing authentic real college classroom surveillance videos. SF-Swin was evaluated on the Kinetics dataset and our student action dataset, demonstrating improved accuracy without a substantial increase in computational effort.
Differential Activation-Based Facial Attribute Editing Using Semantic Segmentation

Face attribute editing aims to change various attributes in the face image and save the specific information of the image, such as hair accessories, jewelry, and background. The most effective approach is differential activation, which aims to accurately activate and edit specific regions of face images during editing. The activation area of the existing differential activation modules often exceeds the scope of the face, resulting in the loss of image information around the face. To solve the above problems, we propose a novel segmentation-based differential activation module, which can more accurately lock the activation position within the face and effectively preserve other information. Experimental results outperform state-of-the-art editing methods in both qualitative and quantitative evaluations.
Copyright Protection Scheme for QR Codes under Unextended Visual Cryptography

Visual cryptography is a type of image encryption technology with higher levels of security, concealment, and ease of use than traditional encryption algorithms. Therefore, visual cryptography can be combined with digital watermarking technology to enhance the security of copyright protection processes. Currently, all types of digital watermarking schemes directly embed the copyrighted image into the carrier image, which cannot avoid passive attacks from malicious extraction. This paper proposes a copyright protection scheme for QR codes under unextended visual cryptography to address the issue of copyright protection. On the one hand, using the shadow image generated by visual cryptography for watermark embedding, even if maliciously extracted, will not disclose any content of the copyrighted image; On the other hand, a third-party copyright authentication center is introduced to authenticate the identity of the copyright owner, which enhances the security of copyright image restoration. Finally, the proposed scheme divides the QR code that displays necessary information into blocks and applies discrete cosine transform. The watermark is embedded into the feature value matrix obtained from the singular value decomposition of the central block, ensuring both robustness and invisibility of the watermark while avoiding malicious removal. After experimental results, it was shown that using the watermark embedding algorithm proposed in this paper for copyright image embedding, the copyright image has excellent robustness, invisibility, and higher security.
A New Raw Holoscopic Image Simulator and Data Generation

A holoscopic imaging system is a method for capturing three-dimensional information with a single camera and aperture, using a micro-lens array between the camera's main lens and image sensor.Due to the lack of holoscopic cameras on the market, researchers need a holoscopic simulator and a dataset to work with.Most of the available holoscopic simulators and datasets generate up-sampled viewpoint images, which are extracted from the holoscopic raw image.This paper proposes a new holoscopic simulator that generates a raw holoscopic image with variable resolution more efficiently.The proposed holoscopic simulator captures a 3D model, generates viewpoint images with a proper baseline separation using a Blender add-on function, and uses a multi-view stereo algorithm to generate ground-truth depth maps.To generate a raw holoscopic image, viewpoint image pixels are remapped to their associated micro-lenses, producing two forms of holoscopic images, a grid of viewpoint images, and a grid of elemental images.The holoscopic simulator is fast and efficient compared to existing holoscopic simulators.A comprehensive holoscopic dataset is also presented, including raw holoscopic 1.0 images and associated viewpoint images at multiple resolutions, as well as a holoscopic toolbox for viewpoint images and elemental images.The proposed holoscopic simulator and dataset offer a valuable resource for researchers interested in holoscopic imaging and provide a new approach to generating holoscopic images and associated data, potentially facilitating research in related areas and advancing the field of holoscopic imaging.
Joint Parameter Estimation of Mixed Blur for Image Restoration

The simultaneous existence of motion blur and defocus blur constitutes the mixed blur, which is a great challenge to image deblurring. Therefore, it is necessary to establish the blur model and estimate blur parameters to perform image restoration. In this paper, we propose a joint parameter estimation method for the mix blurred image. For the defocus radius, we derive it based on the correlation between the blurred image and the defocus kernel in spectrum domain. For the motion length, we first roughly estimate it from the spectrum feature. Based on this value, we choose whether to use the correlation in spectrum or make the peak detection in cepstrum to derive the exact motion length. This joint estimation method improves the accuracy when the motion length is small. The experimental results show that the estimation accuracy of motion length and defocus radius is improved, and the estimation range of radius is expanded.
Video Content Understanding Based on Spatio-Temporal Feature Extraction and Pruning Network

The feature extraction method of traditional video description usually does not consider the correlation between temporal feature and spatial feature, resulting in insufficient comprehensive feature extraction and insufficient correlation. At the same time, the existing network parameters make the model over-rely on the parameter weights during training, which is easy to overfit and make the description inaccurate. In this paper, we design a feature extraction network based on spatiotemporal attention and a new pruning strategy to improve the accuracy of description. First of all, the key frames in the video are extracted by using the time-attention mechanism, and then the important regional information and background information of the key frames are extracted. Then the image features are extracted after the two features are closely related by the spatial fusion function. Secondly, by using variational discard method, the model adaptively adjusts the discard rate of neurons to select an optimal value, which effectively solves the problem of overfitting and makes the output description of the model more accurate. Experimental results on MSVD, a dataset widely used in this field, show that the proposed method can significantly improve the accuracy of video description.
EU-Net: Boundary Enhancement Based Medical Ultrasound Image Segmentation Network

U-Net segmentation network is an effective medical image segmentation model. U-Net, as a fully convolutional network, is vulnerable to the limitation that convolution cannot extract image context information well. Therefore, this paper proposes a boundary enhancement based medical ultrasound image segmentation network: EU-Net, which uses the ConvMixer module at the bottom of the encoder to extract the global contextual information of the image. And a depthwise separable convolution is used in the jump connection to help the network capture the effective features in each layer of the encoder without increasing the computational effort. Ultrasound images are a non-invasive and painless diagnostic modality, but since the ultrasound imaging process is prone to noise, the segmentation of ultrasound images is prone to unclear boundary segmentation, so an active boundary loss (ABL) is added to guide the prediction of boundary movement during each iteration, thus improving the segmentation details of the network. The evaluation on an open source breast ultrasound dataset shows that EU-Net achieves an average IOU of 0.684 and Dice of 0.809.
Drone-YOLO: Improved YOLO for Small Object Detection in UAV

In recent years, object detection in the context of UAV images has seen tremendous progress. However, small object detection still presents a great challenge. The poor performance of detectors on small objects can be attributed to multiple reasons, on one hand, deep networks' abilities are inherently limited with regard to small objects due to having multiple layers of abstractions, on the other hand, most training datasets suffer from class and size imbalance which hinders the model's ability to generalize over a wide range of scales. To overcome these limitations, and improve the accuracy of detection for small objects, we introduce an improved feature pyramid structure based on attentive feature fusion factor, lightweight joint attention to guide multi-scale feature fusion across multiple FPN layers to avoid small object information loss. We also propose a novel copy-paste data augmentation scheme to mitigate the size imbalance issue across datasets allowing better small objects contribution to the overall training loss. We evaluate our model on both MS COCO and VisDrone datasets, experiment results on MS COCO show an improvement of 1.4% and 2.7% on <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{mAP}_{small}$</tex> and AP respectively compared to the baseline. On VisDrone we achieve competitive results compared to SOTA detectors.
A Spatio-Temporal Interactive Network Based Sobel Guided Optical Flow for Facial Depression Analysis

The proficient utilization of machine learning has demonstrated the conspicuous presence of depressive features within facial region. Compared to dynamic descriptors, optical flow is more capable of capturing temporal nuance. However, the complexity of face makes it difficult to extract pure optical flow. In the paper, we propose a spatio-temporal interactive network (STI-Net) for facial depression analysis. By utilizing the SEInceptionV3 to construct the STI-Net backbone including temporal stream and spatial stream, it is more effective to capture facial spatio-temporal characteristics in parallel and achieve tight information interactivity. In the representation of temporal clues, the Sobel guided optical flow (SGOF) is introduced to enhance the edge, which can reduce facial complexity and simplify it to the contour variations. For the feature fusion, the adaptive weighting module (AWM) can assign different weights to temporal and spatial features to explore their correlations. Experiments on AVEC2014 datasets have shown that our method outperforms other visual-based methods in exploring and fusing facial features, and the proposed SGOF can provide guidance for the further application of optical flow in depression recognition.
Generating Chinese Character Skeletons with Parallel Random Recurrent Networks

Chinese character skeletons are key topological description of Chinese character glyphs which provided essential information about structures of the Chinese characters. The Chinese character skeleton can be regarded as sequential points generated using deep neural networks, such as recurrent neural networks (RNN) while time intervals between sequential points which included dynamic information of handwriting process of Chinese characters could not be presented elegantly using RNN. Deep neural networks with long sequence outputs face problems such as gradient vanishing or explosion. Chinese characters had longer sequential points than those of English characters. Here, we generated the Chinese character skeletons using multiple parallel random recurrent networks (RRN) which provided rich dynamic presentation of sequential points. The generation took place at two levels, the stroke level which included a sequence of points, and the character level which included a sequence of strokes. Our method not only generated temporal information of the skeletons but also retained more skeleton feature points with only small-scale training data which made the generated Chinese character skeletons had richer writing detail features than existing methods. Our method avoided the problems of gradient vanishing or explosion, and could be used to quickly generate large-scale and high-quality Chinese character skeletons.
Lattice-Based Privacy Enhanced Identity Protocol for SDO Services

The computational efficiency and storage consumption of the existing lattice-based privacy enhanced identity (LEPID) protocol cannot meet the requirements of the devices in the Internet of Things (IoT). In this paper, we design an efficient lattice-based EPID protocol for Secure Device Onboard (SDO) service (SDO-LEPID). The proposed SDO-LEPID protocol improves lattice-based direct anonymous attestation, and a signature-based revocation approach is added. The experiment results show that when the dimension of the polynomial ring is 512 and the number of signature revocation list users is 1000, the signing speed of SDO-LEPID is about 22 times that of LEPID, the verification speed is about 35 times that of LEPID, the private key size of SDO-LEPID is 6% of LEPID, and the identity certificate size is 9% of LEPID.
A Hybrid CNN-LSTM Model for Video-Based Teaching Style Evaluation

Intelligent teaching style evaluation has grown in popularity. The video is one of the most accessible resources for evaluation. Yet, there is currently no comprehensive, highly accurate method for processing teaching videos. Thankfully, there are numerous top-notch deep learning-based video categorization methods available as references. A hybrid CNN-LSTM model that is effective at modeling time-series teaching style information is what we propose, drawing on the benefits of video classification research. Our efforts' effectiveness is demonstrated through experiments on a dataset we created for in-class teaching.
Emotion Recognition from Facial Images of People in a Mixed Reality Helmet

The research presented in the paper demonstrates how different machine learning methods handle the task of emotion recognition (emotion classification) on facial images of people in a mixed reality helmet. We developed different types of features extracted from facial image to be used as input for machine learning methods. Features were based on the location of facial anthropometric points. Different types of features allowed for the creation of multiple datasets. To optimise the hyperparameters of the proposed methods, we used various strategies in order to maximise classification metrics. The paper shows results of conducted experiments using different machine learning methods, boosting algorithms and also ensembling methods.