Reviewerly/exHarmony huggingface dataset:
Asynchronous Multi-Task Learning Based on One Stage YOLOR Algorithm

The You Only Learn One Representation (YOLOR) approach is an object detector that can encode implicit knowledge and explicit knowledge of multiple tasks simultaneously. However, the requirement of jointly feeding data is not a friendly setting for an edge device due to the high computational cost. A better strategy is to learn the concept of the new task on the device individually, one by one, without access to the old data. In other words, the model has to deal with multiple tasks asynchronously. In this work, we extend the multi-purpose network YOLOR to asynchronous multi-task learning to learn domain invariant features, which focus on capturing the relatedness between the weight of the previous task and data of the subsequent task. Further, as the number of tasks gradually increases, we accumulate significant weights by introducing task-specific masks and expert modules; the former can automatically identify important filters to prevent modification caused by new tasks, and the latter address the kernel space misalignment problem to perform multi-task feature selection. We experimentally demonstrate that the proposed training strategy significantly outperforms the traditional solution in learning multiple tasks at different times on a public dataset, which supports that the proposed approach is more competitive for resource-limited edge devices.
Research on Alignment Method of Civil Aviation Equipment Domain and General Knowledge Graph Entity Based on Graph Neural Network Embedding

Because of the lack of description information, significant attribute differences and substantial relationship heterogeneity existing in the alignment of domain knowledge graphs and general knowledge graphs, the attribute information is embedded through the graph attention mechanism, the entity representation vector obtained by attribute embedding is used as the initial vector of relation embedding, and the graph attention network is used for relation embedding, an entity representation vector containing attribute and relationship information is obtained. The entity representation vector is aligned by similarity calculation. Experimental results on the alignment dataset CAKG-ONTK demonstrate that the proposed model outperforms other typical graph neural network-based entity alignment models. To validate the effectiveness of the relationship embedding method used in this paper, experiments are conducted using GCN, R-GCN, and GAT, which are graph neural network models. The results show that the relationship embedding method proposed in this paper achieves the best performance. Furthermore, to assess the robustness of the model, experiments are conducted using different proportions of entity seed pairs, as well as different hyperparameters. The results demonstrate that the model exhibits strong robustness. Compared with the mainstream alignment model based on graph neural networks, this model has a specific improvement in the Hist@1 and Hist@10 on the domain knowledge graph and general knowledge graph alignment dataset.
Improving Real-time Communication for Educational Metaverse by Alternative WebRTC SFU and Delegating Transmission of Avatar Transform

Maintaining real-time communication quality in metaverse has always been a challenge, especially when the number of participants increase. We introduce a proprietary WebRTC SFU service to an open-source web-based VR platform, to realize a more stable and reliable platform suitable for educational communication of audio, video, and avatar transform. We developed the web-based VR platform and conducted a preliminary validation on the implementation for proof of concept, and high performance in both server and client sides are confirmed, which may indicates better user experience in communication and imply a solution to realize educational metaverse.
Path planning of Automatic Guided Vehicle Based on Voronoi Global Planner

This paper utilized an Autonomous Mobile Robot (AMR) equipped with dual Lidars to construct a map. After building the map, several path planning methods were compared for navigation. The goal was to investigate a path planning method based on Voronoi global planner to find a short movement time with the good stability to reach the destination. The purpose was to enable AMR to find suitable map building and navigation methods in different environments and requirements.
Unifying Local and Global Fourier Features for Image Classification

In the last decade, Convolutional Neural Networks (CNNs) have become a dominant algorithm in solving various domains such as computer vision, self-driving cars, medical imaging, and natural language processing. The core operation of the CNNs is convolution layer that can aggregate input features around local windows in a short-range manner and learn relative positions inside each window. For long-range modeling, common CNNs stack a bunch of convolutional layers that result in high computational costs to enlarge receptive field. Recently, Vision Transformers (ViTs) and its improvements have outperformed CNNs in the rankings of language, vision, and audio research. The main goal of the ViTs is that the model can extract short-range and long-range features in one layer. With this strategy, the network structure of the ViTs is simpler than CNNs. However, ViTs have quadratic complexity with the spatial length of the input feature. In the last year, many methods are proposed to relax the cost of ViTs and bring complicated designs of CNNs into ViT-based models. Inspired by the insightful properties of the ViTs and CNNs, this paper introduces a Local and Global Fourier Network (LGFNet) that jointly learns local and global receptive fields in the frequency domain rather than the spatial or time domain in conventional CNNs and ViTs. The input features, local, and global kernels are transformed to the frequency domain through Fast Fourier Transform. The local features are learned by a convolution between the input feature and local kernels. Concurrently, matrix multiplication between the input feature and global kernels is performed to extract low frequencies from the input Fourier feature. Since local and global Fourier features are complementary, the LGFNet efficiently fuses these information by summation operation based on the similarity degrees of the input signals. Therefore, our LGFNet performs unified representation from the input feature. To evaluate the effectiveness of the proposed method, we conduct experiments on the large-scale dataset ImageNet1k and the small dataset CIFAR100. As a result, the LGFNet surpasses the ViT-based models by a clear margin under similar parameters and GFLOPs.
Voice-Powered 3D Object Generation for the Metaverse

This research proposes a method for generating or modifying 3D objects using a human's voice as input for a machine learning model. The input voice is translated into text using a Google API and processed by a BERT-based text encoder to comprehend each phrase and generate the desired 3D object. The method is designed to make it easier for people with no design knowledge to create 3D objects and contribute to the development of the Metaverse, a virtual reality beyond reality. The shape generation model has been trained on a dataset of 15,038 shapes with 75,344 natural language descriptions, and the system includes a shape auto-encoder to generate different styles of 3D shapes. The proposed method is the first to use speech recognition for the generation of 3D objects in the Metaverse and aims to generate higher quality and faster results than existing methods using text for 3D object generation.
Online Object Tracking on Multiple Cameras with Completely Overlapping Views

This paper presents a tracking solution for multiple objects in a multi camera setup with overlapping fields of view. We utilize this not only to extend the overall operational area but also to improve tracking accuracy in the overlapping areas in case of occlusions and other practical issues. Our method lifts the usual bipartite approach to the multipartite domain using GMMCP which matches detections with their counterparts in all cameras and with their corresponding Kalman based track. This has two important properties, namely online and real-time capability, and it generates an optimal solution to the original problem without simplifications or approximations. It thus outperforms clustering and the bipartite methods as well as an offline method based on the k-shortest path algorithm.
A Real-time System of Two-stage Track Component Classification based on YOLOX-nano and ResNet34

Due to the complex track environment, the use of only one stage for track component identification is prone to inaccurate component positioning, resulting in misjudgment of the target coordinate system and other problems. This paper proposes a two-stage recognition method based on YOLOX-nano and ResNet34, hoping to solve the problem of inaccurate component positioning in the existing classification system and also improve the recognition accuracy. In the first stage, the entire image is preliminarily screened through YOLOX-nano, so that the system can understand the image structure, obtain the possible range of components, and then obtain the leftmost and rightmost positions of the track through Hough Transform. Next, calculate the intersection with the sleeper range obtained in the first stage, and calculate the possible relative position of the component base on the intersection, thereby locking the range where the component is located, and handing this range to ResNet34 in the second stage for component defect detection.
Occurrence Probability of SNI in Accessing Service for Improving Accuracy of Service Identification

Identifying the service to be connected from an IP flow is beneficial for various applications, and a method of identifying the service based on the SNIs occurring in the flow was proposed. It was also shown that the identification accuracy could be improved by excluding some SNIs from those for identification. However, no discussion has been published on the probability of SNIs occurring in actual service accesses. In this paper, we compare the probability of occurrence of each SNI in each service and show that there are redundant SNIs whose probabilities are the same, and there is one SNI that occurs in all services with similar probabilities and is not useful for service identification. We then show that identification accuracy can be improved by excluding the SNI that occurs in all services from the service identification data.
Image Inpainting Using MSCSWin Transformer and Color Correction

Image inpainting has been researched for many years. From traditional methods to current CNN models, they all pursue two targets (structural stability and texture consistency). In this paper, we propose the multi-shift CSWin Transformer (MSCSWin Transformer) and the HSV loss to focus on colors to inpaint images for these two targets. At last, we compare our model with state-of-the-art methods on the Places2 dataset to confirm our proposed module is indeed working.
The influence of different visual cues of women's bags on consumers' attention

E-commerce has become a major shopping channel for people in recent years, and women's bags are one of the most popular products in online stores. This study uses eye-tracking data to analyze the effects of the brightness of product images and the legibility of text within products on subjects' attention and visual tendency. The results showed that more visual attention was obtained when the brightness of the product photo was increased and placed on the right side. Harder-to-read fonts were placed on the left side to get more visual attention. Regardless of the visual presentation, when placed on the right side, a change on the original Z-shaped reading line was observed.
Optimizing Transaction Fee for Different Decentralized Applications in Ethereum

Typically, users can pay higher transaction fee to reduce confirmation time while submitting a transaction in Ethereum. However, the rising transaction fee becomes a major issue. Therefore, a fine-grained gas price suggestion tool is expected to various decentralized applications (dApps) with different Quality of Services (QoS) requirements. In this paper, we proposed a machine learning based gas price prediction approach for optimizing transaction fee in Ethereum. The proposed approach takes features of dApps into consideration. The experimental results show that the proposed approach can provide fine-grained transaction fee estimation for dApps with various QoS requirements.
Visual Traits-Based Recommendation System for Proactive Retailing in Physical Store Environment

A comprehensive visual traits-based recommendation system is designed for proactive retailing in a physical store environment. The proposed system utilizes computer vision algorithms to analyze various visual traits of customers, including facial features, clothing and accessories, to provide targeted product recommendations. The system does not require customers to provide any personal information, making it a less intrusive and more hassle-free approach compared to the conventional membership approach. By providing tailored product suggestions, the system offers retailers the opportunity to enhance their customers’ shopping experience and increase sales. This paper provides a detailed description of the system's design and setup.
Compensation Method of Quantized Deep Learning Models for Edge Devices

Quantization is one of the optimization methods for developing deep learning models for edge devices. Through converting the floating-point into 8bit integer or even lower bitwidth, the model’s storage size can be reduced. As the rounding error exists during the quantization process, the model performance decreases. As a result, a method that can recover model performance is needed. In this research, a compensation method for improving the performance of quantized deep learning models is proposed, which make the quantized model can achieve equal or even better performance compared to the original floating-point model.
A Group Recommendation System for Movies Using Deep Learning

Recommender systems are widely used to assist users in finding potentially interesting content. Most methods analyze personal preferences and provide users with personalized recommendation results. The personal preferences are also easily influenced by their relatives and friends. In this paper, we propose a group recommendation system based on graph attention network. The first stage is to group users according to the mean genre score of the movies, and the similarity between users. The second stage is based on a graph attention network (GAT), we predict users’ rating for the movies by the preferences of the users and group relationship. The simulation results show that the accuracy of our recommendation mechanism is better than the existing recommendations.
Social distance estimation system using low-resolution IR images with Boosting-CNN

Low-resolution infrared (IR) images potentially provide information for estimating social distance between people without capturing personally identifiable information. In this study, we have implemented a social distance estimation system using Boosting-CNN (AdaBoost-based convolutional neural network). Furthermore, the system is embedded on a low-power microcontroller. We have compared the inference time between Boosting-CNNs and single CNNs processed on the microcontroller, and have found that specific Boosting-CNNs had similar performance and relatively shorter inference time than single CNNs. We have also constructed a Boosting-CNN using CNN models with different parameters, and it has been observed that the accuracy is improved by the order of training.
Imbalanced Data for Knowledge Tracing

In the realm of Intelligent Tutoring Systems (ITS), Knowledge Tracing (KT) techniques play a vital role in tracking and assessing student progress and understanding of a subject. However, in practice various data classes are generally collected in a way of underrepresentation, leading to the KT performance degradation. In this work, we propose a data deduplication technique to balance the inputs to improve the KT performance. Our experimental results confirm the efficacy of the proposed scheme in addressing imbalanced data and improving KT performance.
A File Verification Scheme Based on Verkle Trees

Recently, information security has become an important issue. People receive more and more information and often download lots of data and files through the Internet, but there is no way to verify the correctness of the data source. In this paper, we introduce a file verification mechanism based on Verkle trees without the intervention of third-party institutions. Furthermore, it will restore the tampered files through the blockchain network and Inter-Planetary File System (IPFS). In addition to designing the file verification mechanism, we will discuss the overall efficiency and compare the differences between the Verkle tree and Merkle tree architectures.
Adaptive Scale Selection Network for Crowd Counting

Crowd counting is a computer vision task that focuses on accurately estimating the number of people present in a given scene. In the past few years, convolutional neural network-based deep learning techniques have achieved remarkable success in many computer vision tasks, including crowd counting. In the field of crowd counting, large-scale changes have always been a great challenge. To resolve this problem, previous work used multiple branches to obtain information at different scales and combined it. However, purely combining multi-branch features cannot effectively utilize multi-scale information. In this work, we modify the previous multi-branch architecture, which can reasonably select the appropriate scale information. Furthermore, we test our model on the ShanghaiTech dataset and demonstrate the competitive performance of our method.
Applying Super Resolution and Optical Flow to Implement a Roll Call System for a Small Classroom by Webcam

Traditional facial recognition technology has been widely adopted in the technology industry and has almost achieved maximum recognition accuracy. However, it is still challenging to implement a cost-effective attendance system using low-cost webcams in a small classroom measuring approximately 8.5m*6m that can accommodate 20-25 people. The primary reason for this is that webcams are designed for personal use and can only recognize faces up to about 2 meters, making it impossible to recognize the faces of everyone in the classroom. In this study, we utilized YOLOX for small face detection, improved the features of distant small faces through Super-Resolution (SR), extracted facial features, and finally used Dlib for facial recognition. We also used OpenCV's optical flow to track the person's body frame's movement. Therefore, even if the person looks down or turns around, our system can still determine whether the person is still in the classroom. Experiments have demonstrated that this system can extend the webcam range to approximately 4-5 meters. Additionally, under clear facial conditions, this system can recognize whether the person has closed their eyes, serving as a reference for classroom participation.
IoT-Enhanced Remote Proctoring: A New Paradigm for Remote Assessment Integrity

COVID-19 has hastened the adoption of online learning and teaching worldwide, and across all levels of education. Many educational institutions have adopted online proctoring solutions to conduct and ensure academic integrity during online assessments. As most online proctoring solutions include the installation of various libraries or dependencies, this introduces risk to assessment integrity, where students with ill-intentions seek to tamper with these installations in an attempt to bypass measures put in place to deter cheating. To tackle the issue at hand, we developed a proof-of-concept internet-of-things (IoT) tool called RAPID that leverages unconventional cybersecurity and malware techniques for remote proctoring. Our solution does not require prior installation of any software on the test-taker’s machine yet is able to monitor surface level and internal activities on a Windows operating system. Future works on the proof-of-concept could extend the solution to other popular operating systems, as well as capture and analyze more data with greater efficiency.
Unified Object Detection Model for Image Style Convertible Modalities

Object detection is an important task in computer vision. In earlier studies, object detection mostly works on single-modal. But recently, in order to improve detection efficiency, researchers start adding more modals for improving the robustness of detection result. In this paper, we assume that all input modals which can convert to image style are able to complete object detection task through a unified model. We choose YOLOv7 as our baseline model and use RGB and infrared images from the public dataset Camel to test our hypothesis. Based on our experiment results, training RGB and IR images together obtains better performance than training individually, which shows that all image style modalities are possibly handled by a unified model and even achieve better results.
Anomaly Web Page Detection Using HDBSCAN and Deep SVDD

In this paper, we propose a method for determining the abnormal web pages using a Deep support vector data description (SVDD) and HDBSCAN (Hierarchical Density-based Spatial Clustering of Applications with Noise). Deep SVDD replaces the kernel of the SVDD with a neural network and minimizes hyperspheres containing normal data. The proposed method applies HDBSCAN, which is density-based clustering method, to remove abnormal data from the training data before using Deep SVDD. Through experiments on a web page dataset, we showed that HDBSCAN can remove anomalous data and that the performance of Deep SVDD is stabilised by using the removed data as training data.
FaceNet-cGAN: A Deep Learning-based Approach for Masked Face Recognition

Masked faces can pose a challenge for security systems, making it easier for individuals to disguise themselves and evade identification. To address this issue, developing a system that can recognize masked faces can help in identifying potential security threats and preventing criminal activities. Therefore, this research proposes a deep learning-based approach, namely FaceNet-based conditional generative adversarial network (FaceNet-cGAN), to detect and remove masks, which would generate a high-quality, unmasked facial image and improve the accuracy of facial recognition systems. The proposed FaceNet-cGAN approach demonstrated superior performance compared to other existing methods, as evidenced by the obtained SSIM score of 92.2% and PSNR value of 29.97 dB.
Implementation of a Quality Evaluation System for Chest Compression based on OpenPose Model

In emergency medicine, CardioPulmonary Resuscitation (CPR) is usually a necessary and urgent first aid measure for patients whose heart stops beating due to diseases or accidents. However, the success of CPR in saving more patients depends on the popularity of CPR education and training. The more popular the education and training is, the more people can become rescuers in emergency situations and perform CPR on patients to improve their survival rate. In this work, we implement a CPR quality evaluation system, in which there are four modules, they are the pose detection module, the compression frequency module, the compression depth module, the visualized feedback module, respectively. In addition, we build the visualized real-time feedback system to present the overall information of chest compression of learners, including that whether the compression pose is correct or not, the frequency of compression, the depth of compression and the position of compression.