Reviewerly/exHarmony huggingface dataset:
Aerial Object Detection by UAV Based on Improved YOLOv7

Aerial object detection is more difficult compared to general object detection. Indeed, aerial object detection remains a challenging task in computer vision object detection because it is highly susceptible to complex backgrounds. In addition, aerial targets often contain many small objects, and the limited pixel area of these small objects makes it challenging for current mainstream algorithms to detect them effectively. To solve the issues mentioned above, we propose a target detection algorithm called PM-YOLOv7 based on YOLOv7 for aerial images captured by unmanned aerial vehicles (UAVs). In terms of feature extraction, the original P <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">5</inf> feature extraction layer is removed. Based on this, a novel parallel backbone network is proposed. It integrates high-level and low-level features from two nearly identical backbone networks to handle complex images in aerial photography captured by UAVs. In terms of enhancing feature extraction, MLVC is used to capture global contextual information and long-term dependencies of input images. Experimental results on the challenging VisDrone2021 test-dev show that our proposed PM-YOLOv7 can achieve accuracy performance gains compared to YOLOv7.
Dynamic Language Refinement for Graceful Degradation in Scene Text Recognition

Scene text recognition is a mature technology which can get a near-perfect result in regular scenes. However, there are still troubles in recognizing scene texts with image blur, characters missing, and occlusion. Recent end-to-end frameworks have investigated the importance of linguistic knowledge, utilizing either implicit or explicit language models. However, joint training of language and vision models tends to interfere the tunings of each other. In this paper, we present a novel gradient isolation method to train language and vision models separately, yet in an end-to-end architecture. In addition, we propose a dynamic iterative training procedure for graceful refining the language model. Extensive experiments confirm that the proposed method has superiority on regular and irregular scene text images and achieves state-of-the-art results.
Photo Album Creation for Historical Newspaper through Computer Vision by Using ABBYY, VGG Model, and Yolov5

In this paper we presented a novel approach with Computer Vision and Machine/Deep Learning for building digital image albums and storytelling with images from historical newspapers. In total 456 newspaper pages have been used to execute the proposed method. Different methodologies, including ABBYY FineReader, VGG nls-chapbook model, and Yolov5, were compared and evaluated. Image categorization has also been studied with Face Recognition and OCR methodologies. Experimental results showed that the proposed method successfully extracted and categorized the illustrations of 106 actors/actresses which were then used to create their personal photo albums. Our experiment with image detection for photo album building demonstrated the potential of our approach to fit other newspaper or digitized collections. The result photo album of historical personalities allowed users to have a more comprehensive understanding of the newspaper article mentioned about the actors/actresses and their multifaceted biographical information as well.
Multi-scale and Feature Fusion for Expression Recognition Networks

This paper proposes a method for face expression recognition that combines multi-scale region awareness with multi-level feature fusion. It aims to overcome the challenge of extracting features from face regions that vary greatly in scale. Firstly, the paper uses an inverse residual structure to perform feature extraction in a high-dimensional space, compensating for the network’s lack of feature extraction capability. Secondly, the paper introduces multi-scale region awareness to address the multi-scale nature of face regions, using multi-scale feature extraction modules to extract features from different receptive fields, as well as multi-scale down-sampling to reduce the feature loss caused by pooling down-sampling. Thirdly, channels and spatial attention are introduced to suppress invalid regions. Finally, adaptive multi-level feature fusion maximises feature information at different depths of the network while avoiding inconsistencies between features. Experimental results demonstrate the effectiveness of the proposed method, achieving accuracy of 73.66%, 98.98%, and 88.07% in FER2013, CK+, and RAF-DB, respectively.
Scaling Factor and Shift Factor Based Neural Network Pruning

As the accuracy of deep neural network models continues to improve, the depth of the network tends to be deeper. Thus, the requirement of storage and computational resource makes the network more and more difficult to deploy. Compressing neural networks before deployment is an effective method to save computational resources. In this paper, we propose a channel pruning algorithm using the scaling factor and shift factor of the batch normalization layer, which is used to calculate the significance of each channel and remove the less important ones, compressing the deep network model with a slight decrease in accuracy. We apply the algorithm to the Yolov5 model on the Visdrone dataset, and the experiments demonstrate that the number of parameters is reduced by 45.18%, 81.42% and 91.02% when the pruning ratios are 50%, 90% and 95%, respectively, which is better than the previous algorithm.
Quantum Joint Distribution Adaptation on the Universal Quantum Computer

Transfer learning (TL) is one of the most important sub-realms in the field of machine learning. The goal is to achieve knowledge transfer from source domain to target domain by utilizing the similarities among data, tasks, or models. Domain adaptation (DA) is actually a type of transductive TL with assumption that both the marginal and the conditional distribution adaptation are different. In this paper, the process of joint distribution adaptation implemented on the universal quantum computer is presented. Compared to the classical joint distribution adaptation, the performance and application scope of the algorithm are promoted by the advantages of the quantum operations. Specifically, the quantum joint distribution adaptation algorithm proposed in our work can be performed exponentially efficient with quantum computing techniques in contrast to the corresponding classical procedure.
Using Markov Random Field (MRF) Hypergraph Transformer Method for Visual Question Answering (VQA) Application

Visual Question Answering (VQA) is an important task that combines Computer Vision and Natural Language Processing to enable computers to comprehend and answer questions based on image content. However, current VQA models exhibit certain limitations that hinder their practical applicability in real-world scenarios. In this paper, we propose a novel graph-based VQA model, the MRF hypergraph Transformer, which addresses three main issues of existing graph and transformer-based VQA models. Firstly, the lack of feature extraction from images and questions, which we address by replacing the vision-language extractor with CLIP and BERT. Secondly, the restriction of graph-based models to processing pairwise relations, which we overcome by introducing the MRF hypergraph, a new data structure that enables the modeling of high-dimensional relations. Finally, the limited feature fusion module, which we address by considering both graph fusion and vision-language fusion. Our proposed method achieved an impressive accuracy of 69.65% and 73.19% on the VQA v2 validation dataset and test dataset respectively, representing significant improvement compared to the best graph-based VQA models available. Our proposed method has potential applications in human-computer interaction scenarios, such as post-disaster damage assessment and assisting visually impaired people in overcoming visual barriers.
Marine Shellfish Entity Recognition Based on BiLSTM-CRF Model

Named entity recognition is an important research area in natural language processing. For marine shellfish biological identification tasks, this article builds a Bilstm-CRF model for physical identification tasks. The data set used in the training is a self-built data set MSDataset.The experimental results are:After 50 rounds of training, the average F1 value of the various entity recognition results was 89.75%, which was 18% and 5.5% higher than the 71.75% of HMM and 84.25% of CRF respectively, indicating that the entity recognition could be effectively improved after adding the BiLSTM network layer.Meanwhile, the overall results of accuracy and recall of the BiLSTM-CRF model were better than those of the HMM and CRF models. Therefore, the BiLSTM-CRF model can effectively achieve the entity recognition of marine shellfish organisms, and effectively improve the entity recognition accuracy of marine shellfish organisms.
Learning-Based Collision Detection with Estimating Occluded Keypoint for Robotic Manipulators

The increasing importance of industry 4.0 expanded the applications of robotic manipulators. Due to the increase in the number of robotic manipulators at work, the probability of collision becomes a prominent threat. As a result, the urgency for real-time collision detection rises. Thus, we proposed a learning-based collision detection with estimating occluded keypoint for robotic manipulators based on the DeepLabCut toolbox keypoint model and depth camera. By identifying key points in robotic manipulators, we detect collisions through the distance formula and occlusion estimation. The experimental results indicate that method achieves 88.14% accuracy with a speed of 22.91 frames per second (FPS). Due to the characteristics of accuracy and high operation speed, it is suitable for real-time collision detection in realistic environments.
DFF: Distributed Forward-Forward Algorithm for Large-Scale Model in Low-Performance Devices

The advancement of Artificial Intelligence(AI) technology, especially with the success of Large Language models(LLMs) like ChatGPT and image generation models such as Stable Diffusion, signifies that AI has entered a new era of large-scale models. However, the rapid development of AI models has brought about the disaster of unlimited growth in model size such as the latest LLM model, PanGu-Sigma, which has reached an astonishing size of 1085B. For ordinary individuals, the enormous hardware costs for these large-scale models make it impossible to train and infer a large model. Fortunately, Geoffrey Everest Hinton has proposed the Forward-Forward(FF) algorithm, which aims to deconstruct a large model into smaller layers and each layer updates its weights through its own backpropagation process, which means that each layer is independent. Therefore, inspired by Geoffrey Hinton’s work, this paper introduces a new Distributed Forward-Forward(DFF) algorithm, which distributes each layer of the Forward-Forward algorithm model to different devices based on ROS2’s publisher-subscriber communication mechanism. Then, the model can be computed across different devices. The Distributed Forward-Forward(DFF) algorithm system in this paper theoretically allows multiple low-performance devices to jointly run large models that a single device is hard to run. Classification model tests on MNIST and CIFAR10 datasets show that the accuracy is 0.9292 and 0.4249 respectively, which has the comparable performance of the pure Forward-Forward algorithm (0.9315 and 0.4310). Our implementation is available at [Github].
Impact Investigation of Adversarial Samples on CNN-Based SAR Image Target Detectors

In recent years, deep learning algorithms based on convolutional neural networks (CNN) have made breakthroughs in the field of target detection in Synthetic Aperture Radar (SAR) images. However, CNN may be vulnerable to adversarial examples, raising security concerns for practical deployment. To gain a deeper understanding of adversarial examples and provide a theoretical basis for building robust detection networks, this study comprehensively investigates the impacts of adversarial samples generated by deep recognition networks (DRN) in SAR images on the performance of target detection networks with different principles, using YOLOv3 and Faster-RCNN as examples. Experimental results on the MiniSAR vehicle target dataset show that adversarial samples pose a more serious threat to the single-stage detection network YOLOv3. Simply adding small perturbation that are invisible for human eyes to the SAR image can mislead the detector into missing the majority of targets. In contrast, two-stage detection networks like Faster-RCNN exhibit stronger robustness with their detection performance being less influenced by the adversarial examples generated by different DRN.
Target Tracking Techniques for UAV Aerial Images

Aiming at the requirements of target tracking applications of UAV aerial images, the traditional target tracking methods have problems such as too strong dependence on the target motion model, significantly reduced accuracy in occlusion or interference, difficulty in judging tracking failure, and poor real-time performance, which are difficult to meet the practical application requirements. In this paper, a fast target tracking model based on Siamese neural network is designed to make the algorithm have high real-time performance. Combined with actual application requirements, two usage modes are designed, one is automatic target recognition then entering tracking, and the other is manually selecting targets then entering tracking. Based on the actual UAV aerial image data and the public infrared dataset, the average tracking success rate can reach more than 90%, and the test performance on the NVIDIA Jetson Xavier NX board can reach 22 frames per second, which can meet the real-time requirements.
Balanced RoIMasking for Salient-Instance Segmentation

We explore salient-instance segmentation from novel balance perspectives. Salient-instance segmentation includes salient detection and salient segmentation sub-tasks. We focus on the RoIMasking layer in S4Net, which outputs the training data for salient segmentation. Considering the balance between sub-tasks, we propose an expanded smooth RoIMasking (SM) for salient segmentation. SM smooths the training data of salient segmentation, which can reduce the complexity of segmentation and increase the importance of detection in the whole task, leading to the improvement of salient detection sub-task. Moreover, the existing salient segmentation sub-model cannot recover from the salient detection error. We attribute this to the extreme imbalance distribution of salient segmentation training data. To address this problem, an expanded mutilated RoIMasking (MM) is proposed, which introduces false negative samples into training data and balances the distribution of salient segmentation data. Additionally, due to the lack of public dataset for salient-instance segmentation, we build a new dataset: Instance-MSRA1000. By simply embedding SM and MM into the existing network, a improved method with 4.4 points mAP (mean Average Precision) gains can be achieved compared to S4Net. The experimental results on the existing public and new built datasets show that SM and MM significantly performs well.
Dialogue Generation with Skeleton Extraction and Causal Intervention

Retrieval and re-generate, which generates a natural language sentence through re-writing a retrieved sentence using a sequence-to-sequence generation structure, has been proven to be effective in dialogue response generation. However, the rewriting component tends to keep irrelevant information from the retrieval results, leading to low relevance in the re-written results. To address this problem, we propose a new framework (name), which extracts the skeleton from the retrieval results and uses it along with the query to generate responses. Specifically, we use the invariant learning method to capture the skeleton of the retrieval results. This solution employs causal intervention methods to identify noises in the retrieval results, enabling the generation component aware of what to keep and what to generate in the re-writing stage. Experimental results show that our approach improves the relevance and diversity of the generated results compared to the existing generative models.
Metaverse: A Bibliometric Analysis by Internet Crawler and Big Data Technology

Digital platforms have been widely adopted during Covid19 pandemic, but individuals and organizations have encountered various challenges with these digital platforms. Metaverse is a new Internet evolution that addresses the challenges of supporting humans in seamlessly moving between digital spaces and the physical world. Many bibliometric studies have been published on Metaverse, but previous findings on this topic are inconsistent and contradictory. Therefore, we explored trends in Metaverse research through bibliometric analysis. An online search was conducted on 16 February 2023. A total of 706 journal documents were obtained from the Scopus database. The results revealed that China was the most productive country with three China productive institutions, two China productive authors, and four China funding sponsors. This bibliometric analysis results confirmed that the rate of Metaverse research has sharply increased. The top nine most cited documents were classified as secondary research. Taking a socio-technical design approach, the Metaverse research areas identified by these review articles can be grouped into four main levels of computing: mechanical, informational, psychological, and social. The limitations of this bibliometric study were also identified.
Feasibility Study on Enhancing User Experience of Smart Cinemas Using Internet of Things (IoT) Technology

The feasibility of enhancing the user experience of smart cinemas was explored by using the Internet of Things (IoT) technology. As the audience expects better movie quality and service, traditional cinemas face the challenge of providing more personalized, comfortable, and interactive experiences. IoT technology, as an advanced smart solution, has been widely applied in various fields and has shown immense potential in the cinema industry. In this study, focusing on the feasibility, the application of IoT technology was investigated to the current user experience in smart cinemas. By employing IoT sensors, actuators, and cloud servers, smart cinema seats can be personalized based on individual settings stored in the cloud, thus meeting the specific needs of each audience member. The present research result on the feasibility of user data use indicated that the current level of user competence and the availability of IoT technology was supportive. IoT technology enhances the user experience of smart cinemas. With the application of innovative technologies, cinemas cater to audience demands with a more comfortable and personalized movie-watching experience and promote the sustainable development of the cinema business.
An Adaptive Dark-Channel Prior Dehazing Algorithm Based on STM32

To solve the problems caused by low contrast and unclear details in existing defogging algorithms, a dark channel prior adaptive defogging algorithm based on STM32 was proposed in this study. The algorithm improved the original defogging model with an adaptive threshold to adjust the transmission rate. The proposed algorithm avoided the possibility of being detected as white areas, reduced deviation, and effectively saved the required computing resources. At the same time, the post-defogging image was optimized according to the edge characteristics of the image. In the experiment, images with different brightness and complexity were processed by the proposed algorithm for better defogging effects in different scenes. Edge details were obtained, too with better defogging effects. The proposed algorithm laid a foundation for later video defogging.
Impact of Immersive Virtual Reality Technology on Skill Training

The integration of digital technology into teaching and self-learning has yielded remarkable results. However, there are not many research results on the utilization of new technology media in vocational skills training courses. Thus, we developed a virtual reality hygiene-skills training (VR-HST) system to assist in administering training for health skills practice tests. The objective of this study was to examine the impact of this system on learners' learning effectiveness and motivation during vocational skills certificate testing. We recruited 60 student participants from a fashion modeling department who were preparing to take a hygiene skills practice test. In this research, we adopted a quasi-experimental design in which the experimental group received VR-based training, whereas the control group received training based on traditional teaching methods. The experiment was conducted over 6 weeks. The results indicated that the experimental group with the assistance of the VR-HST learning improved their motivation and outperformed the control group in hygiene skills practice. Therefore, incorporating the VR-HS system into the traditional master-apprentice skill teaching and training effectively enhanced the training effectiveness.
New Experience to Operate Traditional Glove Puppetry on Digital Interactive Stage

Puppetry is the most distinctive folk craft and local opera in Taiwan. With the development of digital media and changes in entertainment patterns, troupes have chosen to develop towards theatrical performances. They emphasize the combination of puppet body language and story situations to become a new performance type. Using Radio Frequency IDentification (RFID) as the interface between the puppet and stage scenario, an interactive puppet show device "Puppet come on stage" was designed and manufactured. We also designed exclusive audio and video, background, and special effects for characters. Four aspects were defined for the evaluation: operational safety, design innovation, operational convenience, and content appropriateness. The interactive performance was enabled by combining digital audio-visual media and electronic media for leading actors and puppeteers. The system allows traditional skills to continue in the future.
Research on Relationship Between Bullet Screen Attributes in Medical Popular Science Videos and The audience’s cognitive load

This article aims to explore the relationship between bullet screen attributes in medical popular science videos and the audience’s cognitive load. The evaluation indicators for cognitive load included subjects' blink frequency, gaze count, and response to task completion in eye-tracking experiments. The connection between the audience’s cognitive load and the opacity and speed attributes of bullet screens in popular science videos was investigated, and the impact of different bullet screen attributes on the audience’s cognitive load was explored. By analyzing the relationship between attributes such as bullet screen content, density, speed, and the audience’s cognitive load in watching medical science videos, the attention allocation patterns of the audience towards bullet screens and video content were analyzed using eye-tracking technology. Experimental results indicated that the opacity of bullet screens impacted cognitive load significantly with the lowest cognitive load of audiences. The speed of bullet screens also affected the audience’s cognitive load with slower speeds resulting in a less cognitive impact and lower cognitive load.