Reviewerly/exHarmony huggingface dataset:
HATCAS: A Handwritten Textual Content Analysis System

Due to the rapid increase in digital textual documents, it’s becoming difficult these days to get exact information from the heavy stream of textual data available over the web or at personal storage of mobile or computers. This resulted in development of text content analysis models to present the required information to the users quickly as per their queries. In this regard automatic content analysis on handwritten text added more challenges. Finding appropriate text contents is quite difficult when it comes to searching it from a handwritten note. In this paper, we present a Handwritten Textual Content Analysis System (HATCAS) which takes up user queries on specific topics and provides concise answers to them from available handwritten notes. The system uses handwritten notes covering different subjects to generate the answers of the user queries. The presented model is tested on varied user queries and the obtained results of the experiments show the efficiency of the model in presenting the answers with an average accuracy of 96%.
Detection of Sign Language Alphabets Using a Weighted Ensemble of Neural Networks

Speech disability is a condition that hinders a person's ability to communicate and express their thoughts through speech and hearing. Those with this impairment may use sign language as an alternative mode of communication. Despite the prevalence of sign language, non-signers may encounter difficulties in communicating with signers. However, recent advancements in computer vision and deep learning have greatly improved motion and gesture recognition. The main purpose of this project is to construct a deep learning application that can translate sign language into text, thereby facilitating communication between signers and non-signers. We have used three separate custom made CNNs (Convolutional Neural Networks), and then used them to create a weighted ensemble for detecting the sign language alphabets from a video frame. MNIST dataset is used to train the model here. The proposed CNN model provides the best accuracy of 96.23% in detecting the sign language alphabets. The system has effectiveness for recognition of hand signs.
Front and Rear Classification of Vehicles in Indian Context using Deep Neural Networks

Since the inception of deep learning, we have seen various systems that detect and classify objects into categories, an application of which includes the classification of vehicles into cars, trucks, scooters, etc. To detect whether a vehicle under consideration is moving in the correct lane, we find various hindrances such as faded Markings, shadows, different lighting conditions, rural roads, etc. Hence, one of the important intermediary tasks is to detect the view of the other vehicles that come in the vicinity of the vehicle. This paper works toward the classification of four-wheeled vehicles in visibility as front or rear for Indian roads. We’ve trained and compared three Convolutional Neural Networks for our binary-class classification problem that classifies the view of the vehicle as front or rear based on the data acquired from the dashcam in our vehicle.
Real-Time Facial Recognition Based Smart Attendance Management System Using Haar Cascading and LBPH Algorithm

Taking attendance of the students present in the classes is a regular part of an institution's day-to-day operations everywhere in the world. Traditionally, attendance is taken by a roll call or by inputting data into the computer, which takes a long time and can result in false attendance. To alleviate the time and errors associated with the traditional process, this research paper proposes a solution using real-time facial recognition. The majority of a student's daily attendance can be managed with real-time facial recognition. Face recognition refers to the process of recognizing a student's face to take attendance using face biometric data. There are several research papers that only look at student recognition rates. This study focuses on a facial recognition-based attendance system with a high confidence level and a low false-positive rate. This study demonstrates the capability of facial identification by combining the Local Binary Pattern Histogram (LBPH) algorithm and the Haar cascading algorithms because of their robustness against monotonic grayscale transformations. This provides a facial map of the individual, which aids in the post-image processing of the individual image obtained during attendance. This system can identify students even if they have facial hair or are wearing spectacles. This method's efficiency was higher when compared to traditional techniques; however, it did have several disadvantages that may be readily rectified by enhancing the environment and applying deep learning via machine computing using artificial intelligence.
A Hybrid Approach for Improving the Classification performance of Imbalanced Breast Cancer data

Breast cancer is the most common and significant cause of disease worldwide in women. The treatment of breast cancer requires lots of resources, various breast tumors affect different humans. In recent years researchers have proposed different models for the early prediction of breast cancer. The types of breast tumors are benign and malignant; the benign is innocent and does not harm that much, while the malignant is a cancerous tumor, which is the main reason for cancer in the breast region. So, the classification of the two tumors is very important various pretrained and hybrid model is already used for classification such as CNN (Convolution Neural Network)-based classification, CAD (computer-aided diagnosis), and support vector machine classification. In this paper, we have taken BreakHis dataset and balanced both the classes of tumors cell (number of images benign and malignant) using the data augmentation technique GAN (Generative adversarial network) then applied various pretrained models such as VGG 16, RESNET-50, INCEPTION V3, MOBILE NET V2, DENSENET-201 and achieve the classification accuracy of 92, 87, 90, 79, 92 percent respectively then feature extraction is performed on the data using resnet 50. The accuracy is improved drastically and observed as 96.3%.
Design of Vending Machine Using Visual Automata Simulator &amp; Finite State Machine

Several applications based on the idea of the FSM are now dominated by automata theory. In this study, we'll take a look at the engineering behind a vending machine (VM) that facilitates faster book deliveries to schools. One of the issues with virtualization is its effectiveness. Automata theory relies on the layout of something like the Finite State Machine, as well as a visual automata simulator(VAS) is utilised to help boost performance. In the end, the study will build an FSM with less variables. Our findings will show that the VM's effectiveness & development price may be enhanced as a consequence.
Image Restoration Using Deep Convolutional Neural Networks for Network Congestion Control

Network congestion is getting more and more severe day by day with the deployment of Long-Term Evaluation (LTE) and 5G as more users get added to the networks. The issue gets even more severe with the rapid growth of e-Commerce, online banking, and entertainment platforms like Netflix, Amazon Prime, and YouTube that generate a huge amount of traffic worldwide. In this paper, we purposed to address this traffic congestion issue with a hand-held solution. Here we have proposed a method that can be used for network congestion control. Simulation studies demonstrate the reliability and performance of proposed technique that works for 50 percent compression of the transmitted images which can reduce the size of the image by 50 percent by keeping quality of service. The aim is to reduce the network traffic by half in both the transmitter and receiver end.
A Comparative Analysis of Fraud Detection in Healthcare using Data Balancing &amp; Machine Learning Techniques

Frauds in healthcare sector has become a major problem and is widespread now. It is reported that total healthcare expenditure is increasing exponentially due to fraudulent claims in health insurance sector. It is a planned crime that include providers, doctors/physicians and beneficiaries etc, who together perform certain actions to make fraud claims. Most vulnerable institutions due to such frauds are the insurance companies. In this work as large data set is used so we have performed Exploratory Data Analysis (EDA) and then data preprocessing & feature engineering to create a feasible dataset for further analysis. In this proposed methodology, we have depicted a comparative analysis of results of different machine learning models using two data balancing techniques, i.e Class Weighing Scheme (CWS) and Adaptive Synthetic Oversampling (ADASYN) for oversampling along with the results of these models with unbalanced data set. Our objective is to verify the efficacy of various machine learning models in this domain.
Comparison of Different Domain Randomization Methods for Policy Transfer in Reinforcement Learning

Deep reinforcement learning algorithms have made great progress in the field of control with the help of many high-efficiency simulation environments. However, due to the difference in state distribution and dynamics, these algorithms trained in the simulation cannot be effectively applied to the real world. The ability to reduce the impact of the Sim2Real gap is critical for transferring policy from the simulation to the real world. Although there are many methods for studying the Sim2Real problem, it is difficult to evaluate the performance of different algorithms due to the different evaluation platforms and evaluation metrics. In this paper, we construct a uniform robot navigation scenario, and revisit the ability of the popular domain randomization methods to transfer the policies from the simulation to the real world under the dynamics gaps. With the analysis of the performence in the simulation environment and the real world, we provide some recommendations of the domain randomization methods and hope to make these methods more efficient to use.
Application of Simple Virtual Space Editing Technology in Museum Curation: Take the MindRoom as an Example

This article presents a simple virtual space editing design system, MindRoom, for developing museum virtual exhibitions. MindRoom is implemented with online cloud services and WebGL, including exhibition space construction module, exhibition hotspot editing module, exhibition space roaming module and exhibition space management module. Curators can build virtual exhibitions by selecting scenes, adding and editing ICONS, and binding hypermedia links to ICONS. Audience visit virtual exhibition by browsing the paintings, triggering ICONS, and interacting with hypermedia. User studies show that MindRoom enables curators to develop virtual exhibitions easily, and motivates audiences to learn when they visit.
Emergency Vehicle Identification for Internet of Vehicles Based on Federated Learning and Homomorphic Encryption

With the development of the Internet of Vehicles (IoV), its application has attracted wide attention. Emergency vehicles often have trouble moving in traffic. Therefore, the classification of vehicles into emergency and non-emergency categories is conducive to the development of IoV applications such as emergency rescue services, intelligent traffic management and autonomous driving systems. At the same time, its data is very sensitive in terms of data privacy and security issues. Federated learning, as a framework of machine learning, can be used to improve the privacy and security of data. The trained data is distributed on multiple machines to cooperate with each other for learning. In the process of federated learning, the model needs to be uploaded and downloaded. In order to ensure that the information of the model is not leaked, homomorphic encryption is used to encrypt the model to protect the information of the model. This paper presents a federated learning algorithm for IoV data privacy protection based on homomorphic encryption.
Abnormal Sound Detection of Electrical Equipment Based on Time-Spectrum Information Fusion

In the daily monitoring and maintenance of electrical equipment, the detection of abnormal sound is a very important part, and the high accuracy and efficiency of sound to distinguish the abnormal state of the equipment is more conducive to the troubleshooting of the equipment. At present, the research of algorithms for identifying the working status of electrical equipment operation sound based on machine learning and deep learning has been more widely used, but many algorithms are mainly based on frequency domain sequence modeling, ignoring the information of time sequence. Since the time-domain information in abnormal sounds is difficult to capture compared with the frequency-domain information, a time-domain information feature extraction network is designed by combining the commonly used frequency-domain analysis methods. In this paper, we propose a time-spectrum fusion-based anomaly detection optimization model that combines frequency-domain and time-domain feature fusion to improve the detection performance of anomalous sounds. Our method is experimentally validated on the DCASE 2020 Challenge Task 2 dataset, with an average AUC improvement of 1.35% and an average pAUC improvement of 1.35% compared to the current best method.
Research on target grab of leg-arm cooperative robot based on vision

Leg-arm cooperative robot can not only expand the motion space of traditional manipulator, but also make up the operation ability of traditional mobile robot. This paper studies the target grab of leg-arm cooperative robot based on vision. First of all, independently develop the leg-arm cooperative robot. Secondly, for the operation of the manipulator, a height-angle uncoupling PID algorithm is innovatively designed in the cylindrical coordinate system. The target position information obtained by the YOLOv3 algorithm is used as feedback to realize the accurate control of the manipulator, which can avoid the complex process of trajectory planning. Furthermore, combined with three-element trajectory planning method and motion control scheme under triangular gait, a target grab control strategy for the leg-arm cooperative robot is designed, including the height allocation decision maker and the target tracking controller. Finally, the proposed algorithm and schemes are verified with the entity robot.
Multiple Human Tracking Using Deep Learning with Shadow Clues

Occlusion is an inevitable problem in virtual reality systems where a single camera is used to track multiple players during the interaction. In this work, we observe that a user’s shadow is an important hint for tracking when the user is occluded. Therefore, we propose a novel tracking approach that effectively leverages the shadow information of target users, which leads to more robust tracking in complex environments. Our key idea is to train a shadow detection model based on Mask R-CNN to extract shadows from image frames. To handle different levels of occlusion, we propose to define a series of tracking statuses for occlusion representation. To better evaluate the proposed method, we also contribute a dataset that contains numerous image frames with various forms of human shadows. Experiments demonstrate that our method is not only effective for handling user tracking even in full and long-term occlusions, but also exhibits superior real-time efficiency.
Intelligent Decision-Making Method for Fault Handling Based on Knowledge Graph

The scale of the power grid continues to expand and the stability characteristics become more and more complex. Fast and effective fault handling is an important means to ensure the safe and stable operation of the power grid. A fault handling intelligent decision-making method integrating knowledge graph and data mining algorithm is proposed, and a two-level scheduling knowledge graph covering multiple subgraphs is constructed. The upper atlas calls the lower business sub atlas based on the fault handling rules obtained from historical data learning and training. The lower-level map realizes business functions such as on-duty strategy analysis, stable limit matching, and disposal strategy generation of the security control system. Mining fault handling rules based on indicators such as sensitivity, and using massive historical operation data for dynamic update of two-level knowledge graphs to solve the problem that offline handling rules do not match fault scenarios. Finally, the effectiveness of the method is verified by the actual power grid operation data.
Siamese Convolutional Neural Network Based Visual Servo for Manipulator

A visual servo algorithm based on Siamese Convolution Neural Network is proposed for the manipulator to avoid the requirement of feature extraction and feature matching in the traditional image-based visual servo (IBVS). The algorithm feeds the current image and the desired image into the network at the same time, and outputs the relative pose difference between the two images. A closed-loop control system is constructed through the pose difference, and control the end-effector of the manipulator to reach the desired position to grasp the target workpiece. Meanwhile, in order to meet the large amount of data needed in training the neural network, an algorithm to automatically generate the data set is proposed, which can avoid manual collection and labeling of the data set and greatly save the cost. The simulations show the effectiveness and accuracy of the proposed method by comparing with the traditional feature point based IBVS, and the grasping experiment shows the feasibility of the proposed method in actual practice.
HG-Net: Hybrid Coarse-Fine-Grained Gaze Estimation in Unconstrained Environments

Gaze estimation is a vital indicator of human behavior and has diverse applications, such as virtual reality, human-machine interaction, and medical analysis. However, it remains a challenging task to accurately predict the direction of a person’s gaze due to factors like unique eye appearance, varying lighting conditions, and head poses. Recent advancements in deep learning have improved appearance-based gaze estimation, but there is still room for improvement. In this paper, we propose a multi-loss convolutional neural network that combines coarse and fine classification to achieve higher accuracy. Our model uses an EfficientNet-B3 backbone and a fine-grained classifier trained with the aid of other coarse-angle units. The final prediction result uses integrated regression. We evaluate our model on four datasets collected under unconstrained settings and demonstrate its state-of-the-art accuracy of 3.89°, 6.93°, and 10.78° on MPIIFaceGaze, RT-Gene, and Gaze360 datasets, respectively. The code will be released on https://github.com/hhqiang/HG-Net/ after the paper’s acceptance.
Feeling of Control in Augmented Reality and Virtual Reality for Virtual Object Manipulation

Immersive technologies, such as augmented reality (AR) and virtual reality (VR), have received widespread attention in recent years. With the rapid advancement of head-mounted displays (HMDs) technology, AR and VR offer users the capacity to interact with virtual objects in immersive environments. Although AR and VR are both based on HMDs, their own unique features differentiate them from one another. Besides, virtual object manipulation is an essential element in AR and VR due to the immersive features, and the existing research has concentrated on enhancing the user experience for virtual object manipulation in AR and VR applications. However, there is little literature on the influence of different immersive environments on users’ feeling of control for virtual object manipulation. Therefore, this study aims to compare users’ feeling of control for virtual object manipulation between AR and VR environments via self-report and performance measures. According to the results, the study concludes that the VR environment results in a better feeling of control than the AR environment when users manipulate virtual objects with mid-air gestures.
A Method for Classifying Cognitive Load of Visual Tasks Based on Eye Tracking Features

The measurement of the cognitive load has always been a fundamental technology in human-computer interaction and human factors because cognitive overload will not only cause the reduction of work efficiency and subjective satisfaction of human-computer interaction but also may affect working safety. In this paper, we propose a method to classify the state of cognitive load by collecting eye movement data and extracting features in time domain and frequency domain. We designed a complete experimental procedure, including reading comprehension, contrast judgment, and content search, and collected four classes of experimental data under different difficulties. Some features of eye movement were selected by the ReliefF feature ranking method. Finally, we trained five classifiers to classify the eye movement data of different cognitive load levels. The experimental results show that the highest accuracy of the four classes can reach 90.6%.
MultiBrush: 3D Brush Painting Using Multiple Viewpoints in Virtual Reality

Recently, 3D modeling using the brush tool in virtual reality (VR) has been successfully explored. However, it is still difficult and time-consuming for the user to paint the designed target while seeing around the object and adjusting the object scales multiple times by grasping the entire object. To reduce the burden on users of 3D painting brushes in the VR space, we propose MultiBrush, a design approach that provides additional windows in multiple viewpoints of the target workspace. The proposed approach adopts the viewpoint opposite the object to be designed, and the bird’s-eye viewpoint captures the entire object to the normal viewpoint and displays images by superimposing the images from other viewpoints. To verify the proposed system, we conducted evaluation experiments for system effectiveness and questionnaires to obtain the subjective feedback.
Development of a Virtual Reality Game for Cultural Heritage Education: The Voyage of “Gotheborg”

Video games have been recognized as one of the most engaging forms of entertainment today, and are making a major contribution to the digital future. This research aims to combine the gaming elements of game design and virtual reality technology, and promote cultural heritage education through an engaging and fun experience, especially in a young population. Firstly, we summarized game mechanics and elements used for cultural heritage education through game product analysis and literature review. Secondly, we developed a virtual reality game for cultural heritage education that reproduced the voyage of “Gotheborg” to Canton, including game mechanic, game story, aesthetic expression and game technology. Finally, a pilot test was conducted to validate the effectiveness of our game developed. Our study contributes interdisciplinary research by bringing together currently distinct approaches from virtual reality technology, game design, cultural heritage, and education. It will bring new technology to impact on the digital media, cultural heritage education, and service industries.