Reviewerly/exHarmony huggingface dataset:
Multi-heterogeneous Data Synchronization Method for New Generation Substation Monitoring System

Abstract In principle, the new generation substation monitoring system no longer has duplicate data collection across safety partitions. To avoid information islands between nodes of different safety partitions and to realize efficient interconnection and interoperability of different types of information, this paper proposes a multivariate heterogeneous data synchronization method for the new generation substation monitoring system. Firstly, the overall framework of the structured data synchronization method, synchronization data encapsulation modeling, cache format construction, and subgroup and group packet processing are introduced; secondly, the overall architecture of the unstructured data synchronization method is introduced, including full data updating and difference data updating, and the latter improves the efficiency of data chunking and achieves the incremental synchronization process of the difference by the fast CDC method based on the identification of the local difference; finally, the test validation is carried out based on the actual data for testing and validation. The results show that the multivariate heterogeneous data synchronization methods are 100% correct, and the incremental difference synchronization saves more than 70% of the synchronization time.
Technology Enhanced Mulsemedia Learning: Insights of an Evaluation

Human-Computer Interaction (HCI) is crucial in crafting effective, cost-efficient, and user-friendly e-learning systems. In the realm of e-learning, HCI focuses on creating interfaces and experiences that optimize learner engagement, interaction, and overall learning outcomes. Educators are actively pursuing initiatives to enhance student motivation, engagement, and academic achievement, particularly in Science, Technology, Engineering, and Mathematics (STEM) disciplines. However, learners often lack active engagement with electronic content when adapting and interpreting learning materials. This paper examines the Quality of Experience (QoE) in Science education content via a Technology Enhanced Multimedia Learning (TEML) Web portal featuring multisensory effects like rosemary scent, vibration, and airflow. Our ongoing research explores learners’ emotional states during Mulsemedia-based learning. In this study, we present initial insights into the developed Mulsemedia test bed using IoT devices and a learning Web portal, based on self-assessed QoE questionnaires administered to 60 participants divided into experimental and control groups. The results indicate that both groups had a positive experience in the Mulsemedia-based learning environment, demonstrating its suitability for STEM subjects.
Variational Distillation for Multi-View Learning

Information Bottleneck (IB) provides an information-theoretic principle for multi-view learning by revealing the various components contained in each viewpoint. This highlights the necessity to capture their distinct roles to achieve view-invariance and predictive representations but remains under-explored due to the technical intractability of modeling and organizing innumerable mutual information (MI) terms. Recent studies show that sufficiency and consistency play such key roles in multi-view representation learning, and could be preserved via a variational distillation framework. But when it generalizes to arbitrary viewpoints, such strategy fails as the mutual information terms of consistency become complicated. This paper presents Multi-View Variational Distillation (MV <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> D), tackling the above limitations for generalized multi-view learning. Uniquely, MV <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> D can recognize useful consistent information and prioritize diverse components by their generalization ability. This guides an analytical and scalable solution to achieving both sufficiency and consistency. Additionally, by rigorously reformulating the IB objective, MV <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> D tackles the difficulties in MI optimization and fully realizes the theoretical advantages of the information bottleneck principle. We extensively evaluate our model on diverse tasks to verify its effectiveness, where the considerable gains provide key insights into achieving generalized multi-view representations under a rigorous information-theoretic principle.
Redefine Reading Comprehension with Tarsia: An Innovative Approach in Understanding Reading Text

In the global context, students in Malaysia scored lower than the OECD average in reading, mathematics, and science in PISA conducted in 2018. In the field of English education, reading is perceived as a boring and complex process as well as one of the challenging components where students find it difficult to comprehend and tend to give up easily when they look at complex or longer text. Hence, the purpose of this paper is to describe how Tarsia is used to facilitate students' reading comprehension and to explore the effectiveness of Tarsia in improving students' collaboration and communication in ESL classrooms. By using the ADDIE model as the research model, 34 participants from lower secondary students in Kuala Lumpur were selected based on convenience sampling. The significant impacts on students' reading comprehension were proven through pre and post-tests and a simple survey. TARSIA benefited the students in understanding the reading text easily and inculcating the elements of the 4Cs besides acting as useful teaching aid among teachers. TARSIA could potentially be game-based materials, designed for English reading classrooms for A2-B1 achievers. It is fair to deduce that TARSIA might have high commercialisation potentials.
Blendshape-Based Migratable Speech-Driven 3D Facial Animation with Overlapping Chunking-Transformer

Speech-driven 3D facial animation has attracted an amount of research and has been widely used in games and virtual reality. Most of the latest state-of-the-art methods employ Transformer-based architecture with good sequence modeling capability. However, most of the animations produced by these methods are limited to specific facial meshes and cannot handle lengthy audio inputs. To tackle these limitations, we leverage the advantage of blendshapes to migrate the generated animations to multiple facial meshes and propose an overlapping chunking strategy that enables the model to support long audio inputs. Also, we design a data calibration approach that can significantly enhance the quality of blendshapes data and make lip movements more natural. Experiments show that our method performs better than the methods predicting vertices, and the animation can be migrated to various meshes.
Exploring Co-adaptation in an Ecosystem of Dyadic Interaction

In the field of applied linguistics, Diane Larsen-Freeman is widely recognized, among her numerous contributions, as the originator of Complex Dynamic Systems Theory (CDST). Over the years since her seminal publication (Larsen-Freeman, 1997), CDST has evolved into a meta-theory that guides the field’s thinking on a variety of developmental phenomena, not least the study of second language acquisition. As its theoretical potency rises, so does the need to achieve a tangible understanding of CDST’s core constructs. In this paper, we empirically examine one such construct, co-adaptation, in the context of asynchronous dyadic interaction. A data corpus of 39 emails sent between two college students in China and the United States over the course of seven weeks is analyzed using NVivo 12. Co-adaptation is observed at the pragmatic, discourse, and linguistic levels. True to the spirit of CDST, we describe, and discuss the nuances of, our findings. Considering the novelty of our approach to data analysis, we end by reflecting on the limitations specific to our study, as well as worthwhile directions for future pursuits.
Efficient quantum algorithms to find substructures on finite algebras

When classifying a collection of finite algebras (for instance, in the computational classification of finite semifields), an important task is the determination of substructures such as the right, middle, and left nuclei, the nucleus, and the center. Finding these structures may become computationally expensive when there is no additional information about the algebra properties. In this paper, we introduce quantum algorithms than solve this task efficiently, by formulating it as an instance of the Hidden Subgroup Problem (HSP) {over Abelian groups}. We give detailed constructions of the quantum circuits involved in the process and prove that the overall (quantum) complexity of our algorithm is polynomial in the dimension of the algebra, while a similar approach with classical computers would require an exponential number of queries to the HSP function
Chinese-Vietnamese Cross-Lingual Event Causality Identification Based on Syntactic Graph Convolution

The Chinese-Vietnamese cross-lingual event causality identification aims to identify the cause and effect events from the news text describing the event information and present them in a structured form. The existing event causality extraction model faces the following two challenges: 1) The research work related to event causality extraction is mainly focused on resource-rich monolingual scenarios, and the performance of resource-scarce languages needs to be further improved; 2) Existing event causality identification methods are not good at capturing implicit causal semantic relations. Therefore, we propose a novel Chinese-Vietnamese Cross-lingual Event Causality Identification Based on Syntactic Graph Convolution. Firstly, the Chinese-Vietnamese word vectors are mapped into the same semantic space through pre-trained cross-lingual word embeddings. Then the syntactic graph convolutional neural network is used to capture the deep semantic information of the event sentence. Finally, the in-depth semantic features of event sentences in different languages are obtained by combining the cross-attention mechanism of event types. Experiment results on a self-built dataset that the proposed method outperforms the state-of-the-art models.
Adaptive Channel Pruning for Trainability Protection

Pruning is a widely used method for compressing neural networks, reducing their computational requirements by removing unimportant connections. However, many existing pruning methods prune pre-trained models by using the same pruning rate for each layer, neglecting the protection of model trainability and damaging accuracy. Additionally, the number of redundant parameters per layer in complex models varies, necessitating adjustment of the pruning rate according to model structure and training data. To overcome these issues, we propose a trainability-preserving adaptive channel pruning method that prunes during training. Our approach utilizes a model weight-based similarity calculation module to eliminate unnecessary channels while protecting model trainability and correcting output feature maps. An adaptive sparsity control module assigns pruning rates for each layer according to a preset target and aids network training. We performed experiments on CIFAR-10 and Imagenet classification datasets using networks of various structures. Our technique outperformed comparison methods at different pruning rates. Additionally, we confirmed the effectiveness of our technique on the object detection datasets VOC and COCO.
Minimum Assumption Reconstruction Attacks: Rise of Security and Privacy Threats Against Face Recognition

Facial Recognition (FR), despite its remarkable precision and advancements achieved through deep learning, exhibits vulnerability to security threats, specifically originating from deep generative models proficient in synthesizing deceptive face images. Generative Adversarial Networks (GANs) present substantial risks by showcasing the capacity to exploit potential vulnerabilities within FR systems. While the existing research primarily focuses on the scenario of a compromised database facilitating facial reconstruction attacks, it often overlooks more realistic threats where adversaries attack with a limited number of queries without breaching the database. This work introduces Minimum Assumption Reconstruction Attacks (MARA), offering a realistic attack framework against FR systems. MARA treats an attacker as a regular user interacting with the FR system’s user interface and observing the matching scores. We formulate the MARA attack as an optimization problem, aiming to find a latent vector in the $$\mathcal {W}^+$$ latent space of StyleGAN for generating adversarial face images that can bypass the targeted FR system. A latent space mining strategy is also proposed to enhance attack performance by obtaining ‘good’ initial guesses in the latent space. Our experiments show that MARA achieves performance comparable to false accept attacks while adhering to query limits and mimicking user-like interaction behavior. This study highlights the importance of considering attack models requiring minimal effort from the adversary, an essential perspective for adversarial research that seeks to guard against powerful and less resource-intensive attacks.
HelixNet: Dual Helix Cooperative Decoders for Scene Text Removal

Scene text removal aims to remove scene text from images and fill the resulting gaps with plausible and realistic content. Within the context of scene text removal, two potential sub-tasks exist, i.e., text perception and text removal. However, most existing methods have ignored this premise or only divided this task into two consecutive stages, without considering the interactive promotion relationship between them. By leveraging some transformations, better segmentation results can better guide the process of text removal, and vice versa. These two sub-tasks can mutually promote and co-evolve, creating an intertwined and spiraling process similar to the double helix structure of Deoxyribonucleic acid (DNA) molecules. In this paper, we propose a novel network, HelixNet, incorporating Dual Helix Cooperative Decoders for Scene Text Removal. It is an end-to-end one-stage model with one shared encoder and two interacted decoders for the text segmentation and text removal sub-tasks. Through the use of dual branch information interaction, we can fuse complementary information from each sub-task, achieving interaction between scene text removal and segmentation. Our proposed method is extensively evaluated on publicly available and commonly used real and synthetic datasets. The experimental results demonstrate the promotion effect of the specially designed decoder and also show that HelixNet can achieve state-of-the-art performance.
Expanding the Horizons: Exploring Further Steps in Open-Vocabulary Segmentation

The open vocabulary segmentation (OVS) task has gained significant attention due to the challenges posed by both segmentation and open vocabulary classification, which involves recognizing arbitrary categories. Recent studies have leveraged pretrained Vision-Language models (VLMs) as a new paradigm for addressing this problem, leading to notable achievements. However, our analysis reveals that these methods are not yet fully satisfactory. In this paper, we empirically analyze the key challenges in four main categories: segmentation, dataset, reasoning and recognition. Surprisingly, we observe that the current research focus in OVS primarily revolves around recognition issues, while others remain relatively unexplored. Motivated by these findings, we propose preliminary approaches to address the top three identified issues by integrating advanced models and making adjustments to existing segmentation models. Experimental results demonstrate the promising performance gains achieved by our proposed methods on the OVS benchmark.
A Multi-label Image Recognition Algorithm Based on Spatial and Semantic Correlation Interaction

Multi-Label Image Recognition (MLIR) approaches usually exploit label correlations to achieve good performance. Two types of label correlations principally studied, i.e., the spatial and semantic correlations. However, most of the existing algorithms for multi-label image recognition consider semantic correlations and spatial correlations respectively, and often require additional information support. Although some algorithms simultaneously capture the semantic and spatial correlations of labels, they ignore the intrinsic relationship between the two. Specifically, only considering spatial correlations will misidentify some difficult objects in the image. For example, different categories of objects with similar appearance and close distance are mistaken for the same category, and semantic correlations can constrain the error caused by spatial correlations. In this work, we propose a multi-label image recognition algorithm based on transformer, named Spatial and Semantic Correlation Interaction (SSCI). Transformer is used to model the internal relationship between spatial correlations and semantic correlations to improve the recognition ability of the model for difficult objects. Experiments on the public datasets MS-COCO, VOC2007 and VOC2012 show that the mAP values reach 84.1%, 95.0% and 95.4%, respectively. Compared with other MLIR algorithms, the proposed algorithm can significantly improve the recognition performance of multi-label images.
Boosting Generalization Performance in Person Re-identification

Generalizable person re-identification (ReID) has gained significant attention in recent years as it poses greater challenges in recognizing individuals across different domains and unseen scenarios. Existing methods are typically limited to a single visual modality, making it challenging to capture rich semantic information across different domains. Recently, pre-trained vision-language models like CLIP have shown promising performances in various tasks by linking visual representations with their corresponding text descriptions. This enables them to capture diverse high-level semantics from the accompanying text and obtain transferable features. However, the adoption of CLIP has been hindered in person ReID due to the labels being typically index-based rather than descriptive texts. To address this limitation, we propose a novel Cross-modal framework wIth Conditional Prompt (CICP) framework based on CLIP involving the Description Prompt Module (DPM) that pre-trains a set of prompts to tackle the lack of textual information in person ReID. In addition, we further propose the Prompt Generalization Module (PGM) incorporates a lightweight network that generates a conditional token for each image. This module shifts the focus from being limited to a class set to being specific to each input instance, thereby enhancing domain generalization capability for the entire task. Through extensive experiments, we show that our proposed method outperforms state-of-the-art (SOTA) approaches on popular benchmark datasets.
RDDPA: Real-time Defect Detection via Pruning Algorithm on Steel Surface

Real-time object detectors deployed on general-purpose graphics processing units (GPUs) or embedded devices allow their mass usage in industrial applications at an affordable cost. However, existing state-of-the-art object detectors are difficult to meet the requirements of high accuracy and low inference latency simultaneously in industrial applications on general-purpose devices. In this work, we propose RDDPA, a fast and accurate defect detection framework. RDDPA adopts a novel end-to-end pruning scheme, which can prune the detection network from scratch and achieve real-time detection on general-purpose devices. Additionally, we have developed a new training scheme to minimize the accuracy loss associated with the pruning process. Experimental results on a standard steel surface defect dataset indicate that our model achieves 79.2% mAP (mean Average Precision) at 103.7 FPS (Frames Per Second) on a single mid-end Titan X GPU as well as 40.1 FPS on a single low-end GTX 960M GPU, and outperforms the state-of-the-art defect detectors by about 20× speedup with considerable or higher accuracy.
Development of remote collaboration platform for manufacturing environment using mixed reality

The use of digital manufacturing has become increasingly important to maintain competitiveness in the market as companies embrace Industry 4.0. In most enterprises, there is an increasing demand for remote communication and collaboration as an alternative to travelling between locations. This trend is influenced by many factors, including the recent epidemic of Covid 19, the increasing cost of travel, and the desire to reduce carbon emissions. Companies can implement mixed reality technology, which can supplement the real world with virtual information and help production workers complete tasks. Mixed Reality technology is increasingly being used in industry to immerse users in semi-virtual environments such as factories. This research aims to study and develop a mixed reality solution that allows manufacturers' staff to collaborate and share resources and knowledge at a distance without needing to be in the same location. This study investigates the potential use of Microsoft HoloLens 2 headsets for creating and remotely cooperating in mixed-reality manufacturing environments. Four Unity projects were developed using imported mixed reality tools and other imported resources in Unity for HoloLens 2. Two users can interact and complete a set of tasks within the mixed reality factory while remaining physically separated. The results show that remote collaboration in mixed-reality manufacturing environments can be used in the industry to help with plant planning, learning factories, remote maintenance assistance, and monitoring. The work applies and builds on existing knowledge, demonstrating the potential for rapid development and sharing of mixed-reality manufacturing environments in remote collaboration.
Hand Gesture Translation System based on Multi-Sensor Fusion

With the development of sensors and integrated circuits, various sensors are used to detect hand gesture motion data for gesture recognition and broadly utilized in multiple sectors. However, there is a shortage of research on dual-hand gesture recognition, as most existing works focus on single-hand gesture recognition. This paper proposes that a hand gesture translation system based on multi-sensor fusion. We develop a simple data glove that utilizes motion sensors and bend sensors to collect gesture information. We compare and improve multiple algorithms commonly used for gesture recognition. We categorize 62 Chinese sign language gestures into four classes and use different algorithms for recognition based on the gesture type, which improves efficiency. In the end, the Euclidean Distance (ED) algorithm was employed for the identification of static sign language, resulting in an average recognition accuracy of 99.42%. Dynamic sign language of both hands is recognized using a CNN+LSTM model, dynamic sign language was recognized with an average accuracy of 98.61%.