Reviewerly/exHarmony huggingface dataset:
Engagement detection and enhancement for STEM education through computer vision, augmented reality, and haptics

STEM (Science, technology, engineering, and mathematics) based education is being revolutionized with the rapid development of the applications of intelligent Haptics. Theoretical knowledge may now be put into practice, and complex ideas can now be presented using three-dimensional models that are enhanced in the actual environment via Augmented Reality (AR) and computer Haptics to enhance the student's interest in concepts and studies. This article presents a novel design and development of an AR and Haptics-based STEM product comprising of World Map with Haptic feedback using Vuforia, Unity 3D, and Open-Haptics to enhance student engagement. To detect students' engagement in online learning, a computer vision-based system is also designed and deployed in the real-time website that uses the features, such as facial emotions, pose estimation, and head rotation. Marker-based augmentation is employed, and a sense of touch is included for an immersive experience for the students with the use of the haptic Touch Omni device. Hence, the proposed approach can improve classroom learning activities by using Augmented reality and Haptics-based STEM product and by its evaluation using a computer vision system. To validate the improvement in engagement, a comprehensive user study is performed and analyzed on the proposed setup.
Privacy and Efficiency of Communications in Federated Split Learning

Every day, large amounts of sensitive data are distributed across mobile phones, wearable devices, and other sensors. Traditionally, these enormous datasets have been processed on a single system, with complex models being trained to make valuable predictions. Distributed machine learning techniques such as Federated and Split Learning have recently been developed to protect user data and privacy better while ensuring high performance. Both of these distributed learning architectures have advantages and disadvantages. In this article, we examine these tradeoffs and suggest a new hybrid Federated Split Learning architecture that combines the efficiency and privacy benefits of both. Our evaluation demonstrates how our hybrid Federated Split Learning approach can lower the amount of processing power required by each client running a distributed learning system, and reduce training and inference time while keeping a similar accuracy. We also discuss the resiliency of our approach to deep learning privacy inference attacks and compare our solution to other recently proposed benchmarks.
Virtual, Augmented, and Mixed Reality for Human-robot Interaction: A Survey and Virtual Design Element Taxonomy

Virtual, Augmented, and Mixed Reality for Human-Robot Interaction (VAM-HRI) has been gaining considerable attention in HRI research in recent years. However, the HRI community lacks a set of shared terminology and framework for characterizing aspects of mixed reality interfaces, presenting serious problems for future research. Therefore, it is important to have a common set of terms and concepts that can be used to precisely describe and organize the diverse array of work being done within the field. In this article, we present a novel taxonomic framework for different types of VAM-HRI interfaces, composed of four main categories of virtual design elements (VDEs). We present and justify our taxonomy and explain how its elements have been developed over the past 30 years as well as the current directions VAM-HRI is headed in the coming decade.
Slight Aware Enhancement Transformer and Multiple Matching Network for Real-Time UAV Tracking

Based on the versatility and effectiveness of the siamese neural network, the technology of unmanned aerial vehicle visual object tracking has found widespread application in various fields including military reconnaissance, intelligent transportation, and visual positioning. However, due to complex factors, such as occlusions, viewpoint changes, and interference from similar objects during UAV tracking, most existing siamese neural network trackers struggle to combine superior performance with efficiency. To tackle this challenge, this paper proposes a novel SiamSTM tracker that is based on Slight Aware Enhancement Transformer and Multiple matching networks for real-time UAV tracking. The SiamSTM leverages lightweight transformers to encode robust target appearance features while using the Multiple matching networks to fully perceive response map information and enhance the tracker’s ability to distinguish between the target and background. The results are impressive: evaluation results based on three UAV tracking benchmarks showed superior speed and precision. Moreover, SiamSTM achieves over 35 FPS on NVIDIA Jetson AGX Xavier, which satisfies the real-time requirements in engineering.
Comparing technologies for conveying emotions through realistic avatars in virtual reality‐based metaverse experiences

Abstract With the development of metaverse(s), industry and academia are searching for the best ways to represent users' avatars in shared virtual environments (VEs), where real‐time communication between users is required. The expressiveness of avatars is crucial for transmitting emotions that are key for social presence and user experience, and are conveyed via verbal and non‐verbal facial and body signals. In this paper, two real‐time modalities for conveying expressions in virtual reality (VR) via realistic, full‐body avatars are compared by means of a user study. The first modality uses dedicated hardware (i.e., eye and facial trackers) to allow a mapping between the user's facial expressions/eye movements and the avatar model. The second modality relies on an algorithm that, starting from an audio clip, approximates the facial motion by generating plausible lip and eye movements. The participants were requested to observe, for both the modalities, the avatar of an actor performing six scenes involving as many basic emotions. The evaluation considered mainly social presence and emotion conveyance. Results showed a clear superiority of facial tracking when compared to lip sync in conveying sadness and disgust. The same was less evident for happiness and fear. No differences were observed for anger and surprise.
Color Image Cryptosystem Based on Sine Chaotic Map, 4D Chen Hyperchaotic Map of Fractional-Order and Hybrid DNA Coding

With advancements in computer and communication technologies, the production, utilization and applications of digital images is at an unprecedented rate. Recent applications include military communications, remote sensing, novel engineering designs storage and communications, as well as medical imaging. In most cases, such images convey highly sensitive or confidential information, which creates a strong need for the design of secure and robust color image cryptosystems. Recent literature has shown that fractional-order functions exhibit improved performance over their corresponding integer-order versions. This is especially true in their use in image processing applications. In this research work, we make use of a four-dimensional (4D) hyperchaotic Chen map of fractional-order, in conjunction with a sine chaotic map and a novel hybrid DNA coding algorithm. A thorough numerical analysis is presented, showcasing the security performance and efficiency of the proposed color image cryptosystem. Performance is gauged in terms of resilience against visual, histogram, statistical, entropy, differential, as well as brute-force attacks. Mean values of the metrics computed are as follows. MSE of 9396, PSNR of 8.27 dB, information entropy of 7.997, adjacent pixel correlation coefficient of 0, NPCR of 99.62%, UACI of 33, MAE of 80.57, and a very large key space of 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">744</sup> . The proposed image cryptosystem exhibits low computational complexity, as it encrypts images at a rate of 4.369 Mbps. Furthermore, it passes the NIST SP 800 suite of tests successfully. Comparison of the computed metrics of the proposed image cryptosystem against those reported in the state-of-the-art by counterpart algorithms show that the proposed cryptosystem exhibits comparable or superior values.
Regret theory-based multivariate fusion prediction system and its application to interest rate estimation in multi-scale information systems

Estimating interest rates is a typical multivariate prediction problem that has garnered considerable attention in the finance industry. However, the rising complexity of the prediction environment has presented significant obstacles to the methodology and application of traditional single-scale rational prediction models. Considering this challenge, the paper presents a new multivariate fusion prediction system that aims to tackle the following issues: (1) How to mine prediction information from multi-scale information systems (MSISs)? (2) How to describe the irrational behavior of decision-makers (DMs) in prediction processes? (3) How to improve the prediction performance and generalization capability of the models? Specifically, the score function of each feature under MSISs is primarily established by applying scale rules, regret theory (RT), and dominance relations to thoroughly explore prediction information across varying scales. Second, an adaptive S3WD model is developed for feature selection, which leverages the feature score distribution and the sequential three-way decision (S3WD) concept to avoid the problem of dimensional disasters, thereby improving the accuracy of selected features and reducing the learning cost. Furthermore, to enhance the effectiveness and rationality of the kernel extreme learning machine (KELM), an improved version known as IKELM is developed, which combines the regret-based comprehensive score of each sample. Finally, a multivariate fusion prediction system is constructed and abbreviated as an S3WD-IKELM-PSO system, where the particle swarm optimization (PSO) algorithm is employed to determine optimal parameters. It is worth noting that actual interest rate estimation datasets are utilized to verify the feasibility and validity of the proposed prediction system. According to the experimental results, the S3WD-IKELM-PSO system exhibits superior prediction performance and generalization ability compared to existing machine learning models.
From teacher to teacher-trainer: A qualitative study exploring factors contributing to a successful train-the-trainer digital education program

This paper explores the experience of 14 teacher-trainers involved in a train-the-trainer digital education program in Switzerland. Data collected between August 2019 and June 2021 through focus groups and collaborative portfolios were analyzed through thematic analysis. A qualitative analysis was used to identify the main factors facilitating or hindering their experiences. These facilitators and obstacles were related to individual characteristics, interactions with stakeholders, instructional content, and logistics. Based on these findings, recommendations are provided to help training designers to maximize the success of such initiatives.
Assessing Convolutional Neural Networks Reliability through Statistical Fault Injections

Assessing the reliability of modern devices running CNN algorithms is a very difficult task. Actually, the complexity of the state-of-the-art devices makes exhaustive Fault Injection (FI) campaigns impractical and typically out of the computational capabilities. A possible solution consists of resorting to statistical FI campaigns that allow a reduction in the number of needed experiments by injecting only a carefully selected small part of it. Under specific hypothesis, statistical FIs guarantee an accurate picture of the problem, albeit selecting a reduced sample size. The main problems today are related to the choice of the sample size, the location of the faults, and the correct understanding of the statistical assumptions. The intent of this paper is twofold: first, we describe how to correctly specify statistical FIs for Convolutional Neural Networks; second, we propose a data analysis on the CNN parameters that drastically reduces the number of FIs needed to achieve statistically significant results without compromising the validity of the proposed method. The methodology is experimentally validated on two CNNs, ResNet-20 and MobileNetV2, and the results show that a statistical FI campaign on about 1.21% and 0.55% of the possible faults, provides very precise information of the CNN reliability. The statistical results have been confirmed by the exhaustive FI campaigns on the same cases of study.
One-stage self-supervised momentum contrastive learning network for open-set cross-domain fault diagnosis

Intelligent fault diagnosis models based on transfer learning achieve cross-domain fault identification under small training samples. Existing cross-domain diagnosis models assume the source and the target data share the health condition label space. However, new fault types will occur in the application stage, and it is usually defined as an open-set fault diagnosis problem. In this study, a one-stage self-supervised momentum contrastive learning model (OSSMCL) is proposed for open-set cross-domain fault diagnosis. We propose a self-supervised contrastive learning based momentum encoder to capture the distinguishable features between sample pairs. Different from common self-supervised pipelines, the proposed method is a one-stage framework fusing a meta-learning paradigm, through which OSSMLC can learn to identify new faults with a few labeled samples in the target domain. The one-stage framework can achieve cross-domain fault diagnosis directly without network fine-tuning, and it will reduce the risk of overfitting under limited target data. The effectiveness of the proposed model is evaluated by three open-set cross-domain fault diagnosis experiments. Compared with the state-of-the-art methods, the proposed model can obtain higher diagnostic accuracy in the open-set scenario with small training samples.
ACNS Conference on Cloud and Immersive Technologies in Education: Report

This article serves as an introduction to a collection of selected papers presented at the ACNS Conference on Cloud and Immersive Technologies in Education (CITEd 2022), which took place in Kyiv, Ukraine, on December 21-22, 2022. The collection provides valuable insights into the latest advancements in cloud and immersive technologies in the field of education. The introductory text offers an overview of the conference events and provides concise summaries of the selected papers, offering a glimpse into the diverse topics and research areas covered. These selected papers reflect the cutting-edge research and innovative approaches showcased at CITEd 2022, making them a valuable resource for educators, researchers, and practitioners interested in exploring the potential of cloud and immersive technologies in education.
The influence of anthropomorphic appearance of artificial intelligence products on consumer behavior and brand evaluation under different product types

While artificial intelligence products are widely used in the market, their anthropomorphic appearance design is becoming a frontier issue in product strategy and consumer behavior research. The aim of this study was to investigate the influence of anthropomorphic appearance on consumer behavior and brand evaluation under different AI product types. It was conducted in China, a new but rapidly-growing country in the field of Internet, AI technology and AI product consumption. This study conducted four situational experiments with a 2 (anthropomorphic design: anthropomorphic vs. non-anthropomorphic) × 2 (product type: hedonic vs. utilitarian) between subjects’ experimental design. Data was collected from 1172 Chinese “Digital Natives” by using a structured questionnaire. The findings revealed that for hedonic AI products, anthropomorphic appearance improves consumers' purchase intention and brand evaluation through perceived entertainment, and intelligence level significantly moderates the mediating effect of perceived entertainment; while for practical AI products, anthropomorphic appearance improves consumers' purchase intention and brand evaluation through perceived usefulness, and intelligence level does not significantly moderate the mediating effect of perceived usefulness. There is no significant moderating effect of intelligence level on perceived usefulness. The study contributes to development and validation of a more comprehensive understanding and theoretical foundation of anthropomorphism, and furthermore explores the impact of anthropomorphic appearance on consumer behavior and brand evaluation under different AI product types. This study also provides insights for companies to apply anthropomorphic strategies.
Multimodal learning on graphs for disease relation extraction

Disease knowledge graphs have emerged as a powerful tool for artificial intelligence to connect, organize, and access diverse information about diseases. Relations between disease concepts are often distributed across multiple datasets, including unstructured plain text datasets and incomplete disease knowledge graphs. Extracting disease relations from multimodal data sources is thus crucial for constructing accurate and comprehensive disease knowledge graphs. We introduce REMAP, a multimodal approach for disease relation extraction. The REMAP machine learning approach jointly embeds a partial, incomplete knowledge graph and a medical language dataset into a compact latent vector space, aligning the multimodal embeddings for optimal disease relation extraction. Additionally, REMAP utilizes a decoupled model structure to enable inference in single-modal data, which can be applied under missing modality scenarios. We apply the REMAP approach to a disease knowledge graph with 96,913 relations and a text dataset of 1.24 million sentences. On a dataset annotated by human experts, REMAP improves language-based disease relation extraction by 10.0% (accuracy) and 17.2% (F1-score) by fusing disease knowledge graphs with language information. Furthermore, REMAP leverages text information to recommend new relationships in the knowledge graph, outperforming graph-based methods by 8.4% (accuracy) and 10.4% (F1-score). REMAP is a flexible multimodal approach for extracting disease relations by fusing structured knowledge and language information. This approach provides a powerful model to easily find, access, and evaluate relations between disease concepts.
The “Collections as <scp>ML</scp> Data” checklist for machine learning and cultural heritage

Abstract Within cultural heritage, there has been a growing and concerted effort to consider a critical sociotechnical lens when applying machine learning techniques to digital collections. Though the cultural heritage community has collectively developed an emerging body of work detailing responsible operations for machine learning in galleries, museums, archives, and libraries at the organizational level, there remains a paucity of guidelines created for researchers embarking on machine learning projects with digital collections. The manifold stakes and sensitivities involved in applying machine learning to cultural heritage underscore the importance of developing such guidelines. This article contributes to this need by formulating a detailed checklist with guiding questions and practices that can be employed while developing a machine learning project that utilizes cultural heritage data. I call the resulting checklist the “Collections as ML Data” checklist, which, when completed, can be published with the deliverables of the project. By surveying existing projects, including my own project, Newspaper Navigator, I justify the “Collections as ML Data” checklist and demonstrate how the formulated guiding questions can be employed by researchers.
Breast cancer diagnosis using hybrid AlexNet-ELM and chimp optimization algorithm evolved by Nelder-mead simplex approach

This study proposes a Hybrid AlexNet-Extreme Learning Machine (ELM) approach for breast cancer diagnosis using mammography images. Batch normalization is applied to improve AlexNet's performance, and the chimp optimization algorithm (ChOA) is utilized to avoid sub-optimal solutions in ELM. The Nelder-mead simplex (NEMS) technique is then employed to enhance the convergence behavior of ChOA. The study's main contributions are the proposed hybrid model and the application of ChOA and NEMS techniques to improve the performance of ELM. The proposed model is evaluated using the CBIS-DDSM dataset, with wiener and CALHE filters used as preprocessors. The effectiveness of the classification is examined using five optimization algorithms, and several metrics. The outcomes demonstrate that CALHE filter offered the best performance overall, and AlexNet-BN-ELM-CHOA-NEMS was the most accurate of the five models, with a sensitivity of 96.03 %, a specificity of 94.60 %, and an overall accuracy of 95.32 %. The findings demonstrate the effectiveness of the proposed model in breast cancer diagnosis.
Deep learning supported mitoses counting on whole slide images: A pilot study for validating breast cancer grading in the clinical workflow

Breast cancer (BC) prognosis is largely influenced by histopathological grade, assessed according to the Nottingham modification of Bloom-Richardson (BR). Mitotic count (MC) is a component of histopathological grading but is prone to subjectivity. This study investigated whether mitoses counting in BC using digital whole slide images (WSI) compares better to light microscopy (LM) when assisted by artificial intelligence (AI), and to which extent differences in digital MC (AI assisted or not) result in BR grade variations.Fifty BC patients with paired core biopsies and resections were randomly selected. Component scores for BR grade were extracted from pathology reports. MC was assessed using LM, WSI, and AI. Different modalities (LM-MC, WSI-MC, and AI-MC) were analyzed for correlation with scatterplots and linear regression, and for agreement in final BR with Cohen's κ.MC modalities strongly correlated in both biopsies and resections: LM-MC and WSI-MC (R2 0.85 and 0.83, respectively), LM-MC and AI-MC (R2 0.85 and 0.95), and WSI-MC and AI-MC (R2 0.77 and 0.83). Agreement in BR between modalities was high in both biopsies and resections: LM-MC and WSI-MC (κ 0.93 and 0.83, respectively), LM-MC and AI-MC (κ 0.89 and 0.83), and WSI-MC and AI-MC (κ 0.96 and 0.73).This first validation study shows that WSI-MC may compare better to LM-MC when using AI. Agreement between BR grade based on the different mitoses counting modalities was high. These results suggest that mitoses counting on WSI can well be done, and validate the presented AI algorithm for pathologist supervised use in daily practice. Further research is required to advance our knowledge of AI-MC, but it appears at least non-inferior to LM-MC.