Reviewerly/exHarmony huggingface dataset:
Multi-cue temporal modeling for skeleton-based sign language recognition

Sign languages are visual languages used as the primary communication medium for the Deaf community. The signs comprise manual and non-manual articulators such as hand shapes, upper body movement, and facial expressions. Sign Language Recognition (SLR) aims to learn spatial and temporal representations from the videos of the signs. Most SLR studies focus on manual features often extracted from the shape of the dominant hand or the entire frame. However, facial expressions combined with hand and body gestures may also play a significant role in discriminating the context represented in the sign videos. In this study, we propose an isolated SLR framework based on Spatial-Temporal Graph Convolutional Networks (ST-GCNs) and Multi-Cue Long Short-Term Memorys (MC-LSTMs) to exploit multi-articulatory (e.g., body, hands, and face) information for recognizing sign glosses. We train an ST-GCN model for learning representations from the upper body and hands. Meanwhile, spatial embeddings of hand shape and facial expression cues are extracted from Convolutional Neural Networks (CNNs) pre-trained on large-scale hand and facial expression datasets. Thus, the proposed framework coupling ST-GCNs with MC-LSTMs for multi-articulatory temporal modeling can provide insights into the contribution of each visual Sign Language (SL) cue to recognition performance. To evaluate the proposed framework, we conducted extensive analyzes on two Turkish SL benchmark datasets with different linguistic properties, BosphorusSign22k and AUTSL. While we obtained comparable recognition performance with the skeleton-based state-of-the-art, we observe that incorporating multiple visual SL cues improves the recognition performance, especially in certain sign classes where multi-cue information is vital. The code is available at: https://github.com/ogulcanozdemir/multicue-slr.
Full Feedback Dynamic Neural Network with Exogenous Inputs for Dynamic Data‐Driven Modeling in Nonlinear Dynamic Power Systems

Abstract Dynamic neural networks (DNNs) are widely used in data‐driven modeling of nonlinear control systems. Due to the complexity of the actual operating nonlinear power systems, rigorous dynamic models are always unknown. DNNs can focus on methods that only use input and output information to establish accurate dynamic models and reduce noise in measured data, which is called data‐driven modeling. The core of the DNN is the feedback with memory function. This paper analyzes the traditional Elman neural network (ENN) and nonlinear auto‐regression with exogenous input (NARX) neural network with different structure feedback structures, and proposes a full feedback dynamic neural network (FF‐DNN) with exogenous input. Eight different kinds of neural networks (including ENN, NARX, etc.) are compared and analyzed. The eight kinds of neural networks are applied on the experimental data of the DC‐AC inverter and the power system of Zhejiang Juchuang Smart Technology Company Park in Wenzhou. The experimental results are used to compare the performance of the data‐driven models established by eight different kinds of neural networks under different noise conditions, verify the robustness and generalization performance of dynamic data‐driven modeling based on FF‐DNN, and demonstrate the feasibility and effectiveness of FF‐DNN in actual power systems. © 2023 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.
RetinaMOT: rethinking anchor-free YOLOv5 for online multiple object tracking

Abstract In recent years, YOLOv5 networks have become a research focus in many fields because they are capable of outperforming state-of-the-art (SOTA) approaches in different computer vision tasks. Nevertheless, there is still room for improvement in YOLOv5 in terms of target tracking. We modified YOLOv5 according to the anchor-free paradigm to be on par with other state-of-the-art tracking paradigms and modified the network backbone to design an efficient module, thus proposing the RetinaYOLO detector, which, after combining state-of-the-art tracking algorithms, achieves state-of-the-art performance: we call it RetinaMOT. To the best of our knowledge, RetinaMOT is the first such approach. The anchor-free paradigm SOTA method for the YOLOv5 architecture and RetinaYOLO outperforms all lightweight YOLO architecture methods on the MS COCO dataset. In this paper, we show the details of the RetinaYOLO backbone, embedding Kalman filtering and the Hungarian algorithm into the network, with one framework used to accomplish two tasks. Our RetinaMOT shows that MOTA metrics reach 74.8, 74.1, and 66.8 on MOT Challenge MOT16, 17, and 20 test datasets, and our method is at the top of the list when compared with state-of-the-art methods.
PIHA: Detection method using perceptual image hashing against query-based adversarial attacks

As a representative adversarial example generation method in a black-box scenario, query-based adversarial attack has recently been highlighted due to its high attack success rate and feasibility. To provide the robustness against query-based adversarial attacks, the state-of-the-art defense methods use similarity of queries. Among them, two state-of-the-art defense methods have shown the good performance in the number of detected attack queries: (1) Stateful Detection; and (2) Blacklight. In this paper, we first propose two simple but effective mitigation strategies(i.e., parallelization and brightness adjustment) to show how to mitigate the robustness of the state-of-the-art defense methods. Next, we propose a new defense method, which uses the concept of perceptual hashing. Given a query image, the proposed defense method generates a hash sequence using a perceptual image hashing scheme, called PIHA, and compares the hash sequence with those of previous queries to detect query-based adversarial attacks. Here, a hash sequence has invariance to small perturbations and color changes when detecting query-based adversarial attacks. From the experimental results under various conditions, we show that the proposed defense method provides good robustness against well-known query-based adversarial attacks without or with mitigation strategies applied.
A novel imputation based predictive algorithm for reducing common cause variation from small and mixed datasets with missing values

A B S T R A C TMost process control algorithms need a predetermined target value as an input for a process variable so that the deviation is observed and minimized.In this paper, a novel machine learning algorithm is proposed that has an ability to not only suggest new target values for both categorical and continuous variables to minimize process output variation but also predict the extent to which the variation can be minimized.In foundry processes, an average rejection rate of 3%-5% within batches of castings produced is considered as acceptable and is considered as an effect of the common cause variation.As a result, the operating range for process input values is often not changed during the root cause analysis.The relevant available historical process data is normally limited with missing values and it combines both categorical and continuous variables (mixed dataset).However, technological advancements manufacturing processes provide opportunities to further refine process inputs in order to minimize undesired variation in process outputs.A new linear regression based algorithm is proposed to achieve lower prediction error in comparison to the commonly used linear factor analysis for mixed data (FAMD) method.This algorithm is further coupled with a novel missing data algorithm to predict the process response values corresponding to a given set of values for process inputs.This enabled the novel imputation based predictive algorithm to quantify the effect of a confirmation trial based on the proposed changes in the operating ranges of one or more process inputs.A set of values for optimal process inputs is generated from operating ranges discovered by a recently proposed quality correlation algorithm (QCA) using a Bootstrap sampling method.The odds ratio, which represents a ratio between the probability of occurrence of desired and undesired process output values, is used to quantify the effect of a confirmation trial.The limitations of the underlying PCA based linear model have been discussed and the future research areas have been identified.
Active learning with effective scoring functions for semi-supervised temporal action localization

Temporal Action Localization (TAL) aims to predict both action category and temporal boundary of action instances in untrimmed videos, i.e., start and end time. Existing works usually adopt fully-supervised solutions, however, one of the practical bottlenecks in these solutions is the large amount of labeled training data required. To reduce expensive human label cost, this paper focuses on a rarely investigated yet practical task named semi-supervised TAL and proposes an effective active learning method, named AL-STAL. We leverage four steps for actively selecting video samples with high informativeness and training the localization model, named Train, Query, Annotate, Append. Two scoring functions that consider the uncertainty of localization model are equipped in AL-STAL, thus facilitating the video sample ranking and selection. One takes entropy of predicted label distribution as measure of uncertainty, named Temporal Proposal Entropy (TPE). And the other introduces a new metric based on mutual information between adjacent action proposals, named Temporal Context Inconsistency (TCI). To validate the effectiveness of proposed method, we conduct extensive experiments on three benchmark datasets THUMOS’14, ActivityNet 1.3 and ActivityNet 1.2. Experiment results show that AL-STAL outperforms the existing competitors and achieves satisfying performance compared with fully-supervised learning.
Illocutionary context and management allocation of emoji and other graphicons in Mexican parent school WhatsApp communities

The purpose of this study is to analyze the interactional achievement of emoji and other graphicons in WhatsApp communities created by and for school parents. Using a conversation analytic approach, 4824 messages written by 161 Mexican Spanish users, members of six different WhatsApp communities were examined in this study. The results show that task-oriented speech acts were the most common in this community of practice. Speech acts such as requests and responses to requests were mainly accompanied by graphicons. It was also observed that these speech acts formed adjacency pairs in which requests most often formed the first part of the sequence. It was also found that depending on the communicative purpose of the message and the type of speech act used, the graphicons functioned as either upgraders or mitigators. Graphicons also have the function of managing WhatsApp conversations by signalling conversational turns. Another finding revealed that standalone graphicons formed part of an adjacency system, replacing the verbal part of the message. Drawing on Locher and Watts’ (2005) relational work model, the analysis revealed that the interactional use of graphicons was influenced by the politeness rules of this community of practice.
Power and thermal-aware virtual machine scheduling optimization in cloud data center

Virtual machine (VM) consolidation technology is a commonly used energy-saving method in data centers. Most of the existing methods are committed to consolidating VMs to a small number of servers to improve the utilization of server resources and reduce the total energy consumption while preventing hotspots. However, energy and thermal-aware VM scheduling is a multi-objective optimization problem. Most of the existing related work cannot provide adequate means to adjust the impact of energy consumption and temperature on VM scheduling. Hence, this paper proposes a power and thermal-aware VM dynamic scheduling scheme (PTDS) for cloud data centers. The proposed PTDS dynamically adjusts the VM consolidation scheme by real-time detecting the server temperature and resource utilization rate under the premise of considering the thermal cycle effect of the data center computer room. Power and Thermal Objective Ant Colony Optimization (PTOACO) is proposed in the VM placement. PTOACO improves the defect that the ant colony algorithm easily falls into local optimization and adds control parameters to adjust the bias between sub-objectives. We performed extensive experiments by using real PlanetLab and random workloads. The performance results were compared with several advanced schemes regarding total energy consumption, hotspots, SLA violation rate, etc. The experimental results demonstrate that PTDS reduces energy consumption by 26.69% on average compared with other advanced schemes and ensures a meager SLA violation rate while avoiding hotspots.
Synthetic wavelength scanning interferometry for 3D surface profilometry with extended range of height measurement using multi-colour LED light sources

AbstractWe report three-dimensional surface profilometry with extended range of height measurements using synthetic wavelength scanning interferometry without tunable filters, wavelength-tuning lasers, grating elements. We have used inexpensive multiple colour light emitting diodes (LEDs) and operate them sequentially one by one or combination of two or more colours simultaneously to visualize synthetic wavelength light source. Multiple colour LED light source was synthesized and entire visible range from violet to deep red colour was covered. A wide range of synthetic wavelengths were obtained. Five step phase shifting algorithm was used to recover phase maps with Mirau type interferometer which can be further utilized for 3D height measurements. A simple phase subtraction method was used to reconstruct the phase map at synthetic wavelength. The present system provides extended range of height measurements from sub-wavelength to tens of wavelengths. Experimental results of 3D-surface profile measurements of Si-IC chip and standard step object are presented.KEYWORDS: Synthetic wavelengthmulti-colour LEDextended rangestep-height measurement Disclosure statementNo potential conflict of interest was reported by the author(s).Additional informationFundingThis work was supported by University Grants Commission.
TGVx: Dynamic Personalized POI Deep Recommendation Model

Personalized points-of-interest (POI) recommendation is very important for improving the service quality of location-based social network applications. It has become one of the most popular research directions in the industry and academia. However, the realization of high-quality personalized POI recommendation faces three major challenges: (i) the interest drift issue caused by the spatiotemporal dynamics of user check-in behavior, (ii) how to integrate as much heterogeneous information as possible to alleviate data sparseness and cold start issues, and (iii) how to use implicit feedback to model complex high-order nonlinear user-POI interactions. To jointly address all these challenges, we propose the TGVx recommendation framework and establish a dynamic personalized POI deep recommendation model, where T and G respectively represent time and geographic factors, V represents out-of-town visitors, and x represents time slot number. TGVx is composed of x parallel TGV models where the TG module mines high-order nonlinear user-POI interaction relationships and integrates multisource heterogeneous information, and the V module transfers the check-in records of out-of-town visitors in hometowns and generates pseudo check-in records in the target city. Technically, we design a new unsupervised deep learning network T-SemiDAE for the TG module. We built a POI-word heterogeneous network for the V module and used graph embedding technology to match the most similar POIs across cities and transfer check-in records. The experimental results on the actual datasets show that the TGVx model is always better than other advanced models in terms of accuracy and diversity for local and out-of-town recommendation scenarios. Compared with the best baseline model semi-deep auto-encoder with a conditional layer the average improvement rates of accuracy and diversity of TGVx are 17.1% to 58.6% and 2.25% to 28.86%, respectively. In theory, our research effectively uses data science and analysis methods to design a recommender system. In practice, our research is motivated by practical problems, and the research results have high practical promotion value. History: Accepted by Ram Ramesh, Area Editor for Data Science &amp; Machine Learning. Funding: This work was supported by the National Natural Science Foundation of China [Grants 71831003, 72293563, and 72172025], the Dalian Science and Technology Talent Innovation Support Policy Project [The High-level Talent Innovation Team Project 2022RG17], and the Liaoning Provincial Department of Education [Basic Scientific Research Project LJKMZ20221582]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2023.1286 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2021.0010 ) at ( http://dx.doi.org/10.5281/zenodo.7407123 ).
The relationship between cybersickness and eye-activity in response to varying speed, scene complexity and stereoscopic VR parameters

Eye trackers are non-invasive devices that can be integrated into VR head-mounted displays and the data they seamlessly provide can be instrumental in mitigating cybersickness. However, the connection of eye-activity to cybersickness has not been studied in a broad sense, where the effects of different VR content factors causing cybersickness are examined together. Addressing this gap, we present an extensive investigation of the relationship between eye-activity and cybersickness in response to three major cybersickness factors – navigation speed, scene complexity and stereoscopic rendering – simulated in varied severity. Our findings reveal multiple links between several eye-activity features and user-reported discomfort reports, the most significant of which are associated with speed levels, highlighting the relationship between feeling of vection and eye-activity. The evaluation also established significant differences in eye-activity response with different stimulus types and time spent in VR, suggesting an accumulation effect. Furthermore, the regression analysis hints that blink frequency can be utilized as a significant predictor of cybersickness, regardless of time spent in VR.
Holographic Representation Tools and Technologies for New Learning Actions: DhoMus Project Applied to Pitigliano and Vetulonia Museums, Tuscany

If on the one hand the museum reality, which has its origins in the history of collecting, turns its attention to the collection and custody of objects, on the other hand the exhibited objects necessarily need a way to be told. Museography is part of this dual relationship between the museum and the object. It’s a discipline that researches the different technologies of representation, finding tools to tell both the history of objects and the history of what binds them to their context and to the museum in which they are kept. To realize this story, it’s necessary a phase of study of the objects, carried out by the experts of the various disciplinary sectors, bringing to light all the information. To communicate these data, a specific narrative is designed to promote not only scientific dissemination, preserving the scientific nature of the content, but also cultural dissemination, through storytelling aimed at learning and understanding the object. The technologies currently present in the field of visual simulation provide visual and dynamic tools based on the concept of storytelling. In this paper, we propose the technology of the hologram: a representation that allows us to interface ourselves with the object by observing the history that the museum offers from the outside. Holography and the technologies that implement it today prove to be effective in creating the efficient narrative to which the museum itinerary tends. For this reason, the hologram appears to be a tool of great potential also in the field of teaching and the dissemination of knowledge. In the dialogue between museology and museography, through technological evolution, the museum must look at this type of applications and the case study that we present within DhoMus Project proves to be an interesting real example of experimentation.
The Invariant Ground Truth of Affect

Affective computing strives to unveil the unknown relationship between affect elicitation, manifestation of affect and affect annotations. The ground truth of affect, however, is predominately attributed to the affect labels which inadvertently include biases inherent to the subjective nature of emotion and its labeling. The response to such limitations is usually augmenting the dataset with more annotations per data point; however, this is not possible when we are interested in self-reports via first-person annotation. Moreover, outlier detection methods based on inter-annotator agreement only consider the annotations themselves and ignore the context and the corresponding affect manifestation. This paper reframes the ways one may obtain a reliable ground truth of affect by transferring aspects of causation theory to affective computing. In particular, we assume that the ground truth of affect can be found in the causal relationships between elicitation, manifestation and annotation that remain invariant across tasks and participants. To test our assumption we employ causation inspired methods for detecting outliers in affective corpora and building affect models that are robust across participants and tasks. We validate our methodology within the domain of digital games, with experimental results showing that it can successfully detect outliers and boost the accuracy of affect models. To the best of our knowledge, this study presents the first attempt to integrate causation tools in affective computing, making a crucial and decisive step towards general affect modeling.
Should robots indicate the trustworthiness of information from knowledge graphs?

The demo presents recent work on social robots that provide information from knowledge graphs in online graph databases. Sometimes more cooperative responses can be generated by using taxonomies and other semantic metadata that has been added to the knowledge graphs. Sometimes metadata about data provenance suggests higher or lower trustworthiness of the data. This raises the question whether robots should indicate trustworthiness when providing the information, and whether this should be done explicitly by meta-level comments or implicitly for example by modulating the robots' tone of voice and generating positive and negative affect in the robots' facial expressions.