Reviewerly/exHarmony huggingface dataset:
Hierarchical Graph Transformer-Based Deep Learning Model for Large-Scale Multi-Label Text Classification

Traditional methods of multi-label text classification, particularly deep learning, have achieved remarkable results. However, most of these methods use word2vec technology to represent sequential text information, while ignoring the logic and internal hierarchy of the text itself. Although these approaches can learn the hypothetical hierarchy and logic of the text, it is unexplained. In addition, the traditional approach treats labels as independent individuals and ignores the relationships between them, which not only does not reflect reality but also causes significant loss of semantic information. In this paper, we propose a novel Hierarchical Graph Transformer based deep learning model for large-scale multi-label text classification. We first model the text into a graph structure that can embody the different semantics of the text and the connections between them. We then use a multi-layer transformer structure with a multi-head attention mechanism at the word, sentence, and graph levels to fully capture the features of the text and observe the importance of the separate parts. Finally, we use the hierarchical relationship of the labels to generate the representation of the labels, and design a weighted loss function based on the semantic distances of the labels. Extensive experiments conducted on three benchmark datasets demonstrated that the proposed model can realistically capture the hierarchy and logic of text and improve performance compared with the state-of-the-art methods.
Long Range Acoustic and Deep Features Perspective on ASVspoof 2019

To secure automatic speaker verification (ASV) systems from intruders, robust countermeasures for spoofing attack detection are required. The ASVspoof series of challenge provides a shared anti-spoofing task. The recent edition, ASVspoof 2019, focuses on attacks by both synthetic and replay speech that are referred to as logical and physical access attacks, respectively. In the ASVspoof 2019 submission, we considered novel countermeasures based on long range acoustic features, that are unique in many ways as they are derived using octave power spectrum and subbands, as opposed to the commonly used linear power spectrum. During the post-challenge study, we further investigate the use of deep features that enhances the discriminative ability between genuine and spoofed speech. In this paper, we summarize the findings from the perspective of long range acoustic and deep features for spoof detection. We make a comprehensive analysis on the nature of different kinds of spoofing attacks and system development.
Visualising the expansion and spread of coronavirus disease 2019 by cartograms

Coronavirus disease 2019 (COVID-19) has emerged as a growing focus of global attention and a critical factor in public-health decision making. Towards fighting the COVID-19 outbreak, countries worldwide and international organisations have taken various actions, including promoting the transparency of and public access to disease data. In such public communications, maps have played an important role in that a map is worth a thousand words. Most of these have taken the form of a choropleth map. Here, we propose employing cartograms to visualise both the expansion and spread of COVID-19. We designed a combination of six circular cartograms containing the data of confirmed cases every 48 hours from 24 January to 3 February 2020. Such a design conveys both spatial and temporal information more intuitively and efficiently, so it can be expected to facilitate better public participation in the fight against COVID-19.
Continuous Hospitality with Social Robots at a hotel

Social robots are being increasingly employed in service encounters at hotels. This study explored the possibility that social robots can engage in heartwarming interactions with hotel customers. A collaboration design known as ‘Continuous Hospitality with Social Robots’, in which social robots compensate for gaps in hospitality through heartwarming interaction, was evaluated. A field test was conducted in which social robots engaged in heartwarming interaction with customers in a public area of a hotel and then collected customers’ impressions of the social robots and overall service via a questionnaire and an interview. The results demonstrate social robots’ potential for engaging in heartwarming interactions that enhance overall customer satisfaction through the use of the ‘Continuous Hospitality with Social Robots’ collaboration design. An exploratory analysis suggests that the perceived impressions of the interaction with social robots are influenced by customer gender and the duration of interactions. Furthermore, the results suggest that social robots could be utilized in other roles at hotels, namely effective advertisement through heartwarming interaction and mental support for employees who do not interact with customers.
Adding Culture to UX

For more than a decade, User eXperience (UX) has grown into a core concept of Human--Computer Interaction (HCI) and has been widely disseminated and accepted in the HCI community. At the same time, Cultural Heritage (CH) has been a favored domain for UX research, while the visitor is able to exploit CH material before, during, and after the visit, having different goals and requirements in each phase. Thus, CH organisations need to carefully explore the potential of investing in UX, as visitor satisfaction and intention to return and spread positive word-of-mouth is closely linked to the destination's overall success. A multitude of methods for UX analysis exist, but a clear overview of the current state of the available UX research methods in CH is missing. Recent studies [Othman 2012; Roussou 2018] have successfully developed frameworks to measure UX in cultural spaces, but most of them still lack understanding about how visitors interact with new technologies and simultaneously with the exhibits. In this article, a survey that describes the conceptual frameworks, models, research methodologies, and paradigms of incorporating Cultural User eXperience (CUX) in applications is reported, as well as the overall interaction between UX and CH and the related aspects that are influenced by the CH application domain and the parameters leading to optimization of the CUX. The outcome of this survey lays ground for understanding and defining the concept of UX research in CH, while current challenges and issues for future work are discussed.
User Representations in Human-Computer Interaction

Cursors, avatars, virtual hands or tools, and other rendered graphical objects, enable users to interact with computers such as PCs, game consoles or virtual reality systems. We analyze the role of these various objects from a user perspective under the unifying concept of “User Representations”. These representations are virtual objects that artificially extend the users’ physical bodies, enabling them to manipulate the virtual environment by performing motor actions that are continuously mapped to their User Representations. In this paper, we identify a set of concepts that are relevant for different User Representations, and provide a multidisciplinary review of the multisensory and cognitive factors underlying the control and subjective experience of User Representations. These concepts include visual appearance, multimodal feedback, sense of agency, input methods, peripersonal space, visual perspective, and body ownership. We further suggest a research agenda for these concepts, which can lead the human-computer interaction community towards a wider perspective of how users perceive and interact through their User Representations.
A Neural Network-Based On-device Learning Anomaly Detector for Edge Devices

Semi-supervised anomaly detection is an approach to identify anomalies by learning the distribution of normal data. Backpropagation neural networks (i.e., BP-NNs) based approaches have recently drawn attention because of their good generalization capability. In a typical situation, BP-NN-based models are iteratively optimized in server machines with input data gathered from the edge devices. However, (1) the iterative optimization often requires significant efforts to follow changes in the distribution of normal data (i.e., concept drift), and (2) data transfers between edge and server impose additional latency and energy consumption. To address these issues, we propose ONLAD and its IP core, named ONLAD Core. ONLAD is highly optimized to perform fast sequential learning to follow concept drift in less than one millisecond. ONLAD Core realizes on-device learning for edge devices at low power consumption, which realizes standalone execution where data transfers between edge and server are not required. Experiments show that ONLAD has favorable anomaly detection capability in an environment that simulates concept drift. Evaluations of ONLAD Core confirm that the training latency is 1.95x~6.58x faster than the other software implementations. Also, the runtime power consumption of ONLAD Core implemented on PYNQ-Z1 board, a small FPGA/CPU SoC platform, is 5.0x~25.4x lower than them.
A dynamic adaptive firefly algorithm with globally orientation

This paper proposes a dynamic adaptive firefly algorithm to overcome the disadvantages of the standard firefly algorithm, to improve the convergence rate and solution precision, and to avoid the premature algorithm trapping at the local extreme. It has a global-oriented moving mechanism and can dynamically adjust the step size and attractiveness. First, through the adaptive deviation degree strategy of optimal distance combining with the Gaussian distribution, it optimizes the fixed step-factor to balance the exploration and excavation capabilities of the algorithm and improves the diversity of the population. Second, minimum attractiveness is introduced to the algorithm, and is adaptively changed with the number of iterations, which can avoid random walk due to lack of traction between fireflies. Finally, this paper improves the mobility mechanism based on the position of the current optimal firefly. It enables firefly move with global orientation and also expands the sharing of information between fireflies to improve the overall evolutionary optimization performance of the algorithm. Theoretical analysis proves the convergence and time complexity of the improved algorithm. The simulation results of several test functions and engineering constraint optimization problems show that the improved algorithm has better solution performance, and clearly improves the convergence speed and solution accuracy.
Unsupervised and Semi-Supervised Domain Adaptation for Action Recognition from Drones

We address the problem of human action classification in drone videos. Due to the high cost of capturing and labeling large-scale drone videos with diverse actions, we present unsupervised and semi-supervised domain adaptation approaches that leverage both the existing fully annotated action recognition datasets and unannotated (or only a few annotated) videos from drones. To study the emerging problem of drone-based action recognition, we create a new dataset, NEC-DRONE, containing 5,250 videos to evaluate the task. We tackle both problem settings with 1) same and 2) different action label sets for the source (e.g., Kinectics dataset) and target domains (drone videos). We present a combination of video and instance-based adaptation methods, paired with either a classifier or an embedding-based framework to transfer the knowledge from source to target. Our results show that the proposed adaptation approach substantially improves the performance on these challenging and practical tasks. We further demonstrate the applicability of our method for learning cross-view action recognition on the Charades-Ego dataset. We provide qualitative analysis to understand the behaviors of our approaches.
Weakly Supervised Temporal Action Localization Using Deep Metric Learning

Temporal action localization is an important step towards video understanding. Most current action localization methods depend on untrimmed videos with full temporal annotations of action instances. However, it is expensive and time-consuming to annotate both action labels and temporal boundaries of videos. To this end, we propose a weakly supervised temporal action localization method that only requires video-level action instances as supervision during training. We propose a classification module to generate action labels for each segment in the video, and a deep metric learning module to learn the similarity between different action instances. We jointly optimize a balanced binary cross-entropy loss and a metric loss using a standard backpropagation algorithm. Extensive experiments demonstrate the effectiveness of both of these components in temporal localization. We evaluate our algorithm on two challenging untrimmed video datasets: THUMOS14 and ActivityNet1.2. Our approach improves the current state-of-the-art result for THUMOS14 by 6.5% mAP at IoU threshold 0.5, and achieves competitive performance for ActivityNet1.2.
Self Organizing Federated Learning Over Wireless Networks: A Socially Aware Clustering Approach

The significant proliferation of the Internet of Things (IoT) devices generates an enormous amount of data. Availability of such a large amount of data offers opportunities for using machine learning to enable intelligence in numerous applications. However, centralized machine learning schemes are based on migrating the data from devices to a centralized location for training. Such migration of data from user devices to a centralized location suffers from significant privacy concerns. To cope with this privacy preservation challenge, federated learning is a viable solution which enables learning in a distributed manner without migrating the data from devices to a centralized location. In this paper, we propose a novel federated learning scheme that offers federated learning without using centralized cloud server. First, we present a clustering algorithm based on social awareness which is followed by cluster head selection. Second, we formulate an optimization problem to minimize global federated learning time. Due to the NP-hard nature of the formulated optimization problem, we propose a heuristic algorithm to optimize the global federated learning time. Finally, we present numerical results to validate our proposed algorithm.
Extractive Automatic Text Summarization Based on Lexical-Semantic Keywords

The automatic text summarization (ATS) task consists in automatically synthesizing a document to provide a condensed version of it.Creating a summary requires not only selecting the main topics of the sentences but also identifying the key relationships between these topics.Related works rank text units (mainly sentences) to select those that could form the summary.However, the resulting summaries may not include all the topics covered in the source text because important information may have been discarded.In addition, the semantic structure of documents has been barely explored in this field.Thus, this study proposes a new method for the ATS task that takes advantage of semantic information to improve keyword detection.This proposed method increases not only the coverage by clustering the sentences to identify the main topics in the source document but also the precision by detecting the keywords in the clusters.The experimental results of this work indicate that the proposed method outperformed previous methods with a standard collection.
On-the-Fly Data Loader and Utterance-Level Aggregation for Speaker and Language Recognition

In this article, our recent efforts on directly modeling utterance-level aggregation for speaker and language recognition is summarized. First, an on-the-fly data loader for efficient network training is proposed. The data loader acts as a bridge between the full-length utterances and the network. It generates mini-batch samples on the fly, which allows batch-wise variable-length training and online data augmentation. Second, the traditional dictionary learning and Baum-Welch statistical accumulation mechanisms are applied to the network structure, and a learnable dictionary encoding (LDE) layer is introduced. The former accumulates discriminative statistics from the variable-length input sequence and outputs a single fixed-dimensional utterance-level representation. Experiments were conducted on four different datasets, namely NIST LRE 2007, AP17-OLR, SITW, and NIST SRE 2016. Experimental results show the effectiveness of the proposed batch-wise variable-length training with online data augmentation and the LDE layer, which significantly outperforms the baseline methods.
DeepDance: Music-to-Dance Motion Choreography With Adversarial Learning

The creation of improvised dancing choreographies is an important research field of cross-modal analysis. A key point of this task is how to effectively create and correlate music and dance with a probabilistic one-to-many mapping, which is essential to create realistic dances of various genres. To address this issue, we propose a GAN-based cross-modal association framework, DeepDance, which correlates two different modalities (dance motion and music) together, aiming at creating the desired dance sequence in terms of the input music. Its generator is to predictively produce the dance movements best-fit to current music piece by learning from examples. In another hand, its discriminator acts as an external evaluation from the audience and judges the whole performance. The generated dance movements and the corresponding input music are considered to be well-matched if the discriminator cannot distinguish the generated movements from the training samples according to the estimated probability. By adding motion consistency constraints in our loss function, the proposed framework is able to create long realistic dance sequences. To alleviate the problem of expensive and inefficient data collection, we propose an effective approach to create a large-scale dataset, YouTube-Dance3D, from open data source. Extensive experiments on currently available music-dance datasets and our YouTube-Dance3D dataset demonstrate that our approach effectively captures the correlation between music and dance and can be used to choreograph appropriate dance sequences.
Melanoma Cancer Classification Using ResNet with Data Augmentation

Melanoma skin cancer is cancer that difficult to detect. In this study, have been done melanoma cancer classification using Convolutional Neural Network (CNN). CNN is a class of Deep Neural Network (Deep Learning) and commonly used to analyzing images data. A lot of data used on CNN can greatly affect accuracy. In this study, the objective is to get best ResNet model for classifying melanoma cancer and normal skin images. The dataset that used is ISIC 2018. ResNet is used because the model winning the ILSVRC competition at 2015. ResNet architecture model that used are ResNet 50, 40, 25, 10 and 7 models. The architecture trained using data train that has been augmented and undersampling. The validation result on each model calculated using F1 Score. After validation and F1 Score result from the model obtained, the result compared each other to select the best model. The best architecture is ResNet 50 without augmentation that gives a validation accuracy of 0.83 and f1 score of 0.46.
An efficient K-means clustering algorithm for tall data

The analysis of continously larger datasets is a task of major importance in a wide variety of scientific fields. Therefore, the development of efficient and parallel algorithms to perform such an analysis is a a crucial topic in unsupervised learning. Cluster analysis algorithms are a key element of exploratory data analysis and, among them, the K-means algorithm stands out as the most popular approach due to its easiness in the implementation, straightforward parallelizability and relatively low computational cost. Unfortunately, the K-means algorithm also has some drawbacks that have been extensively studied, such as its high dependency on the initial conditions, as well as to the fact that it might not scale well on massive datasets. In this article, we propose a recursive and parallel approximation to the K-means algorithm that scales well on the number of instances of the problem, without affecting the quality of the approximation. In order to achieve this, instead of analyzing the entire dataset, we work on small weighted sets of representative points that are distributed in such a way that more importance is given to those regions where it is harder to determine the correct cluster assignment of the original instances. In addition to different theoretical properties, which explain the reasoning behind the algorithm, experimental results indicate that our method outperforms the state-of-the-art in terms of the trade-off between number of distance computations and the quality of the solution obtained.
Towards Practical Privacy-Preserving Decision Tree Training and Evaluation in the Cloud

Due to the capacity of storing massive data and providing huge computing resources, cloud computing has been a desirable platform for doing machine learning. However, the issue of data privacy is far from being well solved and thus has been a general concern in the cloud-aided machine learning. In this work, we investigate the study of how to efficiently do decision tree training and evaluation in the cloud and meanwhile achieve privacy preservation. Unlike existing cloud server-assisted model training approaches, in our proposed solution, the whole training process is mostly done by the cloud service provider who owns the machine learning model. Since the cloud cannot directly divide the encrypted dataset according to the best attributes selected, we propose a new method for decision tree training without dataset splitting. Precisely, we design three methods for decision tree training with the different tradeoff between privacy and efficiency. In all of these methods, the outsourced data are not revealed to the cloud service provider. We also propose a privacy-preserving decision tree evaluation scheme where the cloud service provider learns nothing about the user's input and the classification result while the trained model is kept secret to the user who could only learn the classification result. Compared with previous decision tree evaluation work, our scheme achieves desirable privacy preservation against both the user and the cloud service provider, and also minimizes the user's computation and communication costs. Moreover, besides protecting the data confidentiality, our proposed scheme also supports off-line users and thus has good scalability. The real-world dataset-based experimental results demonstrate that our system is of desirable utility and efficiency.