Reviewerly/exHarmony huggingface dataset:
FedACA: An Adaptive Communication-Efficient Asynchronous Framework for Federated Learning

Federated Learning (FL) is a type of distributed machine learning, which avoids sharing privacy and sensitive data with a central server. Despite the advances in FL, current approaches cannot provide satisfactory performance when dealing with heterogeneity in data and unpredictability of system devices. First, straggler devices can adversely impact convergence speed of the global model training. Second, for model aggregation in traditional FL, edge devices communicate frequently with a central server using their local updates. However, this process may encounter communication bottleneck caused by substantial bandwidth usage. To address these challenges, this paper presents an adaptive, communication-efficient and asynchronous FL technique called FedACA comprising feedback loops at two levels. Our approach contains a self-adjusting local training step with active participant selection to accelerate the convergence of the global model. To reduce the communication overhead, FedACA supports an adaptive uploading policy at the edge devices, which leverages the model similarity and L2-norm differences between the current and previous local gradient. It also utilizes contrastive learning to tackle data heterogeneity by regularizing the local training if the local model has deviated from the global model and helps with the model similarity measurement in the uploading policy. Extensive experiments on a benchmark comprising three image datasets with non-independent and identically distributed (non-i.i.d) data show that FedACA adapts well to the straggler effect in asynchronous environments and also provides significant reductions in communication costs compared to other state-of-the-art FL algorithms.
FCH-TTS: Fast, Controllable and High-quality Non-Autoregressive Text-to-Speech Synthesis

Inspired by the success of the non-autoregressive speech synthesis model FastSpeech, we propose FCH-TTS, a fast, controllable and universal neural text-to-speech (TTS) capable of generating high-quality spectrograms. The basic architecture of FCH-TTS is similar to that of FastSpeech, but FCH-TTS uses a simple yet effective attention-based soft alignment mechanism to replace the complex teacher model in FastSpeech, allowing the model to be better adapted to different languages. Specifically, in addition to the control of voice speed and prosody, a fusion module has been designed to better model speaker features in order to obtain the desired timbre. Meanwhile, several special loss functions were applied to ensure the quality of the output mel-spectrogram. Experimental results on the dataset LJSpeech show that FCH-TTS achieves the fastest inference speed compared to all baseline models, while also achieving the best speech quality. In addition, the controllability of the model with respect to prosody, voice speed and timbre was validated on several datasets, and the good performance on the low-resource Tibetan dataset demonstrates the universality of the model.
Generating Diverse Gestures from Speech Using Memory Networks as Dynamic Dictionaries

People naturally enhance their speeches with body motion or gestures. Generating human gestures for digital humans or virtual avatars from speech audio or text remains challenging for its indeterministic nature. We observe that existing neural methods often give gestures with an inadequate amount of movement shift, which can be characterized as slow or dull. Thus, we propose a novel generative model coupled with memory networks to work as dynamic dictionaries for generating gestures with improved diversity. Under the hood of the proposed model, a dictionary network dynamically stores previously appeared pose features corresponding to text features for the generator to lookup, while a pose generation network takes in audio and pose features and outputs the resulting gesture sequences. Seed poses are utilized in the generation process to guarantee the continuity between two speech segments. We also propose a new objective evaluation metric for diversity of generated gestures and succeed in demonstrating that the proposed model has the ability to generate gestures with improved diversity.
A random forest algorithm-based emotion recognition model for eye features

Objective: To develop a random forest algorithm-based model for the recognition of angry, neutral, and happy emotions for eye features and to further analyze the importance of eye features. Method: Raw data were obtained using emotional images from the Chinese Emotional Face System (CAFPS), and the code was used to derive relevant eye features data to build the database. The relevant features were left and right pupil size, left and right visible iris size, Distance between inner corners of eyes, upper and lower eyelid distance, left eye opening and closing, AU1 (inner eyebrow raised), AU2 (outer eyebrow raised), AU4 (overall lowered eyebrow), AU5 (raised upper eyelid), AU6 (raised cheek) and AU7 (eye constriction), a total of 13 eye features, were used to construct an emotion recognition model using the random forest algorithm and to analyze the importance of the features. Results: The differences were statistically significant (p<0.01) in all 13 eye features; the accuracy of the model constructed using the random forest algorithm was 70.2%, the recall was 0.702, the accuracy was 0.977 and the F1 was 0.809. AU6 had the highest importance in the process of constructing the model, accounting for 15.4%. Conclusion: Eye features have a role in the process of building an emotion recognition model, validating the theories related to Chinese medicine eye diagnosis, and combining Chinese medicine eye diagnosis with theories related to Chinese medicine emotions to identify patients' emotions by capturing eye information, which has clinical practice implications.
Towards an Understanding of How Players Make Meaning from Post-Play Process Visualizations

Player-facing, retrospective gameplay visualizations help players track progress and learn from others. Visualizations of a user’s step by step process, may be able to advance retrospective visualization. However, we currently do not know how players make meaning from process visualizations of game data. In this work, we take a first step towards addressing this gap by examining how players make meaning from process visualizations of other players’ gameplay. We identify two interpretation methods comprised of six techniques and discuss what these results mean for future use of player-facing process visualizations.
Interpretable Latent Space to Enable Counterfactual Explanations

Many dimensionality reduction methods have been introduced to map a data space into one with fewer features and enhance machine learning models’ capabilities. This reduced space, called latent space, holds properties that allow researchers to understand the data better and produce better models. This work proposes an interpretable latent space that preserves the similarity of data points and supports a new way of learning a classification model that allows prediction and explanation through counterfactual examples. We demonstrate with extensive experiments the effectiveness of the latent space with respect to different metrics in comparison with several competitors, as well as the quality of the achieved counterfactual explanations.
Comparative analysis of machine learning algorithms for Classification of US Airlines Tweets

Sentiment classification (SC) task is well favoured in e-commerce field as the reviews posted by users, classified into positive and negative categories, help the business holders to understand their current market and other's market strategy and settings as well. This task of SC is widely encouraged by various researchers for different domains such as movie reviews, political reviews, product reviews, airlines and so on. This paper aims to classify the tweets that are shared by the American flight travellers into positive and negative class so that the other people can utilize this information to take their journey related decisions. The task has been accomplished using three machine learning classifiers to classify the tweets data that is collected from Kaggle site. It is seen from the experimental results that SVM outperforms the LR and MNB baselines by 0.9% and 6.23% accuracy respectively.
Blind Recognition of LDPC Codes by Average LLR in Multi-Channels Based on CNN

Coding and decoding play an essential role in communication systems and are the primary conditions to ensure the standard transmission of signals, which can increase stability and reliability after coding. Low Density Parity Check Code (LDPC) coding is a common form of coding, and its parallel iterative decoding algorithm based on a sparse matrix is less complex and more flexible than the Turbo algorithm. However, with the increasing number of protocols and channels, the traditional methods of LDPC codes recognition are not adaptable enough, a more flexible and applicable method is essential. In this paper, we propose a deep learning-based LDPC coding recognition method, which is more adaptable, has higher estimation accuracy than traditional algorithms and has a more straightforward data processing structure compared to 2-dimensional convolutional neural networks and fully connected networks. After training the data of the Additive White Gaussian Noise (AWGN) channel, Rayleigh channel and Rician channel, the network's performance is better in the test, and the estimation accuracy is better than the traditional algorithm.
Show, Tell and Rephrase: Diverse Video Captioning via Two-Stage Progressive Training

Describing a video using natural language is an inherently one-to-many translation task. To generate diverse captions, existing VAE-based generative models typically learn factorized latent codes via one-stage training merely from stand-alone video-caption pairs. However, such a paradigm neglects set-level relationships among captions from the same video, not fully capturing the underlying multimodality of the generative process. To overcome this shortcoming, we leverage neighbouring descriptions for the same video that are articulated with noticeable topics and language variations (i.e., paraphrases). To this end, we propose a novel progressive training method by decomposing the learning of latent variables into two stages that are topic-oriented and paraphrase-oriented, respectively. Specifically, the model learns from divergent topic sentences obtained by semantic-based clustering in the first stage. It is then trained again through paraphrases with a cluster-aware adaptive regularization, allowing more intra-cluster variations. Furthermore, we introduce an overall metric DAUM, a <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D</b> iversity- <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</b> ccuracy <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">U</b> nified <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</b> etric to consider both the precision of the generated caption set and its coverage on the reference set, which has proved to have a higher correlation with human judgment than previous precision-only metrics. Extensive experiments on three large-scale video datasets show that the proposed training strategy can achieve superior performance in terms of accuracy, diversity, and DAUM over several baselines.
ABDPool: Attention-based Differentiable Pooling

Graph Neural Networks (GNNs) have achieved state-of-the-art performance on a wide range of graph-based tasks such as graph classification and node classification. This is because the unique structure of GNNs allows them to effectively learn embeddings for unstructured data. One important operation for graph classification tasks is downsampling or pooling, which obtains graph representations from node representations. However, most GNNs are associated with global pooling, that can not learn hierarchical graph representations. Meanwhile, current hierarchical pooling methods have the shortcomings of unclear node assignment and uniform aggregation. To overcome these drawbacks, we propose an attention-based differentiable pooling operation in this paper, which can learn a hard cluster assignment for nodes and aggregate nodes in each cluster differently by introducing an attention mechanism. Experiments on standard graph classification benchmarks show that our proposed approach performs better when compared with other competing methods.
DAWN: Domain Generalization Based Network Alignment

Network alignment aims to discover nodes in different networks belonging to the same identity. In recent years, the network alignment problem has aroused significant attentions in both industry and academia. With the rapid growth of information, the sizes of networks are usually very large and in most cases we only focus on the alignment of partial networks. However, under this circumstances, the collected network data may be highly biased, and the training and testing data are no longer i.i.d. (identically and independently distributed). Thus, it is difficult for the trained alignment model to have a good performance in the test set. To bridge this gap, in this paper, we propose a novel <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D</u> omain gener <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</u> lization based net <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">W</u> ork alig <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</u> ment approach termed as DAWN. Specifically, in DAWN, we first design a novel invariant feature extraction model which leverages adversarial learning to extract domain-invariant features. Then, we design a novel invariant network alignment model which can achieve global optimum and local optimum simultaneously to learn domain-invariant alignment patterns. Finally, we conduct extensive experiments on the benchmark dataset of Facebook-Twitter, and results show that DAWN can averagely achieve 14.01% higher Hits@k and 10.63% higher MRR@k compared with the state-of-the-art methods.
Complex Table Structure Recognition in the Wild Using Transformer and Identity Matrix-Based Augmentation

Tables are a widely used and efficient data structure. Although people can intuitively understand table contents, it remains challenging for machines, especially the tables taken in the wild. Previous methods mainly focus on scanned or PDF tables, but ignore investigating camera-based tables. This paper treats table structure recognition (TSR) as an image-to-sequence recognition task and adopts an end-to-end trainable model for complex TSR in the wild. Specifically, the model consists of a CNN-based encoder and two Transformer-based decoding branches, which can simultaneously predict the logical and physical structures of a table. Currently available camera-based table datasets are scarce, but deep learning methods heavily rely on large-scale datasets. To alleviate data insufficiency and boost model's performance, we propose a new and effective table data augmentation method, called TabSplitter. Due to the complex structure caused by cells spanning multiple rows or columns, directly cropping will lead to damage and change the properties of these cells. To solve this problem, we proposed a matrix representation, named Identity Matrix (IM), to describe the table structure. Based on IM, we crop the tables and correct the cells whose attributes have changed, thus enhancing data diversity. Furthermore, the proposed IM facilitates the pre-processing of data and post-processing of predictions. Experimental results on several datasets demonstrate the effectiveness of the model and the TabSplitter for TSR, especially for complex tables in the wild.
Smaller is Better: An Analysis of Instance Quantity/Quality Trade-off in Rehearsal-based Continual Learning

The design of machines and algorithms capable of learning in a dynamically changing environment has become an increasingly topical problem with the increase of the size and heterogeneity of data available to learning systems. As a consequence, the key issue of Continual Learning has become that of addressing the stability-plasticity dilemma of connectionist systems, as they need to adapt their model without forgetting previously acquired knowledge. Within this context, rehearsal-based methods i.e., solutions in where the learner exploits memory to revisit past data, has proven to be very effective, leading to performance at state-of-the-art. In our study, we propose an analysis of the memory quantity/quality trade-off adopting various data reduction approaches to increase the number of instances storable in memory. In particular, we investigate complex instance compression techniques such as deep encoders, but also trivial approaches such as image resizing and linear dimensionality reduction. Our findings suggest that the optimal trade-off is severely skewed toward instance quantity, where rehearsal approaches with several heavily compressed instances easily outperform state-of-the-art approaches with the same amount of memory at their disposal. Further, in high memory configurations, deep approaches extracting spatial structure combined with extreme resizing (of the order of 8 × 8 images) yield the best results, while in memory-constrained configurations where deep approaches cannot be used due to their memory requirement in training, a variation of Extreme Learning Machines (ELM) offer a clear advantage. Code and experiments available at https://github.com/francesco-p/smaller-is-better
Improved Transformer with Parallel Encoders for Image Captioning

Image captioning is currently one of the most important multimodal tasks. With Transformer proposed, many Transformer-based models have achieved good performance in image captioning. However, substantial work is still required to improve the performance in the field of image captioning. We propose an improved model that uses the meshed-memory Transformer as its backbone. We propose the use of region features and grid features together. In addition, we use two identical parallel encoders to process region features and grid features separately, and fuse the outputs of each layer of the two encoders to form one of the inputs of the decoder. We comprehensively compare the performance of our model with the existing state-of-the-art models on the official COCO dataset. Experiments show that, on the Karpathy test split, our model outperforms the backbone on all evaluation metrics: for example, it increases BLEU-1 from 80.8% to 81.4%, and CIDEr from 131.2% to 133.5%.
Machine Learning-Based Decision Making in Evolutionary Multiobjective Clustering

Clustering is an important optimization problem which is at the core of many data mining and machine learning applications. Evolutionary multiobjective clustering algorithms optimize multiple partition quality criteria simultaneously, producing a set of trade-off solutions in a single execution. How can a final solution be selected from this set of candidate trade-offs? Decision making is concerned with such a task, acknowledging that obtaining a single solution may represent the ultimate goal in practice. Given the limitations of current techniques, we explore a novel approach to decision making: tackling this task as a supervised learning problem. Our approach attempts to learn from the decision-making process in example (training) problems, with the aim of later applying the learned model to lead the identification of final solutions in new applications. In this paper, we present preliminary results on the evaluation of this proposal. Our experiments include comparisons with respect to a set of decision-making approaches from the literature and consider a collection of synthetic clustering problems with diverse characteristics. The results obtained are promising, highlighting the relevance of devising alternative decision-making strategies, the effectiveness of our proposal, and motivating further research on this matter.
Blockchain-Based Multi-keyword Search on Encrypted COVID-19 Contact Tracing Data

Contact tracing data contain highly sensitive information in which any form of leakage will significantly threaten the security and privacy of the individual. Encryption schemes can be used to ensure confidentiality; however, the usability of the retrieval system will be reduced considerably as the encrypted data does not support searchability. Hence, searchable encryption could be applied to contact tracing data. This paper introduces our implementation of a blockchain-based multi-keyword searchable encryption for COVID-19 contact tracing data. We used Advanced Encryption Standard as the symmetric primitive for encrypting datasets and Attribute-based Encryption as the asymmetric primitive for encrypting the search index. We incorporated Blockchain-based storage into our scheme to assure the integrity of the datasets and search index. It provides immutable storage and ensures data integrity and non-repudiation, as any form of access will be logged automatically. The results show that the scheme works efficiently without compromising any security goals. Compared to the existing searchable contact tracing schemes, it provides more features and maintains efficiency even if a large search index was used within the system.
Explainable Recommendation via Personalized Features on Dynamic Preference Interactions

An Explainable Recommendation system has become more prominent because it could convince the user to trust the prediction. However, the existing Explainable Recommendation systems face one of these two limitations. The first limitation is the utilization of a pre-defined feature set to represent the users/items’ preferences. By representing the preferences with a pre-defined feature set, the uniqueness of each user will be limited by the characteristics of the pre-defined set. For the other limitation, the user/item preference representation in existing models is constructed from the entire past interactions. This can mislead the model because the user’s preferences change over the interaction’s characteristics. Therefore, the entire past interactions might not be able to represent the user’s preferences for the target interaction. In this paper, we proposed an Explainable Recommendation system, which represents the user’s preferences with a personalized feature set and is able to adapt the target user/item preference representation to suit the target interaction’s characteristics. By representing the users/items’ preferences with a personalized feature set, the proposed method is able to give the score of importance in the rating contribution to each feature and uses these scores to assume the reasons for the user’s decision. To make the representation suitable to the target interaction’s characteristics, the proposed model utilizes only the target interaction’s information to represent the preferences of users/items. The proposed method also is not relying on the entire past interactions to avoid the bias of dominant features in the past interactions. For the experiment, the proposed method consistently outperforms all baselines on three benchmark datasets. We also analyze the explanation resulting from the proposed model to support our assumptions that the preferences of the user are unique and dynamically change over the interaction’s characteristics.