Reviewerly/exHarmony huggingface dataset:
Adversarial Directed Graph Embedding

Node representation learning for directed graphs is critically important to facilitate many graph mining tasks. To capture the directed edges between nodes, existing methods mostly learn two embedding vectors for each node, source vector and target vector. However, these methods learn the source and target vectors separately. For the node with very low indegree or outdegree, the corresponding target vector or source vector cannot be effectively learned. In this paper, we propose a novel Directed Graph embedding framework based on Generative Adversarial Network, called DGGAN. The main idea is to use adversarial mechanisms to deploy a discriminator and two generators that jointly learn each node’s source and target vectors. For a given node, the two generators are trained to generate its fake target and source neighbor nodes from the same underlying distribution, and the discriminator aims to distinguish whether a neighbor node is real or fake. The two generators are formulated into a unified framework and could mutually reinforce each other to learn more robust source and target vectors. Extensive experiments show that DGGAN consistently and significantly outperforms existing state-of-the-art methods across multiple graph mining tasks on directed graphs.
What Context Features Can Transformer Language Models Use?

Transformer-based language models benefit from conditioning on contexts of hundreds to thousands of previous tokens.What aspects of these contexts contribute to accurate model prediction?We describe a series of experiments that measure usable information by selectively ablating lexical and structural information in transformer language models trained on English Wikipedia.In both mid-and longrange contexts, we find that several extremely destructive context manipulations-including shuffling word order within sentences and deleting all words other than nouns-remove less than 15% of the usable information.Our results suggest that long contexts, but not their detailed syntactic and propositional content, are important for the low perplexity of current transformer language models. 1
Multilingual Speech Translation from Efficient Finetuning of Pretrained Models

Xian Li, Changhan Wang, Yun Tang, Chau Tran, Yuqing Tang, Juan Pino, Alexei Baevski, Alexis Conneau, Michael Auli. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.
Global Attention Decoder for Chinese Spelling Error Correction

Recent progress has been made in using BERT framework for Chinese spelling error correction (CSC).However, most existing methods correct words based on local contextual information, without considering the influence of error words in sentences.Imposing attention on error contextual information could mislead and decrease the overall performance of CSC.To address this issue, we propose a Global Attention Decoder (GAD) approach for CSC.Specifically, the proposed method learns the global relationship of the potential correct input characters and the candidates of potential error characters.Rich global contextual information is obtained to alleviate the impact of the local error contextual information.In addition, a BERT with Confusion set guided Replacement Strategy (BERT CRS) is designed to narrow the gap between BERT and CSC.The candidates generated by BERT CRS covering the correct character with more than 99.9% probability.To demonstrate the effectiveness of our proposed framework, we test our method on three human-annotated datasets.The experimental results show that our approach outperforms all competitor models by a large margin of up to 6.2%, achieving state-of-the-art methods on all datasets.
Unsupervised Domain Adaptation for Person Re-identification via Heterogeneous Graph Alignment

Unsupervised person re-identification (re-ID) is becoming increasingly popular due to its power in real-world systems such as public security and intelligent transportation systems. However, the person re-ID task is challenged by the problems of data distribution discrepancy across cameras and lack of label information. In this paper, we propose a coarse-to-fine heterogeneous graph alignment (HGA) method to find cross-camera person matches by characterizing the unlabeled data as a heterogeneous graph for each camera. In the coarse-alignment stage, we assign a projection for each camera and utilize an adversarial learning based method to align coarse-grained node groups from different cameras into a shared space, which consequently alleviates the distribution discrepancy between cameras. In the fine-alignment stage, we exploit potential fine-grained node groups in the shared space and introduce conservative alignment loss functions to constrain the graph aligning process, resulting in reliable pseudo labels as learning guidance. The proposed domain adaptation framework not only improves model generalization on target domain, but also facilitates mining and integrating the potential discriminative information across different cameras. Extensive experiments on benchmark datasets demonstrate that the proposed approach outperforms the state-of-the-arts.
Including Signed Languages in Natural Language Processing

Signed languages are the primary means of communication for many deaf and hard of hearing individuals.Since signed languages exhibit all the fundamental linguistic properties of natural language, we believe that tools and theories of Natural Language Processing (NLP) are crucial towards its modeling.However, existing research in Sign Language Processing (SLP) seldom attempt to explore and leverage the linguistic organization of signed languages.This position paper calls on the NLP community to include signed languages as a research area with high social and scientific impact.We first discuss the linguistic properties of signed languages to consider during their modeling.Then, we review the limitations of current SLP models and identify the open challenges to extend NLP to signed languages.Finally, we urge (1) the adoption of an efficient tokenization method; (2) the development of linguistically-informed models; (3) the collection of real-world signed language data; (4) the inclusion of local signed language communities as an active and leading voice in the direction of research.
SIRE: Separate Intra- and Inter-sentential Reasoning for Document-level Relation Extraction

Document-level relation extraction has attracted much attention in recent years.It is usually formulated as a classification problem that predicts relations for all entity pairs in the document.However, previous works indiscriminately represent intra-and inter-sentential relations in the same way, confounding the different patterns for predicting them.Besides, they create a document graph and use paths between entities on the graph as clues for logical reasoning.However, not all entity pairs can be connected with a path and have the correct logical reasoning paths in their graph.Thus many cases of logical reasoning cannot be covered.This paper proposes an effective architecture, SIRE, to represent intra-and inter-sentential relations in different ways.We design a new and straightforward form of logical reasoning module that can cover more logical reasoning chains.Experiments on the public datasets show SIRE outperforms the previous state-of-the-art methods.Further analysis shows that our predictions are reliable and explainable.
Quantum-classical hybrid algorithm for the simulation of all-electron correlation

While chemical systems containing hundreds to thousands of electrons remain beyond the reach of quantum devices, hybrid quantum-classical algorithms present a promising pathway toward a quantum advantage. Hybrid algorithms treat the exponentially scaling part of the calculation-the static correlation-on the quantum computer and the non-exponentially scaling part-the dynamic correlation-on the classical computer. While a variety of algorithms have been proposed, the dependence of many methods on the total wave function limits the development of easy-to-use classical post-processing implementations. Here, we present a novel combination of quantum and classical algorithms, which computes the all-electron energy of a strongly correlated molecular system on the classical computer from the 2-electron reduced density matrix (2-RDM) evaluated on the quantum device. Significantly, we circumvent the wave function in the all-electron calculations by using density matrix methods that only require input of the statically correlated 2-RDM. Although the algorithm is completely general, we test it with two classical density matrix methods, the anti-Hermitian contracted Schrödinger equation (ACSE) and multiconfiguration pair-density functional theories, using the recently developed quantum ACSE method for simulating the statically correlated 2-RDM. We obtain experimental accuracy for the relative energies of all three benzyne isomers and thereby demonstrate the ability of the developed algorithm to achieve chemically relevant and accurate results on noisy intermediate-scale quantum devices.
Systematic-Error-Tolerant Multiqubit Holonomic Entangling Gates

Quantum holonomic gates hold built-in resilience to local noises and provide a promising approach for implementing fault-tolerant quantum computation. We propose to realize high-fidelity holonomic $(N+1)$-qubit controlled gates using Rydberg atoms confined in optical arrays or superconducting circuits. We identify the scheme, deduce the effective multibody Hamiltonian, and determine the working condition of the multiqubit gate. Uniquely, the multiqubit gate is immune to systematic errors, i.e., laser parameter fluctuations and motional dephasing, as the $N$ control atoms largely remain in the very stable qubit space during the operation. We show that ${C}_{N}$-not gates can reach the same level of fidelity at a given gate time for $N\ensuremath{\le}5$ under a suitable choice of parameters, and the gate tolerance against errors in systematic parameters can be further enhanced through optimal pulse engineering. In the case of Rydberg atoms, the proposed protocol is intrinsically different from typical schemes based on Rydberg blockade or antiblockade. Our study paves an alternative way to build robust multiqubit gates with Rydberg atoms trapped in optical arrays or with superconducting circuits. It contributes to current efforts to develop scalable quantum computation with trapped atoms and fabricable superconducting devices.
CTFN: Hierarchical Learning for Multimodal Sentiment Analysis Using Coupled-Translation Fusion Network

Multimodal sentiment analysis is the challenging research area that attends to the fusion of multiple heterogeneous modalities.The main challenge is the occurrence of some missing modalities during the multimodal fusion procedure.However, the existing techniques require all modalities as input, thus are sensitive to missing modalities at predicting time.In this work, the coupled-translation fusion network (CTFN) is firstly proposed to model bi-direction interplay via couple learning, ensuring the robustness in respect to missing modalities.Specifically, the cyclic consistency constraint is presented to improve the translation performance, allowing us directly to discard decoder and only embraces encoder of Transformer.This could contribute to a much lighter model.Due to the couple learning, CTFN is able to conduct bi-direction crossmodality intercorrelation parallelly.Based on CTFN, a hierarchical architecture is further established to exploit multiple bi-direction translations, leading to double multimodal fusing embeddings compared with traditional translation methods.Moreover, the convolution block is utilized to further highlight explicit interactions among those translations.For evaluation, CTFN was verified on two multimodal benchmarks with extensive ablation studies.The experiments demonstrate that the proposed framework achieves state-of-the-art or often competitive performance.Additionally, CTFN still maintains robustness when considering missing modality.
Research on Sports Training Action Recognition Based on Deep Learning

With the rapid development of science and technology in today’s society, various industries are pursuing information digitization and intelligence, and pattern recognition and computer vision are also constantly carrying out technological innovation. Computer vision is to let computers, cameras, and other machines receive information like human beings, analyze and process their semantic information, and make coping strategies. As an important research direction in the field of computer vision, human motion recognition has new solutions with the gradual rise of deep learning. Human motion recognition technology has a high market value, and it has broad application prospects in the fields of intelligent monitoring, motion analysis, human-computer interaction, and medical monitoring. This paper mainly studies the recognition of sports training action based on deep learning algorithm. Experimental work has been carried out in order to show the validity of the proposed research.
The Curse of Dense Low-Dimensional Information Retrieval for Large Index Sizes

Information Retrieval using dense lowdimensional representations recently became popular and showed out-performance to traditional sparse-representations like BM25.However, no previous work investigated how dense representations perform with large index sizes.We show theoretically and empirically that the performance for dense representations decreases quicker than sparse representations for increasing index sizes.In extreme cases, this can even lead to a tipping point where at a certain index size sparse representations outperform dense representations.We show that this behavior is tightly connected to the number of dimensions of the representations: The lower the dimension, the higher the chance for false positives, i.e. returning irrelevant documents.
Benchmarking for biomedical natural language processing tasks with a domain specific ALBERT

The abundance of biomedical text data coupled with advances in natural language processing (NLP) is resulting in novel biomedical NLP (BioNLP) applications. These NLP applications, or tasks, are reliant on the availability of domain-specific language models (LMs) that are trained on a massive amount of data. Most of the existing domain-specific LMs adopted bidirectional encoder representations from transformers (BERT) architecture which has limitations, and their generalizability is unproven as there is an absence of baseline results among common BioNLP tasks.We present 8 variants of BioALBERT, a domain-specific adaptation of a lite bidirectional encoder representations from transformers (ALBERT), trained on biomedical (PubMed and PubMed Central) and clinical (MIMIC-III) corpora and fine-tuned for 6 different tasks across 20 benchmark datasets. Experiments show that a large variant of BioALBERT trained on PubMed outperforms the state-of-the-art on named-entity recognition (+ 11.09% BLURB score improvement), relation extraction (+ 0.80% BLURB score), sentence similarity (+ 1.05% BLURB score), document classification (+ 0.62% F1-score), and question answering (+ 2.83% BLURB score). It represents a new state-of-the-art in 5 out of 6 benchmark BioNLP tasks.The large variant of BioALBERT trained on PubMed achieved a higher BLURB score than previous state-of-the-art models on 5 of the 6 benchmark BioNLP tasks. Depending on the task, 5 different variants of BioALBERT outperformed previous state-of-the-art models on 17 of the 20 benchmark datasets, showing that our model is robust and generalizable in the common BioNLP tasks. We have made BioALBERT freely available which will help the BioNLP community avoid computational cost of training and establish a new set of baselines for future efforts across a broad range of BioNLP tasks.
Learning Spatial-Semantic Relationship for Facial Attribute Recognition with Limited Labeled Data

Recent advances in deep learning have demonstrated excellent results for Facial Attribute Recognition (FAR), typically trained with large-scale labeled data. However, in many real-world FAR applications, only limited labeled data are available, leading to remarkable deterioration in performance for most existing deep learning-based FAR methods. To address this problem, here we propose a method termed Spatial-Semantic Patch Learning (SSPL). The training of SSPL involves two stages. First, three auxiliary tasks, consisting of a Patch Rotation Task (PRT), a Patch Segmentation Task (PST), and a Patch Classification Task (PCT), are jointly developed to learn the spatial-semantic relationship from large-scale unlabeled facial data. We thus obtain a powerful pre-trained model. In particular, PRT exploits the spatial information of facial images in a self-supervised learning manner. PST and PCT respectively capture the pixel-level and image-level semantic information of facial images based on a facial parsing model. Second, the spatial-semantic knowledge learned from auxiliary tasks is transferred to the FAR task. By doing so, it enables that only a limited number of labeled data are required to fine-tune the pre-trained model. We achieve superior performance compared with state-of-the-art methods, as substantiated by extensive experiments and studies.
Solving Misclassification of the Credit Card Imbalance Problem Using Near Miss

In ordinary credit card datasets, there are far fewer fraudulent transactions than ordinary transactions. In dealing with the credit card imbalance problem, the ideal solution must have low bias and low variance. The paper aims to provide an in-depth experimental investigation of the effect of using a hybrid data-point approach to resolve the class misclassification problem in imbalanced credit card datasets. The goal of the research was to use a novel technique to manage unbalanced datasets to improve the effectiveness of machine learning algorithms in detecting fraud or anomalous patterns in huge volumes of financial transaction records where the class distribution was imbalanced. The paper proposed using random forest and a hybrid data-point approach combining feature selection with Near Miss-based undersampling technique. We assessed the proposed method on two imbalanced credit card datasets, namely, the European Credit Card dataset and the UCI Credit Card dataset. The experimental results were reported using performance matrices. We compared the classification results of logistic regression, support vector machine, decision tree, and random forest before and after using our approach. The findings showed that the proposed approach improved the predictive accuracy of the logistic regression, support vector machine, decision tree, and random forest algorithms in credit card datasets. Furthermore, we found that, out of the four algorithms, the random forest produced the best results.
AR in the Architecture Domain: State of the Art

Augmented reality (AR) allows the real and digital worlds to converge and overlap in a new way of observation and understanding. The architectural field can significantly benefit from AR applications, due to their systemic complexity in terms of knowledge and process management. Global interest and many research challenges are focused on this field, thanks to the conjunction of technological and algorithmic developments from one side, and the massive digitization of built data. A significant quantity of research in the AEC and educational fields describes this state of the art. Moreover, it is a very fragmented domain, in which specific advances or case studies are often described without considering the complexity of the whole development process. The article illustrates the entire AR pipeline development in architecture, from the conceptual phase to its application, highlighting each step’s specific aspects. This storytelling aims to provide a general overview to a non-expert, deepening the topic and stimulating a democratization process. The aware and extended use of AR in multiple areas of application can lead a new way forward for environmental understanding, bridging the gap between real and virtual space in an innovative perception of architecture.
GPCA: A Probabilistic Framework for Gaussian Process Embedded Channel Attention

Channel attention mechanisms have been commonly applied in many visual tasks for effective performance improvement. It is able to reinforce the informative channels as well as to suppress the useless channels. Recently, different channel attention modules have been proposed and implemented in various ways. Generally speaking, they are mainly based on convolution and pooling operations. In this paper, we propose Gaussian process embedded channel attention (GPCA) module and further interpret the channel attention schemes in a probabilistic way. The GPCA module intends to model the correlations among the channels, which are assumed to be captured by beta distributed variables. As the beta distribution cannot be integrated into the end-to-end training of convolutional neural networks (CNNs) with a mathematically tractable solution, we utilize an approximation of the beta distribution to solve this problem. To specify, we adapt a Sigmoid-Gaussian approximation, in which the Gaussian distributed variables are transferred into the interval [0,1]. The Gaussian process is then utilized to model the correlations among different channels. In this case, a mathematically tractable solution is derived. The GPCA module can be efficiently implemented and integrated into the end-to-end training of the CNNs. Experimental results demonstrate the promising performance of the proposed GPCA module. Codes are available at https://github.com/PRIS-CV/GPCA.
STRUCTURAL MODEL OF SPIRITUALITY AS A PHILOSOPHICAL PHENOMENON

The issues of the development and formation of the spirituality of the individual in the conditions of the current social crisis do not lose their relevance. Currently, the development of philosophical tools and specific recommendations for the formation of the spirituality of a particular person are becoming more and more in demand. The article deals with the structural model of the philosophical phenomenon of spirituality. In modern psychological and pedagogical literature, spirituality has not yet acquired the status of an established philosophical construct as an independent phenomenon. Based on the analysis of the views of researchers studying spirituality, the author includes the following content characteristics in its structure: moral values, moral qualities, attitude towards people, self-perception and attitude, spiritual abilities, self-development, self-regulation and volitional control. The study presents the characteristics of each component and the rationale for their philosophical significance in the structure of spirituality. Further study and analysis of the identified structural components of the spirituality model will make it possible to develop a diagnostic technique for studying the formation of the meaningful aspects of spirituality.