Reviewerly/exHarmony huggingface dataset:
Temporality- and Frequency-aware Graph Contrastive Learning for Temporal Network

Graph contrastive learning (GCL) methods aim to learn more distinguishable representations by contrasting positive and negative samples. They have received increasing attention in recent years due to their wide application in recommender systems and knowledge graphs. However, almost all GCL methods are applied to static networks and can not be extended to temporal networks directly. Furthermore, recent GCL models treat low- and high-frequency nodes equally in overall training objectives, which hinders the prediction precision. To solve the aforementioned problems, in this paper, we propose a <u>T</u>emporality- and <u>F</u>requency-aware <u>G</u>raph <u>C</u>ontrastive <u>L</u>earning for temporal networks (TF-GCL). Specifically, to learn more diverse representations for infrequent nodes and fully explore temporal information, we first generate two augmented views from the input graph based on topological and temporal perspectives. We then design a temporality and frequency-aware objective function to maximize the agreement between node representations of the two views. Experimental results demonstrate that TF-GCL remarkably achieves more robust node representations and significantly outperforms the state-of-the-art methods on six temporal link prediction benchmark datasets. Considering the reproducibility, we release our code on Github.
BidH: A Bidirectional Hierarchical Model for Nested Named Entity Recognition

Nested Name Entity Recognition is to identify the entities with nested relationships from sentences, which has various applications ranging from relation extraction to semantic understanding. However, existing methods have two drawbacks, i.e., 1) error propagation when identifying entities at different nesting levels and 2) unable to uncover and utilize the complex correlations between the inner and outer entities. To address these two defects, we propose a bidirectional hierarchical(BidH) model for nested name entity recognition. BidH consists of a forward module and a backward module, where the former first extracts the inner entities and then extracts the outer ones, while the latter extracts the entities in the opposite direction. Furthermore, we design an entity masked self attention mechanism to combine the two modules by fusing their predictions and hidden states layer by layer. BidH can effectively deal with error propagation and exploit the correlations between entities at different nesting levels to improve the recognition accuracy. Experiments on the GENIA dataset show that BidH outperforms the state-of-the-art nested named entity recognition models in terms of F1 score.
Dual-Task Learning for Multi-Behavior Sequential Recommendation

Recently, sequential recommendation has become a research hotspot while multi-behavior sequential recommendation (MBSR) that exploits users' heterogeneous interactions in sequences has received relatively little attention. Existing works often overlook the complementary effect of different perspectives when addressing the MBSR problem. In addition, there are two specific challenges remained to be addressed. One is the heterogeneity of a user's intention and the context information, the other one is the sparsity of the interactions of target behavior. To release the potential of multi-behavior interaction sequences, we propose a novel framework named NextIP that adopts a dual-task learning strategy to convert the problem to two specific tasks, i.e., <u>next</u>-<u>i</u>tem prediction and <u>p</u>urchase prediction. For next-item prediction, we design a target-behavior aware context aggregator (TBCG), which utilizes the next behavior to guide all kinds of behavior-specific item sub-sequences to jointly predict the next item. For purchase prediction, we design a behavior-aware self-attention (BSA) mechanism to extract a user's behavior-specific interests and treat them as negative samples to learn the user's purchase preferences. Extensive experimental results on two public datasets show that our NextIP performs significantly better than the state-of-the-art methods.
Knowledge Enhanced Multi-Interest Network for the Generation of Recommendation Candidates

Candidate generation task requires that candidates related to user interests need to be extracted in realtime. Previous works usually transform a user's behavior sequence to a unified embedding, which can not reflect the user's multiple interests. Some recent works like Comirec and Octopus use multi-channel structures to capture users' diverse interests. They cluster users' historical behaviors into several groups, claiming that one group represents one interest. However, these methods have some limitations. First, an item may correspond to multiple interests of users, thereby simply allocating it to just one interest group will make the modeling of users' interests coarse-grained and inaccurate. Second, explaining user interests at the level of items is rather vague and not convincing. In this paper, we propose a Knowledge Enhanced Multi-Interest Network: KEMI, which exploits knowledge graphs to help learn users' diverse interest representations via heterogeneous graph neural networks (HGNNs) and a novel dual memory network. Specifically, we use HGNNs to capture the semantic representation of knowledge entities and a novel dual memory network to learn a user's diverse interests from his behavior sequence. Through memory slots of the user memory network and the item memory network, we can learn multiple interests for each user and each item. Meanwhile, by binding the entities to the channels of memory networks, we enable it to be explained from the perspective of the knowledge graph, which enhances the interpretability and understanding of user interests. We conduct extensive experiments on two industrial and publicly available datasets. Experimental results demonstrate that our model achieves significant improvements over state-of-the-art baseline models.
Adversarial Robustness through Bias Variance Decomposition

Federated learning learns a neural network model by aggregating the knowledge from a group of distributed clients under the privacy-preserving constraint. In this work, we show that this paradigm might inherit the adversarial vulnerability of the centralized neural network, i.e., it has deteriorated performance on adversarial examples when the model is deployed. This is even more alarming when federated learning paradigm is designed to approximate the updating behavior of a centralized neural network. To solve this problem, we propose an adversarially robust federated learning framework, named Fed_BVA, with improved server and client update mechanisms. This is motivated by our observation that the generalization error in federated learning can be naturally decomposed into the bias and variance triggered by multiple clients' predictions. Thus, we propose to generate the adversarial examples via maximizing the bias and variance during server update, and learn the adversarially robust model updates with those examples during client update. As a result, an adversarially robust neural network can be aggregated from these improved local clients' model updates. The experiments are conducted on multiple benchmark data sets using several prevalent neural network models, and the empirical results show that our framework is robust against white-box and black-box adversarial corruptions under both IID and non-IID settings.
Co-Training with Validation

In the scenarios of low-resource natural language applications, Semi-supervised Relation Extraction (SRE) plays a key role in mitigating the scarcity of labelled sentences by harnessing a large amount of unlabeled corpus. Current SRE methods are mainly designed based on the paradigm of Self-Training with Validation (STV), which employs two learners and each of them plays the single role of annotator or validator. However, such a single role setting under-utilizes the potential of learners in promoting new labelled instances from unlabeled corpus. In this paper, we propose a generic SRE paradigm, called Co-Training with Validation (CTV), for making full use of learners to benefit more from unlabeled corpus. In CTV, each learner alternately plays the roles of annotator and validator to generate and validate pseudo-labelled instances. Thus, more high-quality instances are exploited and two learners can be reinforced by each other during the learning process. Experimental results on two public datasets show that our CTV considerably outperforms the state-of-the-art SRE techniques, and works well with different kinds of learners for relation extraction.
Taxonomy-Enhanced Graph Neural Networks

Despite the recent success of Graph Neural Networks (GNNs), their learning pipeline is guided only by the input graph and the desired output of certain tasks, failing to capture useful patterns when not enough data are presented. Existing attempts incorporate auxiliary knowledge to mitigate this issue, most of which are not in a unified structure or hard to obtain. Noticing that nodes in graphs usually form implicit hierarchical structures, we proposed to integrate category taxonomies into the learning process of GNNs. A category taxonomy is a form of domain knowledge with a hierarchical tree structure, which is widely adopted in real-world scenarios. In this paper, we introduce Taxonomy-Enhanced Graph Neural Networks (Taxo-GNN). Specifically, we jointly optimize the taxonomy representation and node representation tasks, where categories in taxonomy are mapped to Gaussian distributions and nodes are embedded with the GNN framework. To characterize the bidirectional interaction between the taxonomy and the graph, the model is comprised of two modules, namely information distillation for taxonomy and knowledge fusion to graph. Information is first distilled from the graph and aligned with the hierarchical structure of the taxonomy in a bottom-to-top mechanism.After that, knowledge brought by the taxonomy is in turn fused to the graph convolution process, in the form of taxonomy-aware aggregation weights and taxonomy-augmented contexts. Extensive experiments on real-world datasets in multiple downstream tasks verify the effectiveness of our model.
Generative Adversarial Zero-Shot Learning for Cold-Start News Recommendation

News recommendation models extremely rely on the interactive information between users and news articles to personalize the recommendation. Therefore, one of their most serious challenges is the cold-start problem (CSP). Their performance is dropped intensely for new users or new news. Zero-shot learning helps in synthesizing a virtual representation of the missing data in a variety of application tasks. Therefore, it can be a promising solution for CSP to generate virtual interaction behaviors for new users or new news articles. In this paper, we utilize the generative adversarial zero-shot learning in building a framework, namely, GAZRec, which is able to address the CSP caused by purely new users or new news. GAZRec can be flexibly applied to any neural news recommendation model. According to the experimental evaluations, applying the proposed framework to various news recommendation baselines attains a significant AUC improvement of 1% - 21% in different cold start scenarios and 1.2% - 6.6% in the regular situation when both users and news have a few interactions.
Plotly.plus, an Improved Dataset for Visualization Recommendation

Visualization recommendation is a novel and challenging field of study, whose aim is to provide non-expert users with automatic tools for insight discovery from data. Advances in this research area are hindered by the absence of reliable datasets on which to train the recommender systems. To the best of our knowledge, Plotly corpus is the only publicly available dataset, but as complained by many authors and discussed in this article, it contains many labeling errors, which greatly limits its usefulness. We release an improved version of the original dataset, named Plotly.plus, which we obtained through an automated procedure with minimal post-editing. In addition to a manual validation by a group of data science students, we demonstrate that when training two state-of-the-art abstract image classifiers on Plotly.plus, systems' performance improves more than twice as much as when the original dataset is used, showing that Plotly.plus facilitates the discovery of significant perceptual patterns.
Diverse Effective Relationship Exploration for Cooperative Multi-Agent Reinforcement Learning

In some complex multi-agent environments, the types of relationships between agents are diverse and their intensity changes during the policy learning process. Theoretically, some of these relationships can facilitate cooperative policy learning. However, acquiring these relationships is an intractable problem. To tackle the problem, we propose a diverse effective relationship exploration based multi-agent reinforcement learning (DERE) method. Specifically, a potential fields model is firstly designed to represent relationships between agents. Then to encourage the exploration of effective relationships, we define an information-theoretic objective function. Finally, an intrinsic reward function is designed to optimize the information-theoretic objective, meanwhile, guide agents to learn more effective collaborative policies. Experimental results show that our method outperforms state-of-the-art methods on both super hard StarCraft II micromanagement tasks (SMAC) and Google Research Football (GRF).
Binary Transformation Method for Multi-Label Stream Classification

Data streams produce extensive data with high throughput from various domains and require copious amounts of computational resources and energy. Many data streams are generated as multi-labeled and classifying this data is computationally demanding. Some of the most well-known methods for Multi-Label Stream Classification are Problem Transformation schemes; however, previous work on this area does not satisfy the efficiency demands of multi-label data streams. In this study, we propose a novel Problem Transformation method for Multi-Label Stream Classification called Binary Transformation, which utilizes regression algorithms by transforming the labels into a continuous value. We compare our method against three of the leading problem transformation methods using eight datasets. Our results show that Binary Transformation achieves statistically similar effectiveness and provides a much higher level of efficiency.
MLadder: An Online Training System for Machine Learning and Data Science Education

Education on machine learning and data science has drawn a lot of attention in both higher education and vocational training. Although various tools and services such as Jupyter Notebook and Google Cloud's AI have been developed for building and training models, they are not suitable for direct use in educational settings. For example, teachers expect a platform where they can easily distribute and grade programming assignments, and students want to quickly start coding and training models without the burden of setting up an environment. To this end, we develop MLadder, an online training system for machine learning and data science education. Specifically, we seamlessly integrate two open-source software, CodaLab and Jupyter Notebook, which are used for hosting assignments and building models, respectively. Moreover, we devise several methods to make the system lightweight and scalable, so that it can be deployed on-premises even with limited resources. We have used MLadder in the machine learning and data science courses in our school and facilitated both teaching and learning.
Towards Self-supervised Learning on Graphs with Heterophily

Recently emerged heterophilous graph neural networks have significantly reduced the reliance on the assumption of graph homophily where linked nodes have similar features and labels. These methods focus on a supervised setting that relies on labeling information heavily and presents the limitations on general graph downstream tasks. In this work, we propose a self-supervised representation learning paradigm on graphs with heterophily (namely HGRL) for improving the generalizability of node representations, where node representations are optimized without any label guidance. Inspired by the designs of existing heterophilous graph neural networks, HGRL learns the node representations by preserving the node original features and capturing informative distant neighbors. Such two properties are obtained through carefully designed pretext tasks that are optimized based on estimated high-order mutual information. Theoretical analysis interprets the connections between HGRL and existing advanced graph neural network designs. Extensive experiments on different downstream tasks demonstrate the effectiveness of the proposed framework.
Memory Graph with Message Rehearsal for Multi-Turn Dialogue Generation

Multi-turn dialogue system has attracted increasing attention in both academic and industry community. Multi-turn dialogue generation task is a challenging work as the relations among words, utterances and external knowledge are extremely complex. However, the existing methods only focus on constructing the relations between current utterance and historical utterances, and they even oversimplify the relation mining process. Moreover, with the accumulation of dialogue information, the deep semantic information is difficult to understand so that it needs a mechanism with the ability of reasoning and digesting information repeatedly, which is ignored by previous methods. In order to solve the above problems, we propose a Memory Graph with Message Rehearsal (MGMR) for dialogue generation based on the cognitive process of human memory. MGMR contains three main modules: sensory memory, short-term memory and long-term memory. Sensory memory converts the current utterance into embeddings from both word-level and sentence-level. We design a message rehearsal module in short-term memory to extract valuable information of current utterance deeply and repeatedly combined with the relative historical dialogue information and external knowledge stored in long-term memory. Furthermore, we innovatively design a memory graph in long-term memory to construct the relations among words, utterances and knowledge. The memory graph achieves three goals: extracting accurate relations between current utterance and historical utterances, updating the historical dialogue information, and achieving knowledge precipitation by expanding memory graph with the key words and relevant external knowledge of current utterance. We evaluate our model on real-world datasets and achieve better performance compared with the existing state-of-the-art methods.
Self-Paced and Discrete Multiple Kernel <i>k</i> -Means

Multiple Kernel K-means (MKKM) uses various kernels from different sources to improve clustering performance. However, most of the existing models are non-convex, which is prone to be stuck into bad local optimum, especially with noise and outliers. To address the issue, we propose a novel Self-Paced and Discrete Multiple Kernel K-Means (SPD-MKKM). It learns the MKKM model in a meaningful order by progressing both samples and kernels from easy to complex, which is beneficial to avoid bad local optimum. In addition, whereas existing methods optimize in two stages: learning the relaxation matrix and then finding the discrete one by extra discretization, our work can directly gain the discrete cluster indicator matrix without extra process. What's more, a well-designed alternative optimization is employed to reduce the overall computational complexity via using the coordinate descent technique. Finally, thorough experiments performed on real-world datasets illustrated the excellence and efficacy of our method.
A Context-Enhanced Generate-then-Evaluate Framework for Chinese Abbreviation Prediction

As a popular form of lexicalization, abbreviation is widely used in both oral and written language and plays an important role in various Natural Language Processing applications. However, current approaches cannot ensure that the predicted abbreviation preserves the meaning of its full form and maintains fluency. In this paper, we introduce a fresh perspective to evaluate the quality of abbreviations within their textual contexts with pre-trained language model. To this end, we propose a novel two-stage generate-then-evaluate framework enhanced by context, which consists of a generation model to generate multiple candidate abbreviations and an evaluation model to evaluate their quality within their contexts. Experimental results show that our framework consistently outperforms all the existing approaches, achieving 53.2% [email protected] performance with a 5.6 points improvement compared to its previous best result. Our code and data are publicly available at https://github.com/HavenTong/CEGE.
DocSemMap 2.0

Methods for automated semantic labeling of data are an indispensable basis for increasing the usability of data. On the one hand, they contribute to the homogenization of the annotations and thus to the increase in quality; on the other hand, they reduce the modeling effort, provided that the quality of the used methodology is sufficient. In the past, research has focused primarily on data- and label-based methods. Another approach that has received recent attention is the incorporation of textual data documentations to support the automatic mapping of datasets to a knowledge graph. However, upon deeper analysis, our recent approach called DocSemMap gives away potential in a number of places. In this paper, we extend the current state of the art approach by uncovering existing shortcomings and presenting our own improvements. Using a sequence-to-sequence model (Seq2Seq), we exploit the context of datasets. An additional introduced classifier provides the linkage of documentation and labels for prediction. Our extended approach achieves a sustainable improvement in comparison to the reference approach.
WDRASS: A Web-scale Dataset for Document Retrieval and Answer Sentence Selection

Open-Domain Question Answering (ODQA) systems generate answers from relevant text returned by search engines, e.g., lexical features-based such as BM25, or embeddings-based such as dense passage retrieval (DPR). Few datasets are available for this task: they mainly focus on QA systems based on machine reading (MR) approach, and show problematic evaluation, mostly based on uncontextualized short answer matching. In this paper, we present WDRASS, a dataset for ODQA based on answer sentence selection (AS2) models, which consider sentences as candidate answers for QA systems. WDRASS consists of ∼64k questions and 800k+ labeled passages and sentences extracted from 30M documents. We evaluate the dataset by training models on it and comparing with the same models trained on Google NQ. Our experiments show that WDRASS significantly improves the performance of retrieval and reranking models, thus boosting the accuracy of downstream QA tasks. We believe our dataset can produce significant impact in advancing IR research.