Reviewerly/exHarmony huggingface dataset:
CORE-SG: Efficient Computation of Multiple MSTs for Density-Based Methods

Several popular density-based methods for unsuper-vised and semi-supervised learning tasks, including clustering and classification, can be formulated as instances of a framework that is based on the processing of a minimum spanning tree of the data, where the edge weights correspond to a form of (unnormalized) density estimate w.r.t. a smoothing parameter <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$m_{pts}$</tex> . While density-based methods are considered to be robust w.r.t. <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$m_{pts}$</tex> in the sense that small changes in its value usually lead to slight or no changes in the resulting structure, wider ranges of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$m_{pts}$</tex> values may lead to different results that a user would like to analyze before choosing the most suitable value for a given data set or application. However, to explore multiple results for a range of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$m_{pts}$</tex> values, until recently, one had to re-run the density-based method for each value in the range independently, which is computationally inefficient. This paper proposes a new computationally efficient approach to compute multiple density-based minimum spanning trees w.r.t. a set of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$m_{pts}$</tex> values by leveraging a graph obtained from a single run of the density-based algorithm, without the need for re-runs of the original algorithm. We present theoretical and experimental results that show that our approach overcomes the drawbacks of the previous state-of-the-art, and it is considerably superior in runtime and graph size while being easier to implement. Our experimental evaluation using synthetic and real data shows that our strategy can lead to speed-up factors of hundreds to thousands of times on the computation of density-based minimum spanning trees.
Synthesizing Privacy Preserving Entity Resolution Datasets

Entity resolution (ER) is a core problem in data integration. Many companies have lots of datasets where ER needs to be conducted to integrate the data. On the one hand, it is nontrivial for non-ER experts within companies to design ER solutions. On the other hand, most companies are reluctant to release their real datasets for multiple reasons (e.g., privacy issues). A typical solution from the machine learning (ML) and the statistical community is to create surrogate (a.k.a. analogous) datasets based on the real dataset, release these surrogate datasets to the public to train ML models, such that these models trained on surrogate datasets can be either directly used or be adapted for the real dataset by the companies. In this paper, we study a new problem of synthesizing surrogate ER datasets using transformer models, with the goal that the ER model trained on the synthesized dataset can be used directly on the real dataset. We propose privacy preserving methods to synthesize ER datasets: we first learn the true similarity distributions of both matching and non-matching entity pairs from real dataset. We then devise algorithms that satisfy differential privacy and can synthesize fake but semantically meaningful entities, add matching and non-matching labels to these fake entity pairs, and ensure that the fake and real datasets have similar distributions. We also describe a method for entity rejection to avoid synthesizing bad fake entities that may destroy the original distributions. Extensive experiments show that ER matchers trained on real and synthetic ER datasets have very close performance on the same test sets - their <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$F1$</tex> scores differ within 6% on 3 commonly used ER datasets, and their average precision, recall differences are less than 5%.
Lasagne: A Multi-Layer Graph Convolutional Network Framework via Node-aware Deep Architecture (Extended Abstract)

In this paper, we propose Lasagne, a novel multi-layer graph convolutional network (GCN) framework to over-come the over-smoothing problem and realize the full poten-tials of deep GCNs. We analyze how node localities affect the information propagation in GCN, propose an adaptive novel node aggregation mechanism and further demystify from a mutual information view. Evaluation results on both real-world benchmark data sets and large-scale industrial production data sets show Lasagne significantly outperforms the state-of- the-art methods without considering the node locality.
Identification for Deep Neural Network: Simply Adjusting Few Weights!

Through the development of powerful algorithms and design tools, deep neural networks (DNNs) have recently approached or even surpassed human-level performance in many real-world applications. Nowadays, since a product-level DNN modeling requires a large amount of training data and expensive computing resources and thus DNN models are considered as valuable data, protecting the intellectual property (IP) of DNN builders becomes an important problem in the security domain. In this paper, we propose a novel watermarking approach that only requires adjusting a few weights, as opposed to prior works that embed watermarks via end-to-end training. The protected model with tiny parameter modifications can output pre-specified labels with carefully selected key samples as inputs, which serves as a strong proof of ownership. Besides, our methodology can be naturally extended to identification, i.e., embedding unique watermarks to identify different users. Watermark embedding is achieved by modifying a very small subset of parameters, guaranteeing a high fidelity while dramatically reducing the computational overhead. The experimental results demonstrate that the proposed algorithm can embed key samples with a high success rate, while well preserving the original functionality of the target model. We show that the proposed method is robust against various transformation attacks.
VC-Tune: Tuning and Exploring Distributed Vertex-Centric Graph Systems

Distributed vertex-centric graph systems, or VC-systems, have achieved tremendous success in the industry. A common usage pattern of VC-systems is multi-processing, or the concurrent processing of multiple unit tasks. Example multi-processing includes answering multiple single-source shortest path queries on a graph. However, concurrent processing of all the unit tasks may overload the system with excessive memory usage, leading to intolerable system delays. To ad-dress the important challenge, we present V C- Tune, a system with a convenient interface to help practitioners orchestrate the unit tasks for improving the overall performance within the system limit. This demonstration allows the audience to interact with our system to explore the configuration space of multi-processing in VC-systems and compare different sys-tem configurations. In addition, we embed into the system an automatic configuration search algorithm, which can directly recommend to the practitioners a suitable configuration that gives a satisfactory system performance. An introduction video is at (https://sites.google.com/view/vc-tune-video).
Triple-Fact Retriever: An explainable reasoning retrieval model for multi-hop QA problem

Nowadays, multi-hop question answer (QA) problem is challenging and not well solved in the QA community. The dominant bottleneck of the multi-hop QA problem is the need for a reasoning retriever to fetch a document path from an open-domain corpus (e.g., Wikipedia). A reasoning retriever aims to collect an evidence document from large corpora at one hop retrieval and aggregate the evidence for subsequent hop retrieval, which yields a document path after multi-hop retrieval. There exist two challenges, (1) to fetch the evidence document in an efficient and explainable way at one hop retrieval and (2) to update the question information by aggregating the evidence from the retrieved document after each hop retrieval. To address these two challenges, we propose a triple-fact-based retrieval model to effectively retrieve a related document path in an explainable way for each question. We extract a structured representation from the unstructured document and utilize the knowledge of pre-trained language model (PLM) to do the semantic-level matching between the question and document. We evaluate the proposed Triple-fact Retriever model on the recently proposed open-domain multi-hop QA dataset, HotpotQA, and a cross-document multi-step Reading Comprehension dataset, Wikihop. The results <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> The source code is available on our website: https://github.com/Rebaccamin/triple_retriever. demonstrate that the Triple-fact retriever outperforms the existing baseline retrieval works.
Joint Evidential $K$-Nearest Neighbor Classification

The performance of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$K$</tex> -nearest neighbor (K-NN) classification depends significantly on the searched neighborhoods of test samples, namely, the neighborhood size <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$K$</tex> and the used distance metric. For the two issues, many methods either to acquire the adaptive <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$K$</tex> or to learn a variant metric have been presented and yielded appropriate performance. However, most of the existing methods ignore the fact that these two factors can be jointly learned. Besides, nearly all the metric learning methods aim to shrink intra-class distance while expanding inter-class distance. In this way, embedding the learned metric directly into the K-NN does not efficiently improve its accuracy. To address these issues, we propose a joint K-NN algorithm with the help of evidence theory, optimizing the joint learning of adaptive <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$K$</tex> and distance matrix based on the feedback from error function. Ablation study demonstrates the performance improvement from the joint learning, and comparison experiments on real-world datasets show that our approach consumes competitive running time and achieves better performance than other state-of-the-art algorithms.
Constructing a skeleton database and enriching it using a Generative Adversarial Network (GAN) simulator to assess human movement

This Ph.D. thesis develops a neural network simulator for quantifying, tagging, and inferring human gestures using an anonymized patient gesture database. Deep Learning (DL) applications require a sufficient data set for training. In this work, we propose enriching a database that contains a limited number of videos of human physiotherapy exercises by generating synthetic data. Our pose generator produces human movement in the form of skeletal vectors. We use OpenPose (OP) to convert videos and images containing multiple individuals into human skeletal. Within every video frame, OP represents each pose of the human skeleton as a vector in three-dimensional Euclidean space. We employ the Generative Adversarial Network (GAN) to generate new samples and control the motion parameters. We rearrange the joints in our skeletal model to emphasize the connections between them by using depth-first search (DFS), a tree structure search algorithm. Moreover, this research examines common challenges associated with capturing human gesture data, including synchronizing activities, temporal, and spatial relations, and how to address them. We intend to build an innovative simulator that will generate a set of human virtual choreography movements from a textual script.
Hybrid Subgraph Matching Framework Powered by Sketch Tree for Distributed Systems

With the rapid growth of graph scale, challenges emerge for subgraph search when the data graph cannot reside in the memory of a single machine. It is important to develop practical algorithms to answer subgraph queries in distributed systems and has attracted extensive attention in recent years. The existing join-based algorithms are natively supported in many distributed engines, but they often suffer from a large number of invalid intermediate results and duplicate computation. The exploration-based algorithms minimize invalid intermediate results, while they are likely to produce results of exponential size. In this paper, we propose an efficient hybrid subgraph matching framework that integrates the advantages of both join-based and exploration-based paradigms. We formulate a novel decomposition for the query graph, namely sketch tree, which can reduce invalid intermediate results and avoid duplicate computation. We implement the proposed algorithm in the Pregel + system and optimize the communication cost powered by the sketch tree. Extensive experiments on real graphs demonstrate that our proposed algorithm significantly outperforms the state-of-the-art join-based and exploration-based methods.
EC-Graph: A Distributed Graph Neural Network System with Error-Compensated Compression

The high training costs of graph neural networks (GNNs) have limited their applicability on large graphs, e.g., graphs with hundreds of millions of vertices which have become common in the era of big data. A few recent studies propose distributed GNN systems. However, these systems may generate high communication costs due to the extensive message passing among graph vertices stored on different machines. To address such limitations, in the paper, 1) we propose a distributed GNN computation system named EC-Graph for CPU clusters, which drastically reduces the communication costs among the machines by message compression; 2) we design a requesting-end compensation method for the embeddings to mitigate the errors induced by compression in the forward propagation and a Bit-Tuner to adaptively balance the model accuracy and message size; and 3) we propose a responding-end compensation approach for the embedding gradients in the backward propagation. Extensive experiments over large real-world datasets show that EC-Graph outperforms state-of-the-art distributed GNN systems on two CPU clusters of different sizes.
Self-reconstructive evidential clustering for high-dimensional data

Although many algorithms have been presented to tackle the curse of dimensionality in high-dimensional clustering, most of these algorithms require prior knowledge of the number of clusters. Besides, these existing algorithms create only a hard or fuzzy partition for high-dimensional objects, which are often located in highly overlapping areas. The adoption of hard/fuzzy partition ignores the ambiguity in the assignment of objects and may lead to performance degradation. To address these issues, we propose a novel self-reconstructive evidential clustering (SREC) algorithm. After learning the correlations between objects from a self-reconstruction process, SREC provides a human-readable chart. Through this chart, users can select several objects existing in the dataset as the cluster centers, instead of just detecting the number of clusters. Under the framework of evidence theory, SREC derives a more flexible credal partition that improves the fault tolerance of clustering. Ablation study demonstrates the benefits of the self-reconstruction and evidence theory. Comparison experiments on real-world datasets show that SREC consumes competitive running time and performs better than other state-of-the-art algorithms. We also apply SREC in a real-world application scenario to illustrate the rationality of selecting cluster centers by human intervention.
Boosting Entity Mention Detection for Targetted Twitter Streams with Global Contextual Embeddings

Microblogging sites, like Twitter, have emerged as ubiquitous sources of information. Two important tasks related to the automatic extraction and analysis of information in Microblogs are Entity Mention Detection (EMD) and Entity Detection (ED). The state-of-the-art EMD systems aim to model the non-literary nature of microblog text by training upon offline static datasets. They extract a combination of surface-level features - orthographic, lexical, and semantic - from individual messages for noisy text modeling and entity extraction. But given the constantly evolving nature of microblog streams, detecting all entity mentions from such varying yet limited context of short messages remains a difficult problem. To this end, we propose a framework named EMD Globalizer, better suited for the execution of EMD learners on microblog streams. It deviates from the processing of isolated microblog messages by existing EMD systems, where learned knowledge from the immediate context of a message is used to suggest entities. Instead, it recognizes that messages within a microblog stream are topically related and often repeat entity mentions, thereby leaving the scope for EMD systems to go beyond the localized processing of individual messages. After an initial extraction of entity candidates by an EMD system, the proposed framework leverages occurrence mining to find additional candidate mentions that are missed during this first detection. Aggregating the local contextual representations of these mentions, a global embedding is drawn from the collective context of an entity candidate within a stream. The global embeddings are then utilized to separate entities within the candidates from false positives. All mentions of said entities from the stream are produced in the framework's final outputs. Our experiments show that EMD Globalizer can enhance the effectiveness of all existing EMD systems that we tested (on average by 25.61 %) with a small additional computational overhead.
Rule Learning over Knowledge Graphs with Genetic Logic Programming

Declarative rules such as Prolog and Datalog rules are common formalisms to express expert knowledge and facts. They play an important role in Knowledge Graph (KG) construction and completion. Such rules not only encode the expert background knowledge and the relational patterns among the data, but also infer new knowledge and insights from them. Formalizing rules is often a laborious manual process, while learning them from data automatically can ease this process. Within the rule hypothesis space, current approaches resort to exhaustive search with a number of heuristics and syntactic restrictions on the rule language, which impacts the efficiency and quality of the outcome rules. In this paper, we extend the rule hypothesis space from usual path rules to general Datalog rule space by proposing a novel Genetic Logic Programming algorithm named Evoda. It is an iterative process to learn high-quality rules over large scale KG for a matter of seconds. We have performed experiments over multiple real-world KGs and various evaluation metrics to show its mining capabilities for higher quality rules and more precise predictions. Additionally, we have applied it on the KG completion tasks to illustrate its competitiveness with several state-of-the-art embedding or neural-based models. The experiments demonstrate the feasibility, effectiveness and efficiency of the Evoda algorithm.
Efficient Learning-based Community-Preserving Graph Generation

Graph generation is beneficial to comprehend the creation of meaningful structures of networks in a broad spec-trum of applications such as social networks and biological net-works. Recent studies tend to leverage deep learning techniques to learn the topology structures in graphs. However, we notice that the community structure, which is one of the most unique and prominent features of the graph, cannot be well captured by the existing graph generators. Moreover, the existing advanced deep learning-based graph generators are not efficient and scalable, which can only handle small graphs. In this paper, we propose a novel community-preserving generative adversarial network (CPGAN) for effective and efficient (scalable) graph simulation. We employ graph convolution networks in the encoder and share parameters in the generation process to transmit information about community structures and preserve the permutation-invariance in CPGAN. We conducted extensive experiments on benchmark datasets, including six sets of real-life graphs. The results demonstrate that CPGAN can achieve a good trade-off between efficiency (scalability) and graph simulation quality for real-life graph simulation compared with state-of-the-art baselines.
Subspace Embedding Based New Paper Recommendation

As huge numbers of academic papers are published every year, it is critical to be able to recommend high quality papers. The typical evaluation method for papers is to use citation information, which however is not applicable to new papers. To address such a shortcoming, in this paper, we consider a novel perspective on the association between the content difference of a paper, with respect to other papers, and its innovation. Since innovation has often domain-specific characteristics and forms, we introduce the concept of subspace to describe the commonly recognized aspects of paper contents, namely background, methods and results. A set of expert rules are formalized to annotate the differences between papers, based on which a twin-network is proposed for learning the embeddings of papers in different subspaces. A series of empirical studies show that there are clear correlations between a paper influence and its difference with others in those subspaces. The results also show the characteristics of innovation in different scientific disciplines. To take into account information about academic networks for paper recommendation, we propose a graph convolutional neural method to combine the paper content with other related elements, where user interests and academic influences are modeled asymmetric. Experimental results on real datasets show that our method is more effective than other baseline methods for new paper recommendation. We also discuss the characteristics of scientific disciplines and authors to show the effectiveness of modeling the asymmetric user interests and influences. Finally, we verify the reusability of our method on a patent dataset. The results show that it is also applicable to academic data with low-resource features.
PSP: Progressive Space Pruning for Efficient Graph Neural Architecture Search

Recently, graph neural network (GNN) has achieved great success in many graph learning tasks such as node classifi-cation and graph classification. However, there is no single GNN architecture that can fit different graph datasets. Designing an effective GNN for a specific graph dataset requires considerable expert experience and huge computational costs. Inspired by the success of neural architecture search (NAS), searching the GNN architectures automatically has attracted more and more attention. Motivated by the fact that the search space plays a critical role in the NAS, we propose a novel and effective graph neural architecture search method called PSP from the perspective of search space design in this paper. We first propose an expressive search space composed of multiple cells. Instead of searching the entire architecture, we focus on searching the architecture of the cell. Then, we propose a progressive space pruning-based algorithm to search the architectures efficiently. Moreover, the data-specific search spaces and architectures ob-tained by PSP can be transferred to new graph datasets based on meta-learning. Extensive experimental results on different types of graph datasets reveal that PSP outperforms the state-of-the-art handcrafted architectures and the existing NAS methods in terms of effectiveness and efficiency.