Miracle-dz/UniFAR-FTUs huggingface dataset:
Agenda: Robust Personalized PageRanks in Evolving Graphs

Given a source node s and a target node t in a graph G, the Personalized PageRank (PPR) from s to t is the probability of a random walk starting from s terminates at t. PPR is a classic measure of the relevance among different nodes in a graph, and has been applied in numerous real-world systems. However, existing techniques for PPR queries are not robust to dynamic real-world graphs, which typically have different evolving speeds. Their performance is significantly degraded either at a lower graph evolving rate (e.g., much more queries than updates) or a higher rate. To address the above deficiencies, we propose Agenda to efficiently process, with strong approximation guarantees, the single-source PPR (SSPPR) queries on dynamically evolving graphs with various evolving speeds. Compared with previous methods, Agenda has significantly better workload robustness, while ensuring the same result accuracy. Agenda also has theoretically-guaranteed small query and update costs. Experiments on up to billion-edge scale graphs show that Agenda significantly outperforms state-of-the-art methods for various query/update workloads, while maintaining better or comparable approximation accuracies.
CDCEO'21 - First Workshop on Complex Data Challenges in Earth Observation

High-resolution remote sensing technology for Earth Observation (EO) has radically changed how we monitor the state of our planet around the clock. An effective interpretation of the resulting complex large-scale time series adopts the best machine learning techniques from signal processing, computer vision, pattern recognition, and artificial intelligence. The First Workshop on Complex Data Challenges in Earth Observation was open to both method development and advanced applications in a wide range of related topics, including image and signal processing, gap-filling, data fusion, feature extraction, prediction of spatio-temporal features, and the detection of rules underlying the observed state transitions and causal relationships. The full agenda, featuring keynotes and a selection of high quality contributed talks is available online at www.iarai. ac.at/cdceo21
Cformer: Semi-Supervised Text Clustering Based on Pseudo Labeling

We propose a semi-supervised learning method called Cformer for automatic clustering of text documents in cases where clusters are described by a small number of labeled examples, while the majority of training examples are unlabeled. We motivate this setting with an application in contextual programmatic advertising, a type of content placement on news pages that does not exploit personal information about visitors but relies on the availability of a high-quality clustering computed on the basis of a small number of labeled samples. To enable text clustering with little training data, Cformer leverages the teacher-student architecture of Meta Pseudo Labels. In addition to unlabeled data, Cformer uses a small amount of labeled data to describe the clusters aimed at. Our experimental results confirm that the performance of the proposed model improves the state-of-the-art if a reasonable amount of labeled data is available. The models are comparatively small and suitable for deployment in constrained environments with limited computing resources. The source code is available at https://github.com/Aha6988/Cformer
Norma: A Hybrid Feature Alignment for Class-Aware Unsupervised Domain Adaptation

Unsupervised domain adaptation is the problem of transferring extracted knowledge from a labeled source domain to an unlabeled target domain. To achieve discriminative domain adaptation recent studies take advantage of target sample pseudo-labels to impose class-aware distribution alignment across the source and target domains. Still, they have some shortcomings such as making decisions based on inaccurate pseudo-labeled samples that mislead the adaptation process. In this paper, we propose a progressive deep feature alignment, called Norma, to tackle class-aware unsupervised domain adaptation for image classification by enforcing inter-class compactness and intra-class discrepancy through a hybrid learning process. To this end, Norma's optimization process is defined based on a novel triplet loss which not only addresses soft prototype alignment but also pushes away multiple negative centroids. Also, to extract maximum discriminative domain knowledge per iteration, we propose a joint positive and negative learning procedure along with an uncertainty-guided progressive pseudo-labeling on the basis of prototype-based clustering and conditional probability. Our experimental results on several benchmarks demonstrate that Norma outperforms the state-of-the-art methods.
Answering POI-recommendation Questions using Tourism Reviews

We introduce the novel and challenging task of answering Points-of-interest (POI) recommendation questions, using a collection of reviews that describe candidate answer entities (POIs). We harvest a QA dataset that contains 47,124 paragraph-sized user questions from travelers seeking POI recommendations for hotels, attractions and restaurants. Each question can have thousands of candidate entities to choose from and each candidate is associated with a collection of unstructured reviews. Questions can include requirements based on physical location, budget, timings as well as other subjective considerations related to ambience, quality of service etc. Our dataset requires reasoning over a large number of candidate answer entities (over 5300 per question on average) and we find that running commonly used neural architectures for QA is prohibitively expensive. Further, commonly used retriever-ranker based methods also do not work well for our task due to the nature of review-documents. Thus, as a first attempt at addressing some of the novel challenges of reasoning-at-scale posed by our task, we present a task specific baseline model that uses a three-stage cluster-select-rerank architecture. The model first clusters text for each entity to identify exemplar sentences describing an entity. It then uses a neural information retrieval (IR) module to select a set of potential entities from the large candidate set. A reranker uses a deeper attention-based architecture to pick the best answers from the selected entities. This strategy performs better than a pure retrieval or a pure attention-based reasoning approach yielding nearly 25% relative improvement in Hits@3 over both approaches. To the best of our knowledge we are the first to present an unstructured QA-style task for POI-recommendation, using real-world tourism questions and POI-reviews.
CANN: Coupled Approximation Neural Network for Partial Domain Adaptation

Unsupervised domain adaptation (UDA) methods aim to transfer knowledge from a labeled source domain to an unlabeled target domain. Most existing UDA methods try to learn domain-invariant features so that the classifier trained by the source labels can automatically be adapted to the target domain. However, recent works have shown the limitations of these methods when label distributions differ between the source and target domains. Especially, in partial domain adaptation (PDA) where the source domain holds plenty of individual labels (private labels) not appeared in the target domain, the domain-invariant features can cause catastrophic performance degradation. In this paper, based on the originally favorable underlying structures of the two domains, we learn two kinds of target features, i.e., the source-approximate features and target-approximate features instead of the domain-invariant features. The source-approximate features utilize the consistency of the two domains to estimate the distribution of the source private labels. The target-approximate features enhance the feature discrimination in the target domain while detecting the hard (outlier) target samples. A novel Coupled Approximation Neural Network (CANN) has been proposed to co-train the source-approximate and target-approximate features by two parallel sub-networks without sharing the parameters. We apply CANN to three prevalent transfer learning benchmark datasets, Office-Home, Office-31, and Visda2017 with both UDA and PDA settings. The results show that CANN outperforms all baselines by a large margin in PDA and also performs best in UDA.
Learning to Learn the Future: Modeling Concept Drifts in Time Series Prediction

Time series prediction has great practical value in a wide range of real-world scenarios such as stock market and retail. Existing methods typically face model aging issue caused by the concept drift: the model performance degrades along time. Undoubtedly, the model aging issue can cause serious damage in practical usage, e.g. wrong predictions in stock price may cause catastrophic losses in the financial domain. Therefore, it is essential to address the model aging issue so as to promise the predictor's performance in the future. In this paper, we propose a novel solution to address the issue. First, we uncover the theoretical connection between the complex concept drift in time series data and the gradients of deep neural networks. Based on this, we propose a novel framework called learning to learn the future. Specifically, we develop a learning method to model the concept drift during the inference stage, which can help the model generalize well in the future. Furthermore, to mitigate the impact of noises and randomness of time series data, we propose to enhance the framework by leveraging similar series in concept drift modeling. To the best of our knowledge, our approach is the first general solution to model aging issue in time series prediction. We conduct extensive experiments on three real-world datasets, which validate the effectiveness of our framework. For instance, it achieves a relative improvement of 33% in stock price prediction over the state-of-the-art methods.
REFORM: Error-Aware Few-Shot Knowledge Graph Completion

Knowledge graphs (KGs) are of great importance in various artificial intelligence systems, such as question answering, relation extraction, and recommendation. Nevertheless, most real-world KGs are highly incomplete, with many missing relations between entities. To discover new triples (i.e., head entity, relation, tail entity), many KG completion algorithms have been proposed in recent years. However, a vast majority of existing studies often require a large number of training triples for each relation, which contradicts the fact that the frequency distribution of relations in KGs often follows a long tail distribution, meaning a majority of relations have only very few triples. Meanwhile, since most existing large-scale KGs are constructed automatically by extracting information from crowd-sourcing data using heuristic algorithms, plenty of errors could be inevitably incorporated due to the lack of human verification, which greatly reduces the performance for KG completion. To tackle the aforementioned issues, in this paper, we study a novel problem of error-aware few-shot KG completion and present a principled KG completion framework REFORM. Specifically, we formulate the problem under the few-shot learning framework, and our goal is to accumulate meta-knowledge across different meta-tasks and generalize the accumulated knowledge to the meta-test task for error-aware few-shot KG completion. To address the associated challenges resulting from insufficient training samples and inevitable errors, we propose three essential modules neighbor encoder, cross-relation aggregation, and error mitigation in each meta-task. Extensive experiments on three widely used KG datasets demonstrate the superiority of the proposed framework REFORM over competitive baseline methods.
Skyline in Crowdsourcing with Imprecise Comparisons

Given an input of a set of objects each one represented as a vector of features in a feature space, the problem of finding the skyline is the problem of determining the subset of objects that are not dominated by any other input object. An example of an application is to find the best hotel(s) with respect to some features (location, price, cleanliness, etc.) The use of the crowd for solving this problem is useful when a score of items according to their features is not available. Yet the crowd can give inconsistent answers. In this paper we study the computation of the skyline when the comparisons between objects are performed by humans. We model the problem using the threshold model [Ajtai et al, TALG 2015] in which the comparison of two objects may create errors/inconsistencies if the objects are close to each other. We provide algorithms for the problem and we analyze the required number of human comparisons and lower bounds. We also evaluate the effectiveness and efficiency of our algorithms using synthetic and real-world data.
On the Diversity and Explainability of Recommender Systems: A Practical Framework for Enterprise App Recommendation

This paper introduces an enterprise app recommendation problem with a new "to-business'' use case, which aims to assist a sales team acting as the bridge connecting the applications and developers with the customers who apply these apps to solve their business problems. Our recommender system is an assistant to the sales team, helping recommend relevant apps to the customers for their businesses and increasing the likelihood of improving sales revenue. Besides recommendation accuracy, recommendation diversity and explainability are even more crucial since they provide more exposure opportunities for app developers and improve the transparency and trustworthiness of the recommender system. To allow the sales team to explore unpopular but relevant apps and understand why such apps are recommended, we propose a novel framework for improving aggregate recommendation diversity and generating recommendation explanations, which supports a wide variety of models for improving recommendation accuracy. The model in our framework is simple yet effective, which can be trained in an end-to-end manner and deployed as a recommendation service easily. Furthermore, our framework can also apply to other generic recommender systems for improving diversity and generating explanations. Experiments on public and private datasets demonstrate the effectiveness of our framework and solution.
LiteGT: Efficient and Lightweight Graph Transformers

Transformers have shown great potential for modeling long-term dependencies for natural language processing and computer vision. However, little study has applied transformers to graphs, which is challenging due to the poor scalability of the attention mechanism and the under-exploration of graph inductive bias. To bridge this gap, we propose a Lite Graph Transformer (LiteGT) that learns on arbitrary graphs efficiently. First, a node sampling strategy is proposed to sparsify the considered nodes in self-attention with only O (Nlog N) time. Second, we devise two kernelization approaches to form two-branch attention blocks, which not only leverage graph-specific topology information, but also reduce computation further to O (1 over 2 Nlog N). Third, the nodes are updated with different attention schemes during training, thus largely mitigating over-smoothing problems when the model layers deepen. Extensive experiments demonstrate that LiteGT achieves competitive performance on both node classification and link prediction on datasets with millions of nodes. Specifically, Jaccard + Sampling + Dim. reducing setting reduces more than 100x computation and halves the model size without performance degradation.
Mining Bursty Groups from Interaction Data

Empirical studies and theoretical models both highlight burstinessas a common temporal pattern in online behavior. A key driver for burstiness is the self-exciting nature of online interactions. For example, posts in online groups often incite posts in response. Such temporal dependencies are easily lost when interaction data is aggregated in snapshots which are subsequently analyzed independently. An alternative is to model individual interactions as a multi-dimensional self-exciting process, thus, enforcing both temporal and network dependencies. Point processes, however, are challenging to employ for large real-world datasets as fitting them incurs super-linear cost in the number of events. How can we efficiently detect online groups exhibiting bursty self-exciting temporal behavior in large real-world datasets? We propose a bursty group detection framework, called MYRON, which explicitly models self-exciting behavior within groups while also accounting for network-wide baseline activity. MYRON imposes bursty temporal structure within a scalable tensor factorization framework to decouple within-group interactions as interpretable factors. Our framework can incorporate different "shapes"of temporal burstiness via wavelet decomposition or kernels forself-exciting behavior. Our evaluation on both synthetic and real-world data demonstrates MYRON's utility in community detection.It is up to 40% more effective in detecting ground truth groups compared to state-of-the-art baselines. In addition, MYRON is able to uncover interpretable bursty patterns of behavior from user-photo interactions in Flickr.
AdaSim: A Recursive Similarity Measure in Graphs

In the literature, various link-based similarity measures such as Adamic/Adar (in short Ada), SimRank, and random walk with restart (RWR) have been proposed. Contrary to SimRank and RWR, Ada is a non-recursive measure, which exploits the local graph structure in similarity computation. Motivated by Ada's promising results in various graph-related tasks, along with the fact that SimRank is a recursive generalization of the co -citation measure, in this paper, we propose AdaSim, a recursive similarity measure based on the Ada philosophy. Our AdaSim provides identical accuracy to that of Ada on the first iteration and it is applicable to both directed and undirected graphs. To accelerate our iterative form, we also propose a matrix form that is dramatically faster while providing the exact AdaSim scores. We conduct extensive experiments with five real-world datasets to evaluate both the effectiveness and efficiency of our AdaSim in comparison with those of existing similarity measures and graph embedding methods in the task of similarity computation of nodes. Our experimental results show that 1) AdaSim significantly improves the effectiveness of Ada and outperforms other competitors, 2) its efficiency is comparable to that of SimRank* while being better than the others, 3) AdaSim is not sensitive to the parameter tuning, and 4) similarity measures are better than embedding methods to compute similarity of nodes.
Knowledge Graph Representation Learning as Groupoid: Unifying TransE, RotatE, QuatE, ComplEx

Knowledge graph (KG) representation learning which aims to encode entities and relations into low-dimensional spaces, has been widely used in KG completion and link prediction. Although existing KG representation learning models have shown promising performance, the theoretical mechanism behind existing models is much less well-understood. It is challenging to accurately portray the internal connections between models and build a competitive model systematically. To overcome this problem, a unified KG representation learning framework, called GrpKG, is proposed in this paper to model the KG representation learning from a generic groupoid perspective. We discover that many existing models are essentially the same in the sense of groupoid isomorphism and further provide transformation methods between different models. Moreover, we explore the applications of GrpKG in the model classification as well as other processes. The experiments on several benchmark data sets validate the effectiveness and superiority of our framework by comparing two proposed models (GrpQ8 and GrpM2) with the state-of-the-art models.
ARGH!: Automated Rumor Generation Hub

It is still challenging to effectively identify rumors due to rapid changes in people's interests and perceptions. To enhance rumor detectors, we first need to better understand which rumors are effective (in terms of bypassing detection) and their characteristics. In this paper, we introduce ARGH, a novel framework to automatically generate rumors using recent advancements in natural language processing, customized to target and generate specific topics. To show the effectiveness of ARGH, we conducted a user study with 212 participants and analyzed how well humans can detect the rumors generated by ARGH, and we also tested its performance against the state-of-the-art rumor detection model PLAN [17]. Surprisingly, the experimental results demonstrate that the generated rumors are significantly harder to identify as rumors than hand-written rumors, degrading the detection accuracy by both humans and machines by 18.87% and 17.62%, respectively. We believe that ARGH will be a useful tool to obtain high quality and evasive rumor datasets quickly, which is often a tedious and time consuming task. Further, our analysis results provide valuable insight into how to characterize evasive rumors and how they can be generated, which will help to enhance the existing rumor detection techniques.
Mining Software Entities in Scientific Literature: Document-level NER for an Extremely Imbalance and Large-scale Task

We present a comprehensive information extraction system dedicated to software entities in scientific literature. This task combines the complexity of automatic reading of scientific documents (PDF processing, document structuring, styled/rich text, scaling) with challenges specific to mining software entities: high heterogeneity and extreme sparsity of mentions, document-level cross-references, disambiguation of noisy software mentions and poor portability of Machine Learning approaches between highly specialized domains. While NER is a key component to recognize new and unseen software, considering this task as a simple NER application fails to address most of these issues. In this paper, we propose a multi-model Machine Learning approach where raw documents are ingested by a cascade of document structuring processes applied not to text, but to layout token elements. The cascading process further enriches the relevant structures of the document with a Deep Learning software mention recognizer adapted to the high sparsity of mentions. The Machine Learning cascade culminates with entity disambiguation to alleviate false positives and to provide software entity linking. A bibliographical reference resolution is integrated to the process for attaching references cited alongside the software mentions. Based on the first gold-standard annotated dataset developed for software mentions, this work establishes a new reference end-to-end performance for this task. Experiments with the CORD-19 publications have further demonstrated that our system provides practically usable performance and is scalable to the whole scientific corpus, enabling novel applications for crediting research software and for better understanding the impact of software in science.