Miracle-dz/UniFAR-FTUs huggingface dataset:
CauseBox: A Causal Inference Toolbox for BenchmarkingTreatment Effect Estimators with Machine Learning Methods

Causal inference is a critical task in various fields such as healthcare, economics, marketing and education. Recently, there have been significant advances through the application of machine learning techniques, especially deep neural networks. Unfortunately, to-date many of the proposed methods are evaluated on different (data, software/hardware, hyperparameter) setups and consequently it is nearly impossible to compare the efficacy of the available methods or reproduce results presented in original research manuscripts. In this paper, we propose a causal inference toolbox (CauseBox) that addresses the aforementioned problems. At the time of publication, the toolbox includes seven state of the art causal inference methods and two benchmark datasets. By providing convenient command-line and GUI-based interfaces, the CauseBox toolbox helps researchers fairly compare the state of the art methods in their chosen application context against benchmark datasets. The code is made public at github.com/paras2612/CauseBox.
iMap: Incremental Node Mapping between Large Graphs Using GNN

Node mapping between large graphs (or network alignment) plays a key preprocessing role in joint-graph data mining applications like social link prediction, cross-platform recommendation, etc. Most existing approaches attempt to perform alignment at the granularity of entire graphs, while handling the whole graphs may lower the scalability and the noisy nodes/edges in the graphs may impact the effectiveness. From the observation that potential node mappings always appear near known corresponding nodes, we propose iMAP, a novel sub-graph expansion based alignment framework to incrementally construct meaningful sub-graphs and perform alignment on each sub-graph pair iteratively, which reduces the unnecessary computation cost in the original raw networks and improves effectiveness via excluding possible noises. Specifically, iMap builds a candidate sub-graph around known matched nodes initially. In each following iteration, iMap trains an alignment model to infer the node mapping relationship between sub-graphs, from which the sub-graphs are further extended and refined. In addition, we design a Graph Neural Network(GNN) based model named MAP on each sub-graph pair in the iMap framework. MAP utilizes trainable Multi-layer Perception (MLP) prediction heads for similarity computation and employs a mixed loss function consisting of the ranking loss for contrastive learning and the cross-entropy loss for classification. Extensive experiments conducted on real social networks demonstrate superior efficiency and effectiveness (above 12% improvement) of our proposed method compared to several state-of-the-art methods.
Discovery of Temporal Graph Functional Dependencies

Temporal Graph Functional Dependencies (TGFDs) are a class of data quality rules imposing topological, attribute dependency constraints over a period of time. To make TGFDs usable in practice, we study the TGFD discovery problem, and show the satisfiability, implication, and validation problems for k-bounded TGFDs are in PTIME. We introduce the TGFDMiner algorithm, which discovers minimal, frequent TGFDs. Our evaluation shows the efficiency and effectiveness of TGFDMiner, and the utility of TGFDs.
Aggregation Techniques in Crowdsourcing: Multiple Choice Questions and Beyond

Crowdsourcing has been leveraged in various tasks and applications, primarily to gather information from human annotators in exchange for a monetary reward. The main challenge associated with crowdsourcing is the low quality of the results, which can stem from multiple reasons, including bias, error, and adversarial behavior. Researchers and practitioners can apply quality control methods to prevent and detect low-quality responses. For example, worker selection methods utilize qualifications and attention check questions before assigning a task. Similarly, task routing identifies the workers who can provide a more accurate response to a given task type using recommender system techniques. In practice, posterior quality control methods are the most common approach to deal with noisy labels once they are obtained. Such methods require task repetition, i.e., assigning the task to multiple crowd-workers, followed by an aggregation mechanism (aka truth inference) to select the most likely answer or request an additional label. A large number of techniques have been proposed for crowdsourcing aggregation covering several types of task types. This tutorial aims to present common and recent label aggregation techniques for multiple-choice questions, multi-class labels, ratings, pairwise comparison, and image/text annotation. We believe that the audience will benefit from the focus on this specific research area to learn about the best techniques to apply in their crowdsourcing projects.
RW-Team: Robust Team Formation using Random Walk

There is a growing need to find meaningful teams in expert networks such as DBLP and GitHub. However, existing team formation methods, such as those based on shortest paths between experts, may generate weakly-connected teams. We demonstrate RW-Team, a robust team formation framework based on a random walk with restart (RWR). We introduce a greedy algorithm to reduce the search space, and we use a Monte Carlo approximation of RWR to improve performance. To handle large graphs, we implement RW-Team in Apache Spark. The proposed demonstration will allow participants to form teams of researchers having various skill sets and explore connections among team members using several graph visualization techniques.
Principled Multi-Aspect Evaluation Measures of Rankings

Information Retrieval evaluation has traditionally focused on defining principled ways of assessing the relevance of a ranked list of documents with respect to a query. Several methods extend this type of evaluation beyond relevance, making it possible to evaluate different aspects of a document ranking (e.g., relevance, usefulness, or credibility) using a single measure (multi-aspect evaluation). However, these methods either are (i) tailor-made for specific aspects and do not extend to other types or numbers of aspects, or (ii) have theoretical anomalies, e.g. assign maximum score to a ranking where all documents are labelled with the lowest grade with respect to all aspects (e.g., not relevant, not credible, etc.). We present a theoretically principled multi-aspect evaluation method that can be used for any number, and any type, of aspects. A thorough empirical evaluation using up to 5 aspects and a total of 425 runs officially submitted to 10 TREC tracks shows that our method is more discriminative than the state-of-the-art and overcomes theoretical limitations of the state-of-the-art.
Understanding Multi-channel Customer Behavior in Retail

Online shopping is gaining popularity. Traditional retailers with physical stores adjust to this trend by allowing their customers to shop online as well as offline, in-store. Increasingly, customers can browse and purchase products across multiple shopping channels. Understanding how customer behavior relates to the availability of multiple shopping channels is an important prerequisite for many downstream machine learning tasks, such as recommendation and purchase prediction. However, previous work in this domain is limited to analyzing single-channel behavior only. In this paper, we provide the first insights into multi-channel customer behavior in retail based on a large sample of 2.8 million transactions originating from 300,000 customers of a food retailer in Europe. Our analysis reveals significant differences in customer behavior across online and offline channels, for example with respect to the repeat ratio of item purchases and basket size. Based on these findings, we investigate the performance of a next basket recommendation model under multi-channel settings. We find that the recommendation performance differs significantly for customers based on their choice of shopping channel, which strongly indicates that future research on recommenders in this area should take into account the particular characteristics of multi-channel retail shopping.
CIExplore: Curiosity and Influence-based Exploration in Multi-Agent Cooperative Scenarios with Sparse Rewards

Learning in a sparse-reward setting is a well-known challenge in RL (Reinforcement Learning). In the single-agent domain, this challenge can be addressed by introducing exploration bonuses driven by intrinsic motivation to encourage agents to visit unseen states. However, naively applying these methods in MARL (Multi-Agent Reinforcement Learning) cooperative settings with sparse rewards results in some inevitable problems: misunderstanding environmental knowledge and lack of collaboration among agents, etc. Based on this, in this paper, we propose the Curiosity and Influence-based Explore (CIExplore) method, which includes a new form of intrinsic reward and an internal counterfactual advantage function. Concretely, the intrinsic reward is a combination of joint curiosity reward and influence reward. The former is the variance of outputs across an ensemble of prediction models that take joint observations and actions of all agents as inputs to predict the next time's joint observations. And the latter quantifies the influence of one agent's behavior on other agents' state-value functions. Given that the joint curiosity reward is shared by all agents, we compute an internal counterfactual advantage function to address this intrinsic reward assignment problem. We demonstrate the efficacy of CIExplore in the multi-agent grid-world environments and show that it is compatible with both on-policy and off-policy MARL algorithms and be scalable to complex settings where agents' number or environment randomness increases.
Stock Trend Prediction with Multi-granularity Data: A Contrastive Learning Approach with Adaptive Fusion

Stock trend prediction plays a crucial role in quantitative investing. Given the prediction task on a certain granularity (e.g., daily trend), a large portion of existing studies merely leverage market data of the same granularity (e.g., daily market data). In financial investment scenarios, however, there exist amounts of finer-grained information (e.g., high-frequency data) that contain more detailed investment signals beyond the original granularity data. This motivates us to investigate how to leverage multi-granularity market data to enhance the accuracy of stock trend prediction. Some straightforward methods, such as concatenating finer-grained data as features or fusing with a model based on finer-grained features, may not lead to more precise stock trend prediction due to some unique challenges. First, the inconsistency of granularity between the target trend and finer-grained data could substantially increase optimization difficulty, such as the relative sparsity of the target trend compared with higher dimensions of finer-grained features. Moreover, the continuously changing financial market state could result in varying efficacy of heterogeneous multi-granularity information, which consequently requires a dynamic approach for proper fusion among them. In this paper, we propose the Contrastive Multi-Granularity Learning Framework (CMLF) to address these challenges. Particularly, we first design two novel contrastive learning objectives at the pre-training stage to address the inconsistency issue by constructing additional self-supervised signals relying on the inherent character of stock data. We also design a gate mechanism based on market-aware technical indicators to fuse the multi-granularity features at each time step adaptively. Extensive experiments on three real-world datasets show significant improvements of our approach over the state-of-the-art baselines on stock trend prediction and profitability in real investing scenarios.
Elastic and Stable Compaction for LSM-tree: A FaaS-Based Approach on TerarkDB

LSM-tree is widely used as a write-optimized storage engine in many NoSQL systems. However, the periodical compaction operations in LSM-tree cost many I/O bandwidths and CPU resources of the local server, resulting in throughput drops of the system. To address this issue, this paper proposes a new compaction scheme based on the FaaS (Functions as a Service) architecture, which is called FaaS Compaction. It utilizes the elastic computing capability of FaaS and always pushes compactions to a FaaS cluster. The FaaS cluster will perform actual compaction operations, which will not affect the processing of the local server. Therefore, we can maintain stable performance even when periodical compactions are triggered. We also present a Parallel Slight Compaction method to solve the timeout problem caused by heavy compactions. We implement the FaaS Compaction based on TerarkDB and a real FaaS cluster and experimentally compare the FaaS Compaction with the RocksDB's local compaction scheme and the state-of-the-art offloading compaction policy. The results suggest the efficiency, stability, and elasticity of our proposal.
Reputation Equity in Ranking Systems

The impact of ranking systems on humans is an aspect that is getting a lot of attention. In this paper, we consider a class of algorithms, known as reputation-based ranking systems, which rank the items based on a reputation score automatically computed for each user. Recent literature introduced the concept of reputation independence, which considers a sensitive attribute of the users (such as gender or age) and makes the reputation scores independent from that attribute. Here, we show that if we consider a different sensitive attribute w.r. t. a user to introduce independence, reputation scores are still biased. To overcome this issue, we propose an approach to attain equity in the reputation scores computation, independently of any sensitive attribute that characterizes the users.
From Pixels to Words: A Scalable Journey of Text Information from Product Images to Retail Catalog

Extracting texts of various shapes, sizes, and orientations from images containing multiple objects is an important problem in many contexts, especially, in connection to E-commerce. In the context of the scale at which Walmart operates, the text from an image can be a richer and more accurate source of data than human inputs and can be used in several applications such as Attribute Extraction, Offensive Text Classification, Product Matching among others. The motivation of this particular work has come from different business requirements such as flagging products whose images contain words that are non-compliant with organizational policies and building an efficient automated system to identify similar products by comparing the information contained in their respective product images and many others. Existing methods fail to address domain specific challenges like high entropy, different orientations, and small texts in product images adequately. In this work, we provide a solution that not only addresses these challenges but is also proven to work at a million image scale for various retail business units within Walmart. Extensive experimentation revealed that our proposed solution has been able to save around 30% computational cost in both the training and the inference stages.
Learning to Bundle Proactively for On-Demand Meal Delivery

On-demand meal delivery (ODMD) platforms such as DoorDash and Ele.me have experienced explosive growth in recent years. Effective logistics optimization strategies that could guarantee high service standards with controlled costs are crucial for the long-term sustainability of these platforms, and yet are also non-trivial due to the nature of ODMD operations. In particular, most of the orders are not known until they are placed by the customers, and any dispatching policy that only considers known requests would risk making myopic decisions in such a setting. In this paper, we propose a novel approach to address this problem. At the core of our method is a learning-based metric called Proactive Bundle Cost Vector (PBCV), which quantifies the easiness of bundling a particular order with future orders. Based on PBCV, we build a proactive bundling policy that that considers the viability of serving unknown requests. Extensive online A/B tests demonstrate that the resultant policy has shown significant improvements of key performance metrics over baseline policies. Our solution has been successfully deployed at one of the world's largest ODMD platforms, serving tens of millions of customers on a daily basis.
Hierarchical Semantics Matching For Heterogeneous Spatio-temporal Sources

Spatio-temporal data are semantically valuable information used for various analytical tasks to identify spatially relevant and temporally limited correlations within a domain. The increasing availability and data acquisition from multiple sources with their typically high heterogeneity are getting more and more attention. However, these sources often lack interconnecting shared keys, making their integration a challenging problem. For example, publicly available parking data that consist of point data on parking facilities with fluctuating occupancy and static location data on parking spaces cannot be directly correlated. Both data sets describe two different aspects from distinct sources in which parking spaces and fluctuating occupancy are part of the same semantic model object. Especially for ad hoc analytical tasks on integrated models, these missing relationships cannot be handled using join operations as usual in relational databases. The reason lies in the lack of equijoin relationships, comparing for equality of strings and additional overhead in loading data up before processing. This paper addresses the optimization problem of finding suitable partners in the absence of equijoin relations for heterogeneous spatio-temporal data, applicable to ad hoc analytics. We propose a graph-based approach that achieves good recall and performance scaling via hierarchically separating the semantics along spatial, temporal, and domain-specific dimensions. We evaluate our approach using public data, showing that it is suitable for many standard join scenarios and highlighting its limitations.
DistRDF2ML - Scalable Distributed In-Memory Machine Learning Pipelines for RDF Knowledge Graphs

This paper presents DistRDF2ML, the generic, scalable, and distributed framework for creating in-memory data preprocessing pipelines for Spark-based machine learning on RDF knowledge graphs. This framework introduces software modules that transform large-scale RDF data into ML-ready fixed-length numeric feature vectors. The developed modules are optimized to the multi-modal nature of knowledge graphs. DistRDF2ML provides aligned software design and usage principles as common data science stacks that offer an easy-to-use package for creating machine learning pipelines. The modules used in the pipeline, the hyper-parameters and the results are exported as a semantic structure that can be used to enrich the original knowledge graph. The semantic representation of metadata and machine learning results offers the advantage of increasing the machine learning pipelines' reusability, explainability, and reproducibility. The entire framework of DistRDF2ML is open source, integrated into the holistic SANSA stack, documented in scala-docs, and covered by unit tests. DistRDF2ML demonstrates its scalable design across different processing power configurations and (hyper-)parameter setups within various experiments. The framework brings the three worlds of knowledge graph engineers, distributed computation developers, and data scientists closer together and offers all of them the creation of explainable ML pipelines using a few lines of code.
Learning to Expand: Reinforced Response Expansion for Information-seeking Conversations

Information-seeking conversation systems are increasingly popular in real-world applications, especially for e-commerce companies. To retrieve appropriate responses for users, it is necessary to compute the matching degrees between candidate responses and users' queries with historical dialogue utterances. As the contexts are usually much longer than responses, it is thus necessary to expand the responses (usually short) with richer information. Recent studies on pseudo-relevance feedback (PRF) have demonstrated its effectiveness in query expansion for search engines, hence we consider expanding response using PRF information. However, existing PRF approaches are either based on heuristic rules or require heavy manual labeling, which are not suitable for solving our task. To alleviate this problem, we treat the PRF selection for response expansion as a learning task and propose a reinforced learning method that can be trained in an end-to-end manner without any human annotations. More specifically, we propose a reinforced selector to extract useful PRF terms to enhance response candidates and a BERT-based response ranker to rank the PRF-enhanced responses. The performance of the ranker serves as a reward to guide the selector to extract useful PRF terms, which boosts the overall task performance. Extensive experiments on both standard benchmarks and commercial datasets prove the superiority of our reinforced PRF term selector compared with other potential soft or hard selection methods. Both case studies and quantitative analysis show that our model is capable of selecting meaningful PRF terms to expand response candidates and also achieving the best results compared with all baselines on a variety of evaluation metrics. We have also deployed our method on online production in an e-commerce company, which shows a significant improvement over the existing online ranking system.
Sequential Search with Off-Policy Reinforcement Learning

Recent years have seen a significant amount of interests in Sequential Recommendation (SR), which aims to understand and model the sequential user behaviors and the interactions between users and items over time. Surprisingly, despite the huge success Sequential Recommendation has achieved, there is little study on Sequential Search (SS), a twin learning task that takes into account a user's current and past search queries, in addition to behavior on historical query sessions. The SS learning task is even more important than the counterpart SR task for most of E-commence companies due to its much larger online serving demands as well as traffic volume. To this end, we propose a highly scalable hybrid learning model that consists of an RNN learning framework leveraging all features in short-term user-item interactions, and an attention model utilizing selected item-only features from long-term interactions. As a novel optimization step, we fit multiple short user sequences in a single RNN pass within a training batch, by solving a greedy knap-sack problem on the fly. Moreover, we explore the use of off-policy reinforcement learning in multi-session personalized search ranking. Specifically, we design a pairwise Deep Deterministic Policy Gradient model that efficiently captures users' long term reward in terms of pairwise classification error. Extensive ablation experiments demonstrate significant improvement each component brings to its state-of-the-art baseline, on a variety of offline and online metrics.