Reviewerly/exHarmony huggingface dataset:
Reasoning with Recommender Systems? Practical Reasoning, Digital Nudging, and Autonomy

Abstract One of the core tasks of recommender systems is often defined as follows: Find good items. Recommender systems are thus designed to support our decision-making by helping us find our way around the online world and guiding us to the things we want. However, relying on recommender systems has a profound effect on our decision-making because they structure the environment in which we make choices. In this contribution, I examine how recommender systems affect our practical reasoning and whether they pose a threat to autonomy, i.e., what influence recommender systems have on our capacity for making our own choices. I argue that a basic requirement for integrating automated recommendations in autonomous decision-making consists in being able to identify the rationale behind recommendations: only if we understand why we are being presented with certain recommendations is it possible for them to be integrated into decision-making in a way that preserves autonomy.
A Methodological Framework for SPL Engineering from DSML

For the last ten years, Software Product Line (SPL) tool developers have been facing the implementation of different variability requirements and the support of SPL engineering activities demanded by emergent domains. Despite several tools exist, few works resolve SPL process for both problem and solution space. Due to these reasons, we propose a methodological framework that overcomes the limits of existing tools and holds all the phases and activities from the requirement design till the product derivation. We start by using a Domain Specific Modelling Language (DSML) for domain description, which allows system designers working closer to the system domain as they can manipulate real concepts. Thereafter, an intermediate phase converts the DSML metamodel to a tree-structured representation similar to Feature Model (FM) notation enriched with extra-information such that cardinality, attributes, constraints, documentation, etc. The objective of this FM is to be used later as a decision tree to guide the generative process of our software factory in the following way: First, the engineer annotates the variation points with variability types such that binding time, granularity, evolution, etc, which are crucial concerns to be considered when generating the products. Second, based upon these annotations, our framework determines the possible useful variability mechanisms that could be employed to implement the product families and the engineers choose the variability programming tactic among them. Finally, the software factory produces the guidelines to implement the realization strategy and derive the related product assets through an assembly process. We provide a real industry running example giving insight into the application of the presented approach.
Different Perspectives on EIO

The Chapter is dedicated to give an overview of the representatives of the legal domain (Lawyers, EJTN Representatives) and the technical community (Digital Forensics and IT Experts) who are involved in the implementation of the EIO, in order to discuss with them the improvement and application of EIO across MSs and better understanding in practise the potential barriers and gaps existing in the different European countries. A survey has been conducted among different stakeholders involved in the EIO procedure: a tailor-made questionnaire has been circulated in order to catch the real situation going on in the different contexts and MSs.
Assessment of Gender-related Differential Item Functioning of Teacher-Made Chemistry test

A good item that will measure the intended domain is expected to be free of biases. But several studies have confirmed that some items in a test reveal biases due to a group of testees.. A generally acceptable analytical technique that can be used to discover biases in test items is the Differential Item Functioning (DIF) which Item Response Theory (IRT) offers to check differences in psychometric properties due to the groups that testees belong. Thus, this study used the DIF technique to detect gender biased items in a teacher made Chemistry test. BILOG-MG was employed using 350 (183 males and 167 females) students from 10 Senior secondary school Two (SSII), randomly drawn from Obio/Akpor Local Government Area of Port Harcourt, River State, Nigeria. The study showed that out of one hundred items, fifty-three items were biased. However, 26(49.1%) out of 53 were in favour of the female while 27(50.9%) were in favour of the male which confirmed biases. DIF is effective in detecting group biases of test items. The study concluded that Differential Item Functioning should always be used by scale developers before collating the final items for a test.
NFT-Based Data Marketplace with Digital Watermarking

In today's digital world, enterprises and individuals are generating massive data that is potentially useful for many data consumers with data driven applications. The emergence of data marketplaces is a step toward helping the data owners to monetize their digital assets and get connected to the potential buyers. The current data marketplaces cannot handle the challenges related to data ownership claims, illegal redistribution, and data ownership traceability. To overcome these problems in a general-purpose market, we propose a marketplace based on watermarking and Non-Fungible Token (NFT) technologies. In the proposed NFT-based marketplace, the owner's data is stored as an NFT where the underlying content of the NFT holds the watermarked data. The watermarked data is obtained by embedding some information about the owners and the buyers into the original data. The embedded information can later be extracted to identify the owner and the buyer of the traded data. Furthermore, the transactions corresponding to the NFT provide verifiable ownership proof and traceable ownership history. A Proof-Of-Concept (POC) implementation of the proposed marketplace that will be integrated within AI-Gallery Data Marketplace service in Huawei Cloud is presented for trading image data. An extensive set of experiments to measure the gas consumption on the blockchain and evaluate the robustness of the watermarked assets against 51 attacks are performed. Finally, a method based on error correction codes is proposed for improving the watermarking robustness in the implemented marketplace. The link for the codes and the POC demo is provided in the appendix.
UCEpic: Unifying Aspect Planning and Lexical Constraints for Generating Explanations in Recommendation

Personalized natural language generation for explainable recommendations plays a key role in justifying why a recommendation might match a user's interests. Existing models usually control the generation process by aspect planning. While promising, these aspect-planning methods struggle to generate specific information correctly, which prevents generated explanations from being convincing. In this paper, we claim that introducing lexical constraints can alleviate the above issues. We propose a model, UCEpic, that generates high-quality personalized explanations for recommendation results by unifying aspect planning and lexical constraints in an insertion-based generation manner.
IPOC: An Adaptive Interval Prediction Model based on Online Chasing and Conformal Inference for Large-Scale Systems

In large-scale systems, due to system complexity and demand volatility, diverse and dynamic workloads make accurate predictions difficult. In this work, we address an online interval prediction problem (OnPred-Int) and adopt ensemble learning to solve it. We depict that the ensemble learning for OnPred-Int is a dynamic deterministic Markov Decision Process (Dd-MDP) and convert it into a stateful online learning task. Then we propose IPOC, a lightweight and flexible model able to produce effective confidence intervals, adapting the dynamics of real-time workload streams. At each time, IPOC selects a target model and executes chasing for it by a designed chasing oracle, during which process IPOC produces accurate confidence intervals. The effectiveness of IPOCis theoretically validated through sublinear regret analysis and satisfaction of confidence interval requirements. Besides, we conduct extensive experiments on 4 real-world datasets comparing with 19 baselines. To the best of our knowledge, we are the first to apply the frontier theory of online learning to time series prediction tasks.
Meta Multi-agent Exercise Recommendation: A Game Application Perspective

Exercise recommendation is a fundamental and important task in the E-learning system, facilitating students' personalized learning. Most existing exercise recommendation algorithms design a scoring criterion (e.g., weakest mastery, lowest historical correctness) in conjunction with experience, and then recommend the recommended knowledge concepts (KCs). These algorithms rely entirely on the scoring criteria by treating exercise recommendations as a centralized system. However, it is a complex problem for the centralized system to choose a limited number of exercises in a period of time to consolidate and learn the KCs efficiently. Moreover, different groups of students (e.g., different countries, schools, or classes) have different solutions for the same group of KCs according to their own situations, in the spirit of competency-based instructing. Therefore, we propose Meta Multi-Agent Exercise Recommendation (MMER). Specifically, we design the multi-agent exercise recommendation module, in which the KCs involved in exercises are considered agents with competition and cooperation among them. And the meta-training stage is designed to learn a robust recommendation module for new student groups. Extensive experiments on real-world datasets validate the satisfactory performance of the proposed model. Furthermore, the effectiveness of the multi-agent and meta-training part is demonstrated for the model in recommendation applications.
The KDD'23 Workshop on Causal Discovery, Prediction and Decision (CDPD 2023)

Causal relationships have been utilized in almost all disciplines, and the research into causal discovery has attracted a lot of attention in the last few years. Traditionally, causal relationships are identified by making use of interventions or randomized controlled experiments. However, conducting such experiments is often expensive or even impossible due to cost or ethical concerns. Therefore, there has been an increasing interest in discovering causal relationships based on observational data, and in the past few decades, significant contributions have been made to this field by computer scientists.
Machine Unlearning in Gradient Boosting Decision Trees

Various machine learning applications take users' data to train the models. Recently enforced legislation requires companies to remove users' data upon requests, i.e.,the right to be forgotten. In the context of machine learning, the trained model potentially memorizes the training data. Machine learning algorithms have to be able to unlearn the user data that are requested to delete to meet the requirement. Gradient Boosting Decision Trees (GBDT) is a widely deployed model in many machine learning applications. However, few studies investigate the unlearning on GBDT. This paper proposes a novel unlearning framework for GBDT. To the best of our knowledge, this is the first work that considers machine unlearning on GBDT. It is not straightforward to transfer the unlearning methods of DNN to GBDT settings. We formalized the machine unlearning problem and its relaxed version. We propose an unlearning framework that efficiently and effectively unlearns a given collection of data without retraining the model from scratch. We introduce a collection of techniques, including random split point selection and random partitioning layers training, to the training process of the original tree models to ensure that the trained model requires few subtree retrainings during the unlearning. We investigate the intermediate data and statistics to store as an auxiliary data structure during the training so that we can immediately determine if a subtree is required to be retrained without touching the original training dataset. Furthermore, a lazy update technique is proposed as a trade-off between unlearning time and model functionality. We experimentally evaluate our proposed methods on public datasets. The empirical results confirm the effectiveness of our framework.
MimoSketch: A Framework to Mine Item Frequency on Multiple Nodes with Sketches

We abstract a MIMO scenario in distributed data stream mining, where a stream of multiple items is mined by multiple nodes. We design a framework named MimoSketch for the MIMO-specific scenario, which improves the fundamental mining task of item frequency estimation. MimoSketch consists of an algorithm design and a policy to schedule items to nodes. MimoSketch's algorithm applies random counting to preserve a mathematically proven unbiasedness property, which makes it friendly to the aggregate query on multiple nodes; its memory layout is dynamically adaptive to the runtime item size distribution, which maximizes the estimation accuracy by storing more items. MimoSketch's scheduling policy balances items among nodes, avoiding nodes being overloaded or underloaded, which improves the overall mining accuracy. Our prototype and evaluation show that our algorithm can improve the item frequency estimation accuracy by an order of magnitude compared with the state-of-the-art solutions, and the scheduling policy further promotes the performance in MIMO scenarios.
The Missing Indicator Method: From Low to High Dimensions

Missing data is common in applied data science, particularly for tabular data sets found in healthcare, social sciences, and natural sciences. Most supervised learning methods only work on complete data, thus requiring preprocessing such as missing value imputation to work on incomplete data sets. However, imputation alone does not encode useful information about the missing values themselves. For data sets with informative missing patterns, the Missing Indicator Method (MIM), which adds indicator variables to indicate the missing pattern, can be used in conjunction with imputation to improve model performance. While commonly used in data science, MIM is surprisingly understudied from an empirical and especially theoretical perspective. In this paper, we show empirically and theoretically that MIM improves performance for informative missing values, and we prove that MIM does not hurt linear models asymptotically for uninformative missing values. Additionally, we find that for high-dimensional data sets with many uninformative indicators, MIM can induce model overfitting and thus test performance. To address this issue, we introduce Selective MIM (SMIM), a novel MIM extension that adds missing indicators only for features that have informative missing patterns. We show empirically that SMIM performs at least as well as MIM in general, and improves MIM for high-dimensional data. Lastly, to demonstrate the utility of MIM on real-world data science tasks, we demonstrate the effectiveness of MIM and SMIM on clinical tasks generated from the MIMIC-III database of electronic health records.
Complementary Classifier Induced Partial Label Learning

In partial label learning (PLL), each training sample is associated with a set of candidate labels, among which only one is valid. The core of PLL is to disambiguate the candidate labels to get the ground-truth one. In disambiguation, the existing works usually do not fully investigate the effectiveness of the non-candidate label set (a.k.a. complementary labels), which accurately indicates a set of labels that do not belong to a sample. In this paper, we use the non-candidate labels to induce a complementary classifier, which naturally forms an adversarial relationship against the traditional PLL classifier, to eliminate the false-positive labels in the candidate label set. Besides, we assume the feature space and the label space share the same local topological structure captured by a dynamic graph, and use it to assist disambiguation. Extensive experimental results validate the superiority of the proposed approach against state-of-the-art PLL methods on 4 controlled UCI data sets and 6 real-world data sets and reveal the usefulness of complementary learning in PLL. The code has been released in the link https://github.com/Chongjie-Si/PL-CL
Specify Robust Causal Representation from Mixed Observations

Learning representations purely from observations concerns the problem of learning a low-dimensional, compact representation which is beneficial to prediction models. Under the hypothesis that the intrinsic latent factors follow some casual generative models, we argue that by learning a causal representation, which is the minimal sufficient causes of the whole system, we can improve the robustness and generalization performance of machine learning models. In this paper, we develop a learning method to learn such representation from observational data by regularizing the learning procedure with mutual information measures, according to the hypothetical factored causal graph. We theoretically and empirically show that the models trained with the learned causal representations are more robust under adversarial attacks and distribution shifts compared with baselines.
Dependence and Model Selection in LLP: The Problem of Variants

The problem of Learning from Label Proportions (LLP) has received considerable research attention and has numerous practical applications. In LLP, a hypothesis assigning labels to items is learned using knowledge of only the proportion of labels found in predefined groups, called bags. While a number of algorithmic approaches to learning in this context have been proposed, very little work has addressed the model selection problem for LLP. Nonetheless, it is not obvious how to extend straightforward model selection approaches to LLP, in part because of the lack of item labels. More fundamentally, we argue that a careful approach to model selection for LLP requires consideration of the dependence structure that exists between bags, items, and labels. In this paper we formalize this structure and show how it affects model selection. We show how this leads to improved methods of model selection that we demonstrate outperform the state of the art over a wide range of datasets and LLP algorithms.
Time-to-Event Modeling with Hypernetwork based Hawkes Process

Many real-world applications are associated with collection of events with timestamps, known as time-to-event data. Earthquake occurrences, social networks, and user activity logs can be represented as a sequence of discrete events observed in continuous time. Temporal point process serves as an essential tool for modeling such time-to-event data in continuous time space. Despite having massive amounts of event sequence data from various domains like social media, healthcare etc., real world application of temporal point process faces two major challenges: 1) it is not generalizable to predict events from unseen event sequences in dynamic environment 2) they are not capable of thriving in continually evolving environment with minimal supervision while retaining previously learnt knowledge. To tackle these issues, we propose HyperHawkes, a hypernetwork based temporal point process framework which is capable of modeling time of event occurrence for unseen sequences and consequently, zero-shot learning for time-to-event modeling. We also develop a hypernetwork based continually learning temporal point process for continuous modeling of time-to-event sequences with minimal forgetting. HyperHawkes augments the temporal point process with zero-shot modeling and continual learning capabilities. We demonstrate the application of the proposed framework through our experiments on real-world datasets. Our results show the efficacy of the proposed approach in terms of predicting future events under zero-shot regime for unseen event sequences. We also show that the proposed model is able to learn the time-to-event sequences continually while retaining information from previous event sequences, mitigating catastrophic forgetting in neural temporal point process.
MedLink: De-Identified Patient Health Record Linkage

A comprehensive patient health history is essential for patient care and healthcare research. However, due to the distributed nature of healthcare services, patient health records are often scattered across multiple systems. Existing record linkage approaches primarily rely on patient identifiers, which have inherent limitations such as privacy invasion and identifier discrepancies. To tackle this problem, we propose linking de-identified patient health records by matching health patterns without strictly relying on sensitive patient identifiers. Our model MedLink solves two challenges faced with the patient linkage task: (1) the challenge of identifying the same patients based on data collected in different timelines as disease progression makes the record matching difficult, and (2) the challenge of identifying distinct health patterns as common medical codes dominate health records and overshadow the more informative low-prevalence codes. To address these challenges, MedLink utilizes bi-directional health prediction to predict future codes forwardly and past codes backwardly, thus accounting for the health progression. MedLink also has a prevalence-aware retrieval design to focus more on the low-prevalence but informative codes during learning. MedLink can be trained end-to-end and is lightweight for efficient inference on large patient databases. We evaluate MedLink against leading baselines on real-world patient datasets, including the critical care dataset MIMIC-III and a large health claims dataset. Results show that MedLink outperforms the best baseline by 4% in top-1 accuracy with only 8% memory cost. Additionally, when combined with existing identifier-based linkage approaches, MedLink can improve their performance by up to 15%.
Trustworthy Machine Learning: Robustness, Generalization, and Interpretability

Machine learning is becoming increasingly important in today's world. Beyond its powerful performances, there has been an emerging concern about the trustworthiness of machine learning, including but not limited to: robustness to malicious attacks, generalization to unseen datasets, and interpretability to explain its outputs. Such concerns are even more urgent in some safety-critical applications such as medical diagnosis and autonomous driving. Trustworthy machine learning (TrustML) aims to tackle these challenges from the perspectives of theory, algorithm, and applications. In this tutorial, we will give a comprehensive introduction to the recent advance of trustworthy machine learning in robustness, generalization, and interpretability. We will cover their problem formulation, related research, popular algorithms, and successful applications. Additionally, we will also introduce some potential challenges for future research. We do hope that this tutorial will not only serve as a platform to understand TrustML, but also raise the awareness of everyone for more trustworthy applications.