Reviewerly/exHarmony huggingface dataset:
ColBERT-FairPRF: Towards Fair Pseudo-Relevance Feedback in Dense Retrieval

Pseudo-relevance feedback mechanisms have been shown to be useful in improving the effectiveness of search systems for retrieving the most relevant items in response to a user's query. However, there has been little work investigating the relationship between pseudo-relevance feedback and fairness in ranking. Indeed, using the feedback from an initial retrieval to revise a query can in principle also allow to optimise objectives beyond relevance, such as the fairness of the search results. In this work, we show how a feedback mechanism based on the successful ColBERT-PRF model can be used for retrieving fairer search results. Therefore, we propose a novel fair feedback mechanism for multiple representation dense retrieval (ColBERT-FairPRF), which enhances the distribution of exposure over groups of documents in the search results by fairly extracting the feedback embeddings that are added to the user's query representation. To fairly extract representative embeddings, we apply a clustering approach since traditional methods based on counting are not applicable in the dense retrieval space. Our results on the 2021 TREC Fair Ranking Track test collection demonstrate the effectiveness of our method compared to ColBERT-PRF, with statistical significant improvements of up to $$\sim $$ 19% in Attention Weighted Ranked Fairness. To the best of our knowledge, ColBERT-FairPRF is the first query expansion method for fairness in multiple representation dense retrieval.
DeCoDE: Detection of Cognitive Distortion and Emotion Cause Extraction in Clinical Conversations

Despite significant evidence linking mental health to almost every major development issue, individuals with mental disorders are among those most at risk of being excluded from development programs. We outline a novel task of detection of Cognitive Distortion and Emotion Cause extraction of associated emotions in conversations. Cognitive distortions are inaccurate thought patterns, beliefs, or perceptions that contribute to negative thinking, which subsequently elevates the chances of several mental illnesses. This work introduces a novel multi-modal mental health conversational corpus manually annotated with emotion, emotion causes, and the presence of cognitive distortion at the utterance level. We propose a multitasking framework that uses multi-modal information as inputs and uses both external commonsense knowledge and factual knowledge from the dataset to learn both tasks at the same time. This is because commonsense knowledge is a key part of understanding how and why emotions are implied. We achieve commendable performance gains on the cognitive distortion detection task (+3.91 F1%) and the emotion cause extraction task (+3 ROS points) when compared to the existing state-of-the-art model.
COILcr: Efficient Semantic Matching in Contextualized Exact Match Retrieval

Lexical exact match systems that use inverted lists are a fundamental text retrieval architecture. A recent advance in neural IR, COIL, extends this approach with contextualized inverted lists from a deep language model backbone and performs retrieval by comparing contextualized query-document term representation, which is effective but computationally expensive. This paper explores the effectiveness-efficiency tradeoff in COIL-style systems, aiming to reduce the computational complexity of retrieval while preserving term semantics. It proposes COILcr, which explicitly factorizes COIL into intra-context term importance weights and cross-context semantic representations. At indexing time, COILcr further maps term semantic representations to a smaller set of canonical representations. Experiments demonstrate that canonical representations can efficiently preserve term semantics, reducing the storage and computational cost of COIL-based retrieval while maintaining model performance. The paper also discusses and compares multiple heuristics for canonical representation selection and looks into its performance in different retrieval settings.
A Mask-Based Logic Rules Dissemination Method for Sentiment Classifiers

Disseminating and incorporating logic rules inspired by domain knowledge in Deep Neural Networks (DNNs) is desirable to make their output causally interpretable, reduce data dependence, and provide some human supervision during training to prevent undesirable outputs. Several methods have been proposed for that purpose but performing end-to-end training while keeping the DNNs informed about logical constraints remains a challenging task. In this paper, we propose a novel method to disseminate logic rules in DNNs for Sentence-level Binary Sentiment Classification. In particular, we couple a Rule-Mask Mechanism with a DNN model which given an input sequence predicts a vector containing binary values corresponding to each token that captures if applicable a linguistically motivated logic rule on the input sequence. We compare our method with a number of state-of-the-art baselines and demonstrate its effectiveness. We also release a new Twitter-based dataset specifically constructed to test logic rule dissemination methods and propose a new heuristic approach to provide automatic high-quality labels for the dataset.
Item Graph Convolution Collaborative Filtering for Inductive Recommendations

Graph Convolutional Networks (GCN) have been recently employed as core component in the construction of recommender system algorithms, interpreting user-item interactions as the edges of a bipartite graph. However, in the absence of side information, the majority of existing models adopt an approach of randomly initialising the user embeddings and optimising them throughout the training process. This strategy makes these algorithms inherently transductive, curtailing their ability to generate predictions for users that were unseen at training time. To address this issue, we propose a convolution-based algorithm, which is inductive from the user perspective, while at the same time, depending only on implicit user-item interaction data. We propose the construction of an item-item graph through a weighted projection of the bipartite interaction network and to employ convolution to inject higher order associations into item embeddings, while constructing user representations as weighted sums of the items with which they have interacted. Despite not training individual embeddings for each user our approach achieves state of-the-art recommendation performance with respect to transductive baselines on four real-world datasets, showing at the same time robust inductive performance.
C$$^2$$LIR: Continual Cross-Lingual Transfer for Low-Resource Information Retrieval

This paper proposes a method to train information retrieval (IR) model for a low-resource language with a small corpus and no parallel sentences. Although neural IR models based on pretrained language models (PLMs) have shown high performance in high-resource languages (HRLs), building PLM for LRLs is challenging. We propose C $$^2$$ LIR, a method to build a high-performing neural IR model for LRL, with dictionary-based pretraining objectives for cross-lingual transfer from HRL. Experiments on the monolingual and cross-lingual IR in diverse low-resource scenarios show the effectiveness and data efficiency of C $$^2$$ LIR.
Privacy-Preserving Fair Item Ranking

Users worldwide access massive amounts of curated data in the form of rankings on a daily basis. The societal impact of this ease of access has been studied and work has been done to propose and enforce various notions of fairness in rankings. Current computational methods for fair item ranking rely on disclosing user data to a centralized server, which gives rise to privacy concerns for the users. This work is the first to advance research at the conjunction of producer (item) fairness and consumer (user) privacy in rankings by exploring the incorporation of privacy-preserving techniques; specifically, differential privacy and secure multi-party computation. Our work extends the equity of amortized attention ranking mechanism to be privacy-preserving, and we evaluate its effects with respect to privacy, fairness, and ranking quality. Our results using real-world datasets show that we are able to effectively preserve the privacy of users and mitigate unfairness of items without making additional sacrifices to the quality of rankings in comparison to the ranking mechanism in the clear.
Dynamic Micro-cluster-Based Streaming Data Clustering Method for Anomaly Detection

The identification of anomalies in a data stream is a difficulty for decision-making in real time. A memory-constrained online detection system that is able to quickly detect the concept drift of streaming data is required because the constant arrival of massive amounts of streaming data with changing characteristics makes real-time and efficient anomaly detection a difficult task. This is because of the nature of the data itself, which is constantly changing. In this study, a novel model for detecting anomalies using dynamic micro-clusters scheme is developed. The macro-clusters are generated from a network of connected micro-clusters. When new data items are added, the normal patterns that are formed in macro-clusters will update in tandem with the dynamic micro-clusters in an incremental fashion. An outlier may be understood from both a global and a local perspective by examining the global and local densities respectively. The effectiveness of the suggested approach was evaluated with the use of three different datasets. The findings of the experiment demonstrate that the suggested method is superior to earlier algorithms in terms of both the accuracy of detection and the level of computing complexity it requires.
FairDistillation: Mitigating Stereotyping in Language Models

Large pre-trained language models are successfully being used in a variety of tasks, across many languages. With this ever-increasing usage, the risk of harmful side effects also rises, for example by reproducing and reinforcing stereotypes. However, detecting and mitigating these harms is difficult to do in general and becomes computationally expensive when tackling multiple languages or when considering different biases. To address this, we present FairDistillation: a cross-lingual method based on knowledge distillation to construct smaller language models while controlling for specific biases. We found that our distillation method does not negatively affect the downstream performance on most tasks and successfully mitigates stereotyping and representational harms. We demonstrate that FairDistillation can create fairer language models at a considerably lower cost than alternative approaches.
MFDG: A Multi-Factor Dialogue Graph Model for Dialogue Intent Classification

Interest in speaker intent classification has been increasing in multi-turn dialogues, as the intention of a speaker is one of the components for dialogue understanding. While most existing methods perform speaker intent classification at utterance-level, the dialogue-level comprehension is ignored. To obtain a full understanding of dialogues, we propose a Multi-Factor Dialogue Graph Model (MFDG) for Dialogue Core Intent (DCI) classification. The model gains an understanding of the entire dialogue by explicitly modeling multi factors that are essential for speaker-specific and contextual information extraction across the dialogue. The main module of MFDG is a heterogeneous graph encoder, where speakers, local discourses, and utterances are modelled in a graph interaction manner. Based on the framework of MFDG, we propose two variants, MFDG-EN and MFDG-EE, to fuse domain knowledge into the dialogue graph. We apply MFDG and its two variants to a real-world online customer service dialogue system on the e-commerce website, JD, in which the MFDG can help achieving an automatic intent-oriented classification of finished service dialogues, and the MFDG-EE can further promote dialogue comprehension with a well-designed knowledge graph. Experiments on this in-house JD dataset and a public DailyDialog dataset demonstrate that MFDG performs reasonably well in multi-turn dialogue classification.
Adversarial Mask: Real-World Universal Adversarial Attack on Face Recognition Models

Deep learning-based facial recognition (FR) models have demonstrated state-of-the-art performance in the past few years, even when wearing protective medical face masks became commonplace during the COVID-19 pandemic. Given the outstanding performance of these models, the machine learning research community has shown increasing interest in challenging their robustness. Initially, researchers presented adversarial attacks in the digital domain, and later the attacks were transferred to the physical domain. However, in many cases, attacks in the physical domain are conspicuous, and thus may raise suspicion in real-world environments (e.g., airports). In this paper, we propose Adversarial Mask, a physical universal adversarial perturbation (UAP) against state-of-the-art FR models that is applied on face masks in the form of a carefully crafted pattern. In our experiments, we examined the transferability of our adversarial mask to a wide range of FR model architectures and datasets. In addition, we validated our adversarial mask's effectiveness in real-world experiments (CCTV use case) by printing the adversarial pattern on a fabric face mask. In these experiments, the FR system was only able to identify 3.34% of the participants wearing the mask (compared to a minimum of 83.34% with other evaluated masks). A demo of our experiments can be found at: https://youtu.be/_TXkDO5z11w .
Class-Incremental Learning via Knowledge Amalgamation

Catastrophic forgetting has been a significant problem hindering the deployment of deep learning algorithms in the continual learning setting. Numerous methods have been proposed to address the catastrophic forgetting problem where an agent loses its generalization power of old tasks while learning new tasks. We put forward an alternative strategy to handle the catastrophic forgetting with knowledge amalgamation (CFA), which learns a student network from multiple heterogeneous teacher models specializing in previous tasks and can be applied to current offline methods. The knowledge amalgamation process is carried out in a single-head manner with only a selected number of memorized samples and no annotations. The teachers and students do not need to share the same network structure, allowing heterogeneous tasks to be adapted to a compact or sparse data representation. We compare our method with competitive baselines from different strategies, demonstrating our approach's advantages. Source-code: github.com/Ivsucram/CFA .
Supervised Graph Contrastive Learning for Few-Shot Node Classification

Graphs present in many real-world applications, such as financial fraud detection, commercial recommendation, and social network analysis. But given the high cost of graph annotation or labeling, we face a severe graph label-scarcity problem, i.e., a graph might have a few labeled nodes. One example of such a problem is the so-called few-shot node classification. A predominant approach to this problem resorts to episodic meta-learning. In this work, we challenge the status quo by asking a fundamental question whether meta-learning is a must for few-shot node classification tasks. We propose a new and simple framework under the standard few-shot node classification setting as an alternative to meta-learning to learn an effective graph encoder. The framework consists of supervised graph contrastive learning with novel mechanisms for data augmentation, subgraph encoding, and multi-scale contrast on graphs. Extensive experiments on three benchmark datasets (CoraFull, Reddit, Ogbn) show that the new framework significantly outperforms state-of-the-art meta-learning based methods.
MRF-UNets: Searching UNet with Markov Random Fields

UNet [27] is widely used in semantic segmentation due to its simplicity and effectiveness. However, its manually-designed architecture is applied to a large number of problem settings, either with no architecture optimizations, or with manual tuning, which is time consuming and can be sub-optimal. In this work, firstly, we propose Markov Random Field Neural Architecture Search (MRF-NAS) that extends and improves the recent Adaptive and Optimal Network Width Search (AOWS) method [4] with (i) a more general MRF framework (ii) diverse M-best loopy inference (iii) differentiable parameter learning. This provides the necessary NAS framework to efficiently explore network architectures that induce loopy inference graphs, including loops that arise from skip connections. With UNet as the backbone, we find an architecture, MRF-UNet, that shows several interesting characteristics. Secondly, through the lens of these characteristics, we identify the sub-optimality of the original UNet architecture and further improve our results with MRF-UNetV2. Experiments show that our MRF-UNets significantly outperform several benchmarks on three aerial image datasets and two medical image datasets while maintaining low computational costs. The code is available at: https://github.com/zifuwanggg/MRF-UNets .
MAVIPER: Learning Decision Tree Policies for Interpretable Multi-agent Reinforcement Learning

Many recent breakthroughs in multi-agent reinforcement learning (MARL) require the use of deep neural networks, which are challenging for human experts to interpret and understand. On the other hand, existing work on interpretable reinforcement learning (RL) has shown promise in extracting more interpretable decision tree-based policies from neural networks, but only in the single-agent setting. To fill this gap, we propose the first set of algorithms that extract interpretable decision-tree policies from neural networks trained with MARL. The first algorithm, IVIPER, extends VIPER, a recent method for single-agent interpretable RL, to the multi-agent setting. We demonstrate that IVIPER learns high-quality decision-tree policies for each agent. To better capture coordination between agents, we propose a novel centralized decision-tree training algorithm, MAVIPER. MAVIPER jointly grows the trees of each agent by predicting the behavior of the other agents using their anticipated trees, and uses resampling to focus on states that are critical for its interactions with other agents. We show that both algorithms generally outperform the baselines and that MAVIPER-trained agents achieve better-coordinated performance than IVIPER-trained agents on three different multi-agent particle-world environments.
Differentially Private Bayesian Neural Networks on Accuracy, Privacy and Reliability

Bayesian neural network (BNN) allows for uncertainty quantification in prediction, offering an advantage over regular neural networks that has not been explored in the differential privacy (DP) framework. We fill this important gap by leveraging recent development in Bayesian deep learning and privacy accounting to offer a more precise analysis of the trade-off between privacy and accuracy in BNN. We propose three DP-BNNs that characterize the weight uncertainty for the same network architecture in distinct ways, namely DP-SGLD (via the noisy gradient method), DP-BBP (via changing the parameters of interest) and DP-MC Dropout (via the model architecture). Interestingly, we show a new equivalence between DP-SGD and DP-SGLD, implying that some non-Bayesian DP training naturally allows for uncertainty quantification. However, the hyperparameters such as learning rate and batch size, can have different or even opposite effects in DP-SGD and DP-SGLD. Extensive experiments are conducted to compare DP-BNNs, in terms of privacy guarantee, prediction accuracy, uncertainty quantification, calibration, computation speed, and generalizability to network architecture. As a result, we observe a new tradeoff between the privacy and the reliability. When compared to non-DP and non-Bayesian approaches, DP-SGLD is remarkably accurate under strong privacy guarantee, demonstrating the great potential of DP-BNN in real-world tasks.
Factors influencing the Supply Chain Management in e-Health using UTAUT model

&lt;abstract&gt; &lt;p&gt;Logistics in the healthcare industry involves coordinating the distribution of medical supplies and equipment across various departments and organizations. Supply Chain Management can help healthcare facilities identify weaknesses and devise strategies to address them. Using the Unified Theory of Acceptance and Use of Technology (UTAUT), the study investigates the motivations behind the individuals’ desire to use Internet of Things (IoT) solutions in healthcare. In order to better understand the factors that influence the use of IoT for e-HMS, a survey was administered to 210 healthcare IoT users. The study focuses on the potential medicinal applications of IoT technologies and incorporates the concepts of performance expectations, healthcare hazard, and trust (PHT) and perceived enabling circumstances (PFC) to complement past findings in the field. Overall, the study appears to be focused on contributing to the existing knowledge about the factors that influence the adoption of IoT technologies in healthcare, and it emphasizes the importance of considering theoretical constructs such as PHT and PFC in this context. The findings of the study can be used by IoT creators, medical experts, and vendors to optimize e-HMS and provide insight into the potential and limitations of UTAUT simulation to improve the logistic of Supply Chain Management in healthcare 4.0. The results have been analyzed by applying machine learning classifiers and have been visualized using different metrics.&lt;/p&gt; &lt;/abstract&gt;
Coreference Resolution through a seq2seq Transition-Based System

Abstract Most recent coreference resolution systems use search algorithms over possible spans to identify mentions and resolve coreference. We instead present a coreference resolution system that uses a text-to-text (seq2seq) paradigm to predict mentions and links jointly. We implement the coreference system as a transition system and use multilingual T5 as an underlying language model. We obtain state-of-the-art accuracy on the CoNLL-2012 datasets with 83.3 F1-score for English (a 2.3 higher F1-score than previous work [Dobrovolskii, 2021]) using only CoNLL data for training, 68.5 F1-score for Arabic (+4.1 higher than previous work), and 74.3 F1-score for Chinese (+5.3). In addition we use the SemEval-2010 data sets for experiments in the zero-shot setting, a few-shot setting, and supervised setting using all available training data. We obtain substantially higher zero-shot F1-scores for 3 out of 4 languages than previous approaches and significantly exceed previous supervised state-of-the-art results for all five tested languages. We provide the code and models as open source.1