bluuebunny/crossref_metadata_2025 huggingface dataset:
Counterfactual Prediction Under Selective Confounding
Authors: Kiani Sohaib , Barton Jared , Sushinsky Jon , Heimbach Lynda , Luo Bo
Published: September 2023

This research addresses the challenge of conducting interpretable causal inference between a binary treatment and its resulting outcome when not all confounders are known. Confounders are factors that have an influence on both the treatment and the outcome. We relax the requirement of knowing all confounders under desired treatment, which we refer to as Selective Confounding, to enable causal inference in diverse real-world scenarios. Our proposed scheme is designed to work in situations where multiple decision-makers with different policies are involved and where there is a re-evaluation mechanism after the initial decision to ensure consistency. These assumptions are more practical to fulfill compared to the availability of all confounders under all treatments. To tackle the issue of Selective Confounding, we propose the use of dual-treatment samples. These samples allow us to employ two-step procedures, such as Regression Adjustment or Doubly-Robust, to learn counterfactual predictors. We provide both theoretical error bounds and empirical evidence of the effectiveness of our proposed scheme using synthetic and real-world child placement data. Furthermore, we introduce three evaluation methods specifically tailored to assess the performance in child placement scenarios. By emphasizing transparency and interpretability, our approach aims to provide decision-makers with a valuable tool. The source code repository of this work is located at https://github.com/sohaib730/CausalML.
Towards Flexible Time-to-Event Modeling: Optimizing Neural Networks via Rank Regression
Authors: Lee Hyunjun , Lee Junhyun , Choi Taehwa , Kang Jaewoo , Choi Sangbum
Published: September 2023

Time-to-event analysis, also known as survival analysis, aims to predict the time of occurrence of an event, given a set of features. One of the major challenges in this area is dealing with censored data, which can make learning algorithms more complex. Traditional methods such as Cox’s proportional hazards model and the accelerated failure time (AFT) model have been popular in this field, but they often require assumptions such as proportional hazards and linearity. In particular, the AFT models often require pre-specified parametric distributional assumptions. To improve predictive performance and alleviate strict assumptions, there have been many deep learning approaches for hazard-based models in recent years. However, representation learning for AFT has not been widely explored in the neural network literature, despite its simplicity and interpretability in comparison to hazard-focused methods. In this work, we introduce the Deep AFT Rank-regression model for Time-to-event prediction (DART). This model uses an objective function based on Gehan’s rank statistic, which is efficient and reliable for representation learning. On top of eliminating the requirement to establish a baseline event time distribution, DART retains the advantages of directly predicting event time in standard AFT models. The proposed method is a semiparametric approach to AFT modeling that does not impose any distributional assumptions on the survival time distribution. This also eliminates the need for additional hyperparameters or complex model architectures, unlike existing neural network-based AFT models. Through quantitative analysis on various benchmark datasets, we have shown that DART has significant potential for modeling high-throughput censored time-to-event data.
S2M: Converting Single-Turn to Multi-Turn Datasets for Conversational Question Answering
Authors: Li Baokui , Zhang Sen , Zhang Wangshu , Chen Yicheng , Yang Changlin , Hu Sen , Xu Teng , Liu Siye , Li Jiwei
Published: September 2023

Supplying data augmentation to conversational question answering (CQA) can effectively improve model performance. However, there is less improvement from single-turn datasets in CQA due to the distribution gap between single-turn and multi-turn datasets. On the other hand, while numerous single-turn datasets are available, we have not utilized them effectively. To solve this problem, we propose a novel method to convert single-turn datasets to multi-turn datasets. The proposed method consists of three parts, namely, a QA pair Generator, a QA pair Reassembler, and a question Rewriter. Given a sample consisting of context and single-turn QA pairs, the Generator obtains candidate QA pairs and a knowledge graph based on the context. The Reassembler utilizes the knowledge graph to get sequential QA pairs, and the Rewriter rewrites questions from a conversational perspective to obtain a multi-turn dataset S2M. Our experiments show that our method can synthesize effective training resources for CQA. Notably, S2M ranks 1st place on the QuAC leaderboard (https://quac.ai/) at the time of submission (Aug 24th, 2022).
BEDCOE: Borderline Enhanced Disjunct Cluster Based Oversampling Ensemble for Online Multi-Class Imbalance Learning
Authors: Li Shuxian , Song Liyan , Cheung Yiu-ming , Yao Xin
Published: September 2023

Multi-class imbalance learning usually confronts more challenges especially when learning from streaming data. Most existing methods focus on manipulating class imbalance ratios, disregarding other data properties such as the borderline and the disjunct. Recent studies have shown non-negligible impact of disregarding these properties on deteriorating predictive performance. Online multi-class imbalance would further exacerbate such negative impact. To abridge the research gap of online multi-class imbalance learning, we propose to enhance the number of training times of borderline samples based on the disjunct class-wise clusters that are adaptively constructed over time for each class individually. Specifically, we propose a borderline enhanced strategy for ensemble aiming to increase the number of training times of samples neighboring to borderline areas of different classes. We also propose to generate synthetic samples for training based on the adaptively learned disjunct clusters that are maintained for each class individually online, catering for online multi-class imbalance problem directly. These two components construct the Borderline Enhanced Disjunct Cluster Based Oversampling Ensemble (BEDCOE). Experimental studies are conducted and demonstrate the effectiveness of BEDCOE and each of its components in dealing with online multi-class imbalance.
Diffusion Multi-Unit Auctions with Diminishing Marginal Utility Buyers
Authors: Liu Haolin , Lian Xinyuan , Zhao Dengji
Published: September 2023

We consider an auction design problem where a seller sells multiple homogeneous items to a set of connected buyers. Each buyer only knows the buyers she directly connects with and has a diminishing marginal utility valuation for the items. The seller initially only connects to some buyers who can be directly invited to the sale by the seller. Our goal is to design an auction to incentivize the buyers who are aware of the auction to further invite their neighbors to join the auction. This is challenging because the buyers are competing for the items and they would not invite each other by default. Thus, rewards need to be given to buyers who invite their neighbors, but the rewards should be carefully designed to guarantee both invitation incentives and the seller’s revenue. Solutions have been proposed recently for the settings where each buyer requires at most one unit but they are proved problematic. We move this forward to propose the very first diffusion auction for the multi-unit demand settings to improve both the social welfare and the seller’s revenue.
Epistemic JAADL: A Modal Logic for Joint Abilities with Imperfect Information
Authors: Liu Zhaoshuai , Liang Aiting , Liu Yongmei
Published: September 2023

Coordination and joint ability are important problems in representation and reasoning about multi-agent systems. Ghaderi et al. presented a formalization of joint ability of coalitions in the expressive first-order language of the situation calculus. Essentially, a coalition has joint ability to achieve a goal if after iterated elimination of dominated strategies, any remaining joint strategy achieves the goal. Based on their work, Liu et al. proposed JAADL, a modal logic for joint abilities under strategy commitments. In this paper, we propose EJAADL, an epistemic extension of JAADL, for imperfect information games where agents may have incomplete knowledge or even false beliefs about the world. Like Ghaderi et al.’s work, elimination of dominated strategies is now based on beliefs about the world, rather than facts about the world as in JAADL. Strategies are required to be uniform, i.e., they select the same action in all accessible histories. We illustrate EJAADL with examples, analyze its properties, and show that model checking memoryless EJAADL is in EXPTIME. Moreover, we consider the fragment of EJAADL without the iterated elimination operator, and show that model-checking the memoryless version of this fragment can be done in PSPACE.
Improving Bi-Objective Shortest Path Search with Early Pruning
Authors: Mandow L. , Pérez de la Cruz J.L.
Published: September 2023

Bi-objective search problems are a useful generalization of shortest path search. This paper reviews some recent contributions for the solution of this problem with emphasis on the efficiency of the dominance checks required for pruning, and introduces a new algorithm that improves time efficiency over previous proposals. Experimental results are presented to show the performance improvement using a set of standard problems over bi-objective road maps.
A Bayesian Optimization Framework for Finding Local Optima in Expensive Multimodal Functions
Authors: Mei Yongsheng , Lan Tian , Imani Mahdi , Subramaniam Suresh
Published: September 2023

Bayesian optimization (BO) is a popular global optimization scheme for sample-efficient optimization in domains with expensive function evaluations. The existing BO techniques are capable of finding a single global optimum solution. However, finding a set of global and local optimum solutions is crucial in a wide range of real-world problems, as implementing some of the optimal solutions might not be feasible due to various practical restrictions (e.g., resource limitation, physical constraints, etc.). In such domains, if multiple solutions are known, the implementation can be quickly switched to another solution, and the best possible system performance can still be obtained. This paper develops a multimodal BO framework to effectively find a set of local/global solutions for expensive-to-evaluate multimodal objective functions. We consider the standard BO setting with Gaussian process regression representing the objective function. We analytically derive the joint distribution of the objective function and its first-order derivatives. This joint distribution is used in the body of the BO acquisition functions to search for local optima during the optimization process. We introduce variants of the well-known BO acquisition functions to the multimodal setting and demonstrate the performance of the proposed framework in locating a set of local optimum solutions using multiple optimization problems.
THUNDER: Named Entity Recognition Using a Teacher-Student Model with Dual Classifiers for Strong and Weak Supervisions
Authors: Oh Seongwoong , Jung Woohwan , Shim Kyuseok
Published: September 2023

Strong and weak supervisions have complementary characteristics. However, utilizing both supervisions for named entity recognition (NER) has not been extensively studied. Moreover, the existing works address only incomplete annotations and neglects inaccurate annotations during NER model training. To effectively utilize weak labels, we introduce an auxiliary classifier that learns from weak labels. Furthermore, we adopt the teacher-student framework to handle both incomplete and inaccurate weak labels. A teacher model is first trained using both strongly and weakly supervised data, and next generates pseudo labels to replace weak labels. Then, the student model is trained so that the main classifier learns from both strong labels and confident pseudo labels while the auxiliary classifier learns from less confident pseudo labels. We also incorporate data augmentation through ChatGPT to generate additional annotated sentences to improve model performance and generalization capabilities. The experimental results with different weak supervisions demonstrate that our proposed method surpasses existing techniques.
Detecting Out-of-Distribution Objects Using Neuron Activation Patterns
Authors: Olber Bartłomiej , Radlak Krystian , Chachuła Krystian , Łyskawa Jakub , Frątczak Piotr
Published: September 2023

Object detection is essential to many perception algorithms used in modern robotics applications. Unfortunately, the existing models share a tendency to assign high confidence scores for out-of-distribution (OOD) samples. Although OOD detection has been extensively studied in recent years by the computer vision (CV) community, most proposed solutions apply only to the image recognition task. Real-world applications such as perception in autonomous vehicles struggle with far more complex challenges than classification. In our work, we focus on the prevalent field of object detection, introducing Neuron Activation PaTteRns for out-of-distribution samples detection in Object detectioN (NAPTRON). Performed experiments show that our approach outperforms state-of-the-art methods, without the need to affect in-distribution (ID) performance. By evaluating the methods in two distinct OOD scenarios and three types of object detectors we have created the largest open-source benchmark for OOD object detection.
Graph Neural Networks for Mapping Variables Between Programs
Authors: Orvalho Pedro , Piepenbrock Jelle , Janota Mikoláš , Manquinho Vasco
Published: September 2023

Automated program analysis is a pivotal research domain in many areas of Computer Science — Formal Methods and Artificial Intelligence, in particular. Due to the undecidability of the problem of program equivalence, comparing two programs is highly challenging. Typically, in order to compare two programs, a relation between both programs’ sets of variables is required. Thus, mapping variables between two programs is useful for a panoply of tasks such as program equivalence, program analysis, program repair, and clone detection. In this work, we propose using graph neural networks (GNNs) to map the set of variables between two programs based on both programs’ abstract syntax trees (ASTs). To demonstrate the strength of variable mappings, we present three use-cases of these mappings on the task of program repair to fix well-studied and recurrent bugs among novice programmers in introductory programming assignments (IPAs). Experimental results on a dataset of 4166 pairs of incorrect/correct programs show that our approach correctly maps 83% of the evaluation dataset. Moreover, our experiments show that the current state-of-the-art on program repair, greatly dependent on the programs’ structure, can only repair about 72% of the incorrect programs. In contrast, our approach, which is solely based on variable mappings, can repair around 88.5%.
Using Earley Parser for Recognizing Totally Ordered Hierarchical Plans
Authors: Pantůčková Kristýna , Barták Roman
Published: September 2023

Earley Parser is a top-down parser proposed for context-free grammars and used, for example, in the grammar constraint. Parsing trees of context-free grammars are very close to task decomposition trees used in hierarchical planning, specifically when the actions are totally ordered. This paper suggests using the Earley Parser to recognize totally ordered hierarchical plans. Given a sequence of actions – a prefix of the plan – and a task decomposition model, the plan recognition problem asks which task decomposes to a plan containing the given plan prefix. We will show that the Earley parser significantly increases the speed of plan recognition compared to the existing bottom-up parsing-based plan recognizer. The Earley parser’s performance is also on a par with the planning-based plan recognizer despite not using any planning heuristics.
AlegAATr the Bandit
Authors: Pedersen Ethan , Crandall Jacob
Published: September 2023

One design strategy for developing intelligent agents is to create N distinct behaviors, each of which works effectively in particular tasks and circumstances. At each time step during task execution, the agent, or bandit, chooses which of the N behaviors to use. Traditional bandit algorithms for making this selection often (1) assume the environment is stationary, (2) focus on asymptotic performance, and (3) do not incorporate external information that is available to the agent. Each of these simplifications limits these algorithms such that they often cannot be used successfully in practice. In this paper, we propose a new bandit algorithm, called AlegAATr, as a step toward overcoming these deficiencies. AlegAATr leverages a technique called Assumption-Alignment Tracking (AAT), proposed previously in the robotics literature, to predict the performance of each behavior in each situation. It then uses these predictions to decide which behavior to use at any given time. We demonstrate the effectiveness of AlegAATr in selecting behaviors in three problem domains: repeated games, ad hoc teamwork, and a human-robot pick-n-place task.
An Easy Rejection Sampling Baseline via Gradient Refined Proposals
Authors: Raff Edward , McLean Mark , Holt James
Published: September 2023

Rejection sampling is a common tool for low dimensional problems (d ≤ 2), often touted as an “easy” way to obtain valid samples from a distribution f(·) of interest. In practice it is non-trivial to apply, often requiring considerable mathematical effort to devise a good proposal distribution g(·) and select a supremum C. More advanced samplers require additional mathematical derivations, limitations on f(·), or even cross-validation, making them difficult to apply. We devise a new approximate baseline approach to rejection sampling that works with less information, requiring only a differentiable f(·) be specified, making it easier to use. We propose a new approach to rejection sampling by refining a parameterized proposal distribution with a loss derived from the acceptance threshold. In this manner we obtain comparable or better acceptance rates on current benchmarks by up to 7.3×, while requiring no extra assumptions or any derivations to use: only a differentiable f(·) is required. While approximate, the results are correct with high probability, and in all tests pass a distributional check. This makes our approach easy to use, reproduce, and efficacious.
Specifying Prior Beliefs over DAGs in Deep Bayesian Causal Structure Learning
Authors: Rittel Simon , Tschiatschek Sebastian
Published: September 2023

We consider the principled incorporation of prior knowledge in deep learning based Bayesian approaches to causal structure learning via the prior belief. In particular, we investigate how to include knowledge about individual edges and causal dependencies in the prior over the underlying directed acyclic graph (DAG). While conceptually simple, substantial challenges arise because the acyclicity of a DAG limits the modeling choices of the marginal distributions over its edges. Specifying the marginals iteratively unveils their dependencies and ensures a sound formulation of the probability distribution over DAGs. We provide recipes for formulating valid priors over DAGs for two recent deep learning based Bayesian approaches to causal structure learning and demonstrate empirically that using this prior knowledge can enable significantly more sample-efficient causal structure search.
Seq2Space: A Lightweight Solution for Sequence Modelling
Authors: Rixen Joel , Renz Matthias
Published: September 2023

This paper proposes a novel method for sequence modelling which we call Seq2Space. The basic idea is to project sequential information into the channel dimension. The Seq2Space layer outperforms the Transformer on every dataset contained in the Long Range Arena (LRA) benchmark as well as on the WSJ0-2 Mix benchmark for single-channel speech separation. Compared to previous methods which were tested on the LRA, the proposed Seq2Space layer does not quite reach the accuracy of the convolution-based methods. It is, however, more than twice as fast as the next fastest method as well as the most memory efficient, and still reaches an average accuracy of 71.15%. On the WSJ0-2Mix, the Seq2Space layer outperforms all other sequence modelling methods in our experiments except for the MEGA layer. By replacing Transformers with the Seq2Space layer on a current SOTA method, we are able to reach 22.8 dB SI-SDR improvement, which is comparable to current SOTA while being significantly faster and more memory efficient during both training and inference.
Learning to Collaborate by Grouping: A Consensus-Oriented Strategy for Multi-Agent Reinforcement Learning
Authors: Ruan Jingqing , Hao Xiaotian , Li Dong , Mao Hangyu
Published: September 2023

Multi-agent systems require effective coordination between groups and individuals to achieve common goals. However, current multi-agent reinforcement learning (MARL) methods primarily focus on improving individual policies and do not adequately address group-level policies, which leads to weak cooperation. To address this issue, we propose a novel Consensus-oriented Strategy (CoS) that emphasizes group and individual policies simultaneously. Specifically, CoS comprises two main components: (a) the vector quantized group consensus module, which extracts discrete latent embeddings that represent the stable and discriminative group consensus, and (b) the group consensus-oriented strategy, which integrates the group policy using a hypernet and the individual policies using the group consensus, thereby promoting coordination at both the group and individual levels. Through empirical experiments on cooperative navigation tasks with both discrete and continuous spaces, as well as google research football, we demonstrate that CoS outperforms state-of-the-art MARL algorithms and achieves better collaboration, thus providing a promising solution for achieving effective coordination in multi-agent systems.
Rationale-Guided Few-Shot Classification to Detect Abusive Language
Authors: Saha Punyajoy , Sheth Divyanshu , Kedia Kushal , Mathew Binny , Mukherjee Animesh
Published: September 2023

Abusive language is a concerning problem in online social media. Past research on detecting abusive language covers different platforms, languages, demographies, etc. However, models trained using these datasets do not perform well in cross-domain evaluation settings. To overcome this, a common strategy is to use a few samples from the target domain to train models to get better performance in that domain (cross-domain few-shot training). However, this might cause the models to overfit the artefacts of those samples. A compelling solution could be to guide the models toward rationales, i.e., spans of text that justify the text’s label. This method has been found to improve model performance in the in-domain setting across various NLP tasks. In this paper, we propose RGFS (Rationale-Guided Few-Shot Classification) for abusive language detection. We first build a multitask learning setup to jointly learn rationales, targets, and labels, and find a significant improvement of 6% macro F1 on the rationale detection task over training solely rationale classifiers. We introduce two rationale-integrated BERT-based architectures (the RGFS models) and evaluate our systems over five different abusive language datasets, finding that in the few-shot classification setting, RGFS-based models outperform baseline models by about 7% in macro F1 scores and perform competitively to models finetuned on other source domains. Furthermore, RGFS-based models outperform LIME/SHAP-based approaches in terms of plausibility and are close in performance in terms of faithfulness. Disclaimer: This paper contains material that many will find offensive or hateful. However, this cannot be avoided owing to the nature of the work.
A Paraconsistency Framework for Inconsistency Handling in Qualitative Spatial and Temporal Reasoning
Authors: Salhi Yakoub , Sioutis Michael
Published: September 2023

Inconsistency handling is a fundamental problem in knowledge representation and reasoning. In this paper, we study this problem in the context of qualitative spatio-temporal reasoning, a framework for reasoning about space and time in a symbolic, human-like manner, by following an approach similar to that used for defining paraconsistent logics; paraconsistency allows deriving informative conclusions from inconsistent knowledge bases by mainly avoiding the principle of explosion. Inspired by paraconsistent logics, such as Priest’s logic LPm, we introduce the notion of paraconsistent scenario (i.e., a qualitative solution), which can be seen as a scenario that allows a conjunction of base relations between two variables, e.g., x precedes ∧ follows y. Further, we present several interesting theoretical properties that concern paraconsistent scenarios, including computational complexity results, and describe two distinct approaches for computing paraconsistent scenarios and solving other related problems. Moreover, we provide implementations of our two methods for computing paraconsistent scenarios and experimentally evaluate them using different strategies/metrics. Finally, we show that our paraconsistent scenario notion allows us to adapt to qualitative reasoning one of the well-known inconsistency measures employed in the propositional case, namely, contension measure.
The Emotions of the Crowd: Learning Image Sentiment from Tweets via Cross-Modal Distillation
Authors: Serra Alessio , Carrara Fabio , Tesconi Maurizio , Falchi Fabrizio
Published: September 2023

Trends and opinion mining in social media increasingly focus on novel interactions involving visual media, like images and short videos, in addition to text. In this work, we tackle the problem of visual sentiment analysis of social media images – specifically, the prediction of image sentiment polarity. While previous work relied on manually labeled training sets, we propose an automated approach for building sentiment polarity classifiers based on a cross-modal distillation paradigm; starting from scraped multimodal (text + images) data, we train a student model on the visual modality based on the outputs of a textual teacher model that analyses the sentiment of the corresponding textual modality. We applied our method to randomly collected images crawled from Twitter over three months and produced, after automatic cleaning, a weakly-labeled dataset of ∼1.5 million images. Despite exploiting noisy labeled samples, our training pipeline produces classifiers showing strong generalization capabilities and outperforming the current state of the art on five manually labeled benchmarks for image sentiment polarity prediction.
Balancing Fairness and Efficiency in 3D Repeated Matching in Ridesharing
Authors: Shakya Garima , Yokoo Makoto
Published: September 2023

Ride-hailing services’ main feature is mediating the assignment and transactions between drivers and passengers. Essentially, they decide on the quality of passengers’ experience and the drivers’ workload balancing. To boost the company’s profit, these services try to maximize the utility for the passengers by optimizing the matching, resulting in shorter waiting times and better service availability. Often, in the process of maximizing revenue, drivers’ interests get sidelined. We focus on two objectives: efficiency (minimizing total distance traveled by drivers) and fairness (minimizing the maximum traveled distance by any driver) for shared-mode rides, where the vehicles’ capacity is two passengers. We theoretically show the relation between the optimal solutions of both objectives and as the problem is computationally intractable, we propose a heuristic algorithm to achieve an approximately optimal solution. We also propose a re-assignment-based algorithm when the aim is to achieve maximum matching with fairness up to a given threshold, if that is feasible. The experimental analysis for the proposed algorithms on real-world data from Chicago city shows that our approach can significantly improve fairness for drivers without losing much efficiency.
Create and Find Flatness: Building Flat Training Spaces in Advance for Continual Learning
Authors: Shi Wenhang , Chen Yiren , Zhao Zhe , Lu Wei , Yan Kimmo , Du Xiaoyong
Published: September 2023

Catastrophic forgetting remains a critical challenge in the field of continual learning, where neural networks struggle to retain prior knowledge while assimilating new information. Most existing studies emphasize mitigating this issue only when encountering new tasks, overlooking the significance of the pre-task phase. Therefore, we shift the attention to the current task learning stage, presenting a novel framework, C&F (Create and Find Flatness), which builds a flat training space for each task in advance. Specifically, during the learning of the current task, our framework adaptively creates a flat region around the minimum in the the loss landscape. Subsequently, it finds the parameters’ importance to the current task based on their flatness degrees. When adapting the model to a new task, constraints are applied according to the flatness and a flat space is simultaneously prepared for the impending task. We theoretically demonstrate the consistency between the created and found flatness. In this manner, our framework not only accommodates ample parameter space for learning new tasks but also preserves the preceding knowledge of earlier tasks. Experimental results exhibit C&F’s state-of-the-art performance as a standalone continual learning approach and its efficacy as a framework incorporating other methods. Our work is available at https://github.com/Eric8932/Create-and-Find-Flatness.
A Simple Debiasing Framework for Out-of-Distribution Detection in Human Action Recognition
Authors: Sim Minho , Lee Young-Jun , Lee Dongkun , Lee Jongwhoa , Choi Ho-Jin
Published: September 2023

In real-world scenarios, detecting out-of-distribution (OOD) action is important when deploying a deep learning-based human action recognition (HAR) model. However, HAR models are easily biased to static information in the video (e.g., background), which can lead to performance degradation of OOD detection methods. In this paper, we propose a simple debiasing framework for out-of-distribution detection in human action recognition. Specifically, our framework eliminates patches with static bias in video using attention maps extracted from the video vision transformer model. Experimental results show that our framework achieves consistent performance improvement on multiple OOD action detection methods and challenging benchmarks. Furthermore, we introduce two new OOD action detection tasks, Kinetics-400 vs. Kinetics-600 exclusive and Kinetics-400 vs. Kinetics-700 exclusive, to validate our method in a setting close to the real-world scenario. With extensive experiments, we demonstrate the effectiveness of our attention-based masking, and in-depth analysis validates the effect of static bias on OOD action detection. The source code and supplementary materials are available at: https://github.com/Simcs/attention-masking.
Minimum Target Sets in Non-Progressive Threshold Models: When Timing Matters
Authors: Soltani Hossein , Zehmakan Ahad N. , Hushmandi Ataabak B.
Published: September 2023

Let G be a graph, which represents a social network, and suppose each node v has a threshold value τ(v). Consider an initial configuration, where each node is either positive or negative. In each discrete time step, a node v becomes/remains positive if at least τ(v) of its neighbors are positive and negative otherwise. A node set S is a Target Set (TS) whenever the following holds: if S is fully positive initially, all nodes in the graph become positive eventually. We focus on a generalization of TS, called Timed TS (TTS), where it is permitted to assign a positive state to a node at any step of the process, rather than just at the beginning. We provide graph structures for which the minimum TTS is significantly smaller than the minimum TS, indicating that timing is an essential aspect of successful target selection strategies. Furthermore, we prove tight bounds on the minimum size of a TTS in terms of the number of nodes and maximum degree when the thresholds are assigned based on the majority rule. We show that the problem of determining the minimum size of a TTS is NP-hard and provide an Integer Linear Programming formulation and a greedy algorithm. We evaluate the performance of our algorithm by conducting experiments on various synthetic and real-world networks. We also present a linear-time exact algorithm for trees.
Unit Refutations of Difference Constraint Systems
Authors: Subramani K. , Wojciechowski Piotr
Published: September 2023

This paper is concerned with a refutation system (proof system) for a class of linear constraint systems called difference constraint systems (DCS). In particular, we study the refutability of DCSs in the unit refutation (UR) system. Recall that a difference constraint is a linear relationship of the form: xi-xj ≤ bij and a DCS is a conjunction of such constraints. Associated with a refutation system are three important features, viz., (a) Soundness, (b) Completeness, and (c) Efficiency. The UR system is sound and efficient; however, it is incomplete, in that unsatisfiable DCSs may not have unit refutations. We establish that this refutation system is efficient in that there exists a tractable algorithm for determining if a DCS has a UR. Investigating weak (incomplete) refutation systems leads to a better understanding of the inference rules required for establishing contradictions in the given constraint system. Thus, this study is well-motivated. Despite the fact that unit refutations can be exponentially long in terms of the input system size, we provide a compact representation of these refutations. This compact representation is an important contribution of this paper.
An Empirical Study of Retrieval-Enhanced Graph Neural Networks
Authors: Wang Dingmin , Liu Shengchao , Wang Hanchen , Grau Bernardo Cuenca , Song Linfeng , Tang Jian , Song Le , Liu Qi
Published: September 2023

Graph Neural Networks (GNNs) are effective tools for graph representation learning. Most GNNs rely on a recursive neighborhood aggregation scheme, named message passing, thereby their theoretical expressive power is limited to the first-order Weisfeiler-Lehman test (1-WL). An effective approach to this challenge is to explicitly retrieve some annotated examples used to enhance GNN models. While retrieval-enhanced models have been proved to be effective in many language and vision domains, it remains an open question how effective retrieval-enhanced GNNs are when applied to graph datasets. Motivated by this, we want to explore how the retrieval idea can help augment the useful information learned in the graph neural networks, and we design a retrieval-enhanced scheme called GRAPHRETRIEVAL, which is agnostic to the choice of graph neural network models. In GRAPHRETRIEVAL, for each input graph, similar graphs together with their ground-true labels are retrieved from an existing database. Thus they can act as a potential enhancement to complete various graph property predictive tasks. We conduct comprehensive experiments over 13 datasets, and we observe that GRAPHRETRIEVAL is able to reach substantial improvements over existing GNNs. Moreover, our empirical study also illustrates that retrieval enhancement is a promising remedy for alleviating the long-tailed label distribution problem.
MonoSKD: General Distillation Framework for Monocular 3D Object Detection via Spearman Correlation Coefficient
Authors: Wang Sen , Zheng Jin
Published: September 2023

Monocular 3D object detection is an inherently ill-posed problem, as it is challenging to predict accurate 3D localization from a single image. Existing monocular 3D detection knowledge distillation methods usually project the LiDAR onto the image plane and train the teacher network accordingly. Transferring LiDAR-based model knowledge to RGB-based models is more complex, so a general distillation strategy is needed. To alleviate cross-modal problem, we propose MonoSKD, a novel Knowledge Distillation framework for Monocular 3D detection based on Spearman correlation coefficient, to learn the relative correlation between cross-modal features. Considering the large gap between these features, strict alignment of features may mislead the training, so we propose a looser Spearman loss. Furthermore, by selecting appropriate distillation locations and removing redundant modules, our scheme saves more GPU resources and trains faster than existing methods. Extensive experiments are performed to verify the effectiveness of our framework on the challenging KITTI 3D object detection benchmark. Our method achieves state-of-the-art performance until submission with no additional inference computational cost. Our codes are available at https://github.com/Senwang98/MonoSKD.
Normalized Stochastic Heavy Ball with Adaptive Momentum1
Authors: Wen Ziqing , Deng Xiaoge , Sun Tao , Li Dongsheng
Published: September 2023

The heavy ball momentum technique is widely used in accelerating the machine learning training process, which has demonstrated significant practical success in optimization tasks. However, most heavy ball methods require a preset hyperparameter that will result in excessive tuning, and a calibrated fixed hyperparameter may not lead to optimal performance. In this paper, we propose an adaptive criterion for the choice of the normalized momentum-related hyperparameter, motivated by the quadratic optimization training problem, to eliminate the adverse for tuning the hyperparameter and thus allow for a computationally efficient optimizer. We theoretically prove that our proposed adaptive method promises convergence for L-Lipschitz functions. In addition, we verify its practical efficiency on existing extensive machine learning benchmarks for image classification tasks. The numerical results show that besides the speed improvement, our proposed methods enjoy advantages, including more robust to large learning rates and better generalization.
Contrastive Learning with Diverse Samples
Authors: Wu Lilei , Liu Jie
Published: September 2023

Unsupervised visual representation learning has gained much attention from the computer vision community because of the recent contrastive learning achievements. Current work mainly adopts instance discrimination as the pretext task, which treats every single instance as a different class (negative) and uses a collection of data augmentation techniques to generate more examples (positive) for each class. The idea is straightforward and efficient but will generally cause similar instances to be classified into different classes. Such problem has been defined as “class collision” in some previous works and is shown to hurt the representation ability. Motivated by this observation, we present a solution to address this issue by filtering similar negative examples from each mini-batch. Concretely, we model the problem as a Determinantal Point Process (DPP) so that similar instances can be filtered stochastically, and diverse samples are expected to be sampled for contrastive training. Besides, we further introduce a priority term for each instance, which indicates the hardness of its positives, so that instances with more hard positives are more likely to be sampled for contributing to the optimization. Our sampling can be efficiently implemented in a feed-forward manner and further accelerated by our encouraged complement DPP. Extensive experimental results demonstrate our priority over the standard setup of contrastive learning.
Highly-Efficient Robinson-Foulds Distance Estimation with Matrix Correction
Authors: Yu Fangchen , Bao Rui , Mao Jianfeng , Li Wenye
Published: September 2023

Phylogenetic trees are essential in studying evolutionary relationships, and the Robinson-Foulds (RF) distance is a widely used metric to calculate pairwise dissimilarities between phylogenetic trees, with various applications in both the biology and computing communities. However, generating a precise RF distance matrix becomes difficult or even intractable when tree information is partially missing. To address this issue, we introduce a novel distance correction algorithm for estimating the RF distance matrix of incomplete phylogenetic trees. Our method innovatively harnesses the assumption of Euclidean embedding, correcting an approximate distance matrix into a valid distance metric, guaranteed to be closer to the unknown ground-truth. Despite its simplicity, our approach exhibits robust performance, efficiency, and scalability in empirical evaluations, outperforming classical distance correction algorithms and holding potential benefits in downstream applications. Our code is available at https://github.com/CUHKSZ-Yu/EMC.
Privacy-Enhanced AI Assistants Based on Dialogues and Case Similarity
Authors: Zhan Xiao , Sarkadi Ştefan , Such Jose
Published: September 2023

Personal assistants (PAs) such as Amazon Alexa, Google Assistant and Apple Siri are now widespread. However, without adequate safeguards and controls their use may lead to privacy risks and violations. In this paper, we propose a model for privacy-enhancing PAs. The model is an interpretable AI architecture that combines 1) a dialogue mechanism for understanding the user and getting online feedback from them, with 2) a decision-making mechanism based on case-based reasoning considering both user and scenario similarity. We evaluate our model using real data about users’ privacy preferences, and compare its accuracy and demand for user involvement with both online machine learning and other, more interpretable, AI approaches. Our results show that our proposed architecture is more accurate and requires less intervention from the users than existing approaches.
Deep Unsupervised Hashing with Hyperbolic Multi-Structure Learning
Authors: Zhao Chuang , Ling Hefei , Shi Yuxuan , Chen Jiazhong , Cao Qiang
Published: September 2023

Unsupervised hashing aims to learn a compact binary hash code to represent complex image content without label information. Existing deep unsupervised hashing methods typically first employ extracted image embeddings to construct semantic similarity structures and then map the images into compact hash codes while preserving the semantic similarity structure. However, the limited representation power of embeddings in Euclidean space and the inadequate exploration of the similarity structure in current methods often result in poorly discriminative hash codes. In this paper, we propose a novel method called Hyperbolic Multi-Structure Hashing (HMSH) to address these issues. Specifically, to increase the representation power of embeddings, we propose to map embeddings from Euclidean space to hyperbolic space and use the similarity structure constructed in hyperbolic space to guide hash learning. Meanwhile, to fully explore the structural information, we investigate four kinds of data structures, including local neighborhood structure, global clustering structure, inter/intra-class variation and variation under perturbation. Different data structures can complement each other, which is beneficial for hash learning. Extensive experimental results on three benchmark image datasets show that HMSH significantly outperforms state-of-the-art unsupervised hashing methods for image retrieval.
CICERO: A GPT2-Based Writing Assistant to Investigate the Effectiveness of Specialized LLMs’ Applications in e-Justice
Authors: Calamo Marco , De Luzi Francesca , Macrì Mattia , Mencattini Tommaso , Mecella Massimo
Published: September 2023

Does it make sense to develop specialized writing assistants in the era of LLMs - Large Language Models? In this paper, we present CICERO, a specialized writing assistant we have developed for writing (pieces of) sentences in the Italian legal system. Our proposed solution involves fine-tuning a transformer on a pre-processed corpus of Italian civil judgments, resulting in a novel language model that can be deployed as a writing assistant for legal users to improve text writing efficiency. The model can also be further fine-tuned for use in other law-related natural language processing tasks. The experimental validation of CICERO allows us also to draw interesting insights on the meaningful, if any, of developing specialized tools for assisting specific classes of users and knowledge workers, in an era in which we witness the widespread adoption of LLMs.
Repetitive Behaviors in Alzheimer’s Disease: A Systematic Review and Meta-Analysis
Authors: Polin Clément , Gellé Thibaut , Auditeau Emilie , Adou Caroline , Clément Jean-Pierre , Calvet Benjamin
Published: September 2023

Background: Repetitive behaviors (RBs) are a well-known symptom of Alzheimer’s disease (AD); however, they have been little studied and have not been the subject of any specific literature review. Objective: To conduct a systematic review of all studies to document RBs in AD. Methods: An extensive literature search combining five databases and a meta-analysis were conducted to investigate the frequency, nature, and cognitive correlates of RBs in AD. Results: Ten studies were included in the review. Seven studies out of ten investigated the frequency of RBs in patients with AD, which ranged from 52.3% to 87%. A meta-analysis showed an overall frequency of 66.3% (95% CI: 55.5; 77.1) of patients exhibiting RBs in AD, but important heterogeneity was observed between studies. Three studies investigated the predominant nature of RBs in AD. Verbal RBs, complex behavioral stereotypies, and simple motor stereotypies have been identified to different degrees depending on the level of dementia. Most verbal RBs are underpinned by episodic memory impairment, while simple motor stereotypies and complex behavioral stereotypies are mostly underpinned by executive dysfunction. Conclusions: The current review seems to suggest that there are two types of mechanisms underpinning RBs involved in AD. The first is observed especially in the mild stages of the disease and is mediated by episodic memory impairment. The second occurs later and is mediated by executive impairment. Additional studies should be conducted to improve the knowledge about RBs in AD and thus improve their management. Systematic review registration number: PROSPERO 2022: CRD42022310027