Reviewerly/exHarmony huggingface dataset:
Harmonizing Feature Attributions Across Deep Learning Architectures: Enhancing Interpretability and Consistency

Enhancing the interpretability and consistency of machine learning models is critical to their deployment in real-world applications. Feature attribution methods have gained significant attention, which provide local explanations of model predictions by attributing importance to individual input features. This study examines the generalization of feature attributions across various deep learning architectures, such as convolutional neural networks (CNNs) and vision transformers. We aim to assess the feasibility of utilizing a feature attribution method as a future detector and examine how these features can be harmonized across multiple models employing distinct architectures but trained on the same data distribution. By exploring this harmonization, we aim to develop a more coherent and optimistic understanding of feature attributions, enhancing the consistency of local explanations across diverse deep-learning models. Our findings highlight the potential for harmonized feature attribution methods to improve interpretability and foster trust in machine learning applications, regardless of the underlying architecture.
Flexible Automation of Quantified Multi-Modal Logics with Interactions

Quantified modal logics have numerous applications in mathematics, computer science, AI, philosophy, and further fields. For AI applications, in particular in knowledge representation and reasoning, modal logic formalisms often employ multiple modalities with different properties and intricate interaction schemes. In this paper, a format for representing such non-trivial modal logic set-ups, and reasoning problems within these logics, is presented. Automated reasoning for this large family of modal logics is then enabled via a translation to classical higher-order logic.
LESS is More: LEan Computing for Selective Summaries

An agent in pursuit of a task may work with a corpus containing text documents. To perform information retrieval on the corpus, the agent may need annotations—additional data associated with the documents. Subjective Content Descriptions (SCDs) provide additional location-specific data for text documents. SCDs can be estimated without additional supervision for any corpus of text documents. However, the estimated SCDs lack meaningful descriptions, i.e., labels consisting of short summaries. Labels are important to identify relevant SCDs and documents by the agent and its users. Therefore, this paper presents LESS, a LEan computing approach for Selective Summaries, which can be used as labels for SCDs. LESS uses word distributions of the SCDs to compute labels. In an evaluation, we compare the labels computed by LESS with labels computed by large language models and show that LESS computes similar labels but requires less data and computational power.
Interactive Link Prediction as a Downstream Task for Foundational GUI Understanding Models

AI models that can recognize and understand the semantics of graphical user interfaces (GUIs) enable a variety of use cases ranging from accessibility to automation. Recent efforts in this domain have pursued the development of a set of foundation models: generic GUI understanding models that can be used off-the-shelf to solve a variety of GUI-related tasks, including ones that they were not trained on. In order to develop such foundation models, meaningful downstream tasks and baselines for GUI-related use cases will be required. In this paper, we present interactive link prediction as a downstream task for GUI understanding models and provide baselines as well as testing tools to effectively and efficiently evaluate predictive GUI understanding models. In interactive link prediction, the task is to predict whether tapping on an element on one screen of a mobile application (source element) navigates the user to a second screen (target screen). If this task is solved sufficiently, it can demonstrate an understanding of the relationship between elements and components across screens and enable various applications in GUI design automation and assistance. To encourage and support research on interactive link prediction, this paper contributes (1) a pre-processed large-scale dataset of links in mobile applications (18,830 links from 5,362 applications) derived from the popular RICO dataset, (2) performance baselines from five heuristic-based and two learning-based GUI understanding models, (3) a small-scale dataset of links in mobile GUI prototypes including ratings from an online study with 36 end-users for out-of-sample testing, and (4) a Figma plugin that can leverage link prediction models to automate and assist mobile GUI prototyping.
Associative Reasoning for Commonsense Knowledge

Associative reasoning refers to the human ability to focus on knowledge that is relevant to a particular problem. In this process, the meaning of symbol names plays an important role: when humans focus on relevant knowledge about the symbol ice, similar symbols like snow also come into focus. In this paper, we model this associative reasoning by introducing a selection strategy that extracts relevant parts from large commonsense knowledge sources. This selection strategy is based on word similarities from word embeddings and is therefore able to take the meaning of symbol names into account. We demonstrate the usefulness of this selection strategy with a case study from creativity testing.
Stepsize Learning for Policy Gradient Methods in Contextual Markov Decision Processes

Policy-based algorithms are among the most widely adopted techniques in model-free RL, thanks to their strong theoretical groundings and good properties in continuous action spaces. Unfortunately, these methods require precise and problem-specific hyperparameter tuning to achieve good performance, and tend to struggle when asked to accomplish a series of heterogeneous tasks. In particular, the selection of the step size has a crucial impact on their ability to learn a highly performing policy, affecting the speed and the stability of the training process, and often being the main culprit for poor results. In this paper, we tackle these issues with a Meta Reinforcement Learning approach, by introducing a new formulation, known as meta-MDP, that can be used to solve any hyperparameter selection problem in RL with contextual processes. After providing a theoretical Lipschitz bound to the difference of performance in different tasks, we adopt the proposed framework to train a batch RL algorithm to dynamically recommend the most adequate step size for different policies and tasks. In conclusion, we present an experimental campaign to show the advantages of selecting an adaptive learning rate in heterogeneous environments.
BipNRL: Mutual Information Maximization on Bipartite Graphs for Node Representation Learning

Unsupervised representation learning on bipartite graphs is of particular interest due to the lack of explicit labels and its application in a variety of domains. Several unsupervised representation learning methods make use of mutual information maximization between node-level and graph/sub-graph-level information. We argue that such methods are better suited to graph level tasks than node level tasks because local information in node representation is dominated by graph level information. Additionally, current approaches rely on building a global summary vector in which information diminishes as the graph scales. In this work, we concentrate on optimizing representations for node-level downstream tasks such as node classification/regression. The Bipartite Node Representation Learning (BipNRL) method, which aims to learn the node embeddings by maximizing MI between the node’s representation and self features without using summary vector is introduced. Furthermore, we propose the decoupled graph context loss on projected graphs to preserve the bipartite graph structure, enhancing representation for node-level tasks. Finally, we utilize attention-based neighborhood aggregation and modify the neighbourhood sampling algorithm to account for the high variance of node degree that is common in bipartite graphs. Extensive experiments show that BipNRL achieves cutting-edge results on multiple downstream tasks across diverse datasets.
Symbolic Regression via Control Variable Genetic Programming

Learning symbolic expressions directly from experiment data is a vital step in AI-driven scientific discovery. Nevertheless, state-of-the-art approaches are limited to learning simple expressions. Regressing expressions involving many independent variables still remain out of reach. Motivated by the control variable experiments widely utilized in science, we propose Control Variable Genetic Programming (CVGP) for symbolic regression over many independent variables. CVGP expedites symbolic expression discovery via customized experiment design, rather than learning from a fixed dataset collected a priori. CVGP starts by fitting simple expressions involving a small set of independent variables using genetic programming, under controlled experiments where other variables are held as constants. It then extends expressions learned in previous generations by adding new independent variables, using new control variable experiments in which these variables are allowed to vary. Theoretically, we show CVGP as an incremental building approach can yield an exponential reduction in the search space when learning a class of expressions. Experimentally, CVGP outperforms several baselines in learning symbolic expressions involving multiple independent variables.
SALAS: Supervised Aspect Learning Improves Abstractive Multi-document Summarization Through Aspect Information Loss

Abstractive multi-document summarization (MDS) aims at summarizing and paraphrasing the salient key information in multiple documents. For dealing with the long-input issue brought by multiple documents, most previous work extracts salient sentence-level information from the input documents and then performs summarizing on the extracted information. However, the aspects of documents are neglected. The limited ability to discover the content on certain aspects hampers the key information seeking and ruins the comprehensiveness of the generated summaries. To solve the issue, we propose a novel Supervised Aspect-Learning Abstractive Summarization framework (SALAS) and a new aspect information loss (AILoss) to learn aspect information to supervise the generating process heuristically. Specifically, SALAS adopts three probes to capture aspect information as both constraints of the objective function and supplement information to be expressed in the representations. Aspect information is explicitly discovered and exploited to facilitate generating comprehensive summaries by AILoss. We conduct extensive experiments on three public datasets. The experimental results demonstrate that SALAS outperforms previous state-of-the-art (SOTA) baselines, achieving a new SOTA performance on the three MDS datasets. We make our code for SALAS publicly available ( https://github.com/Hytn/AspectSum ).
Enhancing Table Retrieval with Dual Graph Representations

Table retrieval aims to rank candidate tables for answering natural language query, in which the most critical problem is how to learn informative representations for structured tables. Most previous methods roughly flatten the table and send it into a sequence encoder, ignoring the structure information of tables and the semantic interaction between table cells and contexts. In this paper, we propose a dual graph based method to perceive the semantics and structure of tables, so as to preferably support the downstream table retrieval task. Inspired by human cognition, we first decouple a table into the row view and column view, then build dual graphs from these two views with the consideration of table contexts. Afterward, intra-graph and inter-graph interactions are iteratively performed for aggregating and exchanging local row- and column-oriented features respectively, and an adaptive fusion strategy is eventually tailor-made for sophisticated table representations. In this way, the table structure and semantic information are well considered with dual-graph modeling. Consequently, the input query can match the target tables based on their full-fledged table representations and achieve the ultimate ranking results more accurately. Extensive experiments verify the superiority of our dual graphs over strong baselines on two table retrieval datasets WikiTables and WebQueryTable. Further analyses also confirm the adaptability for row-/column-oriented tables, and show the rationality and generalization of dual graphs. The source code is available at https://github.com/ty33123/DualG .
Eigensubspace of Temporal-Difference Dynamics and How It Improves Value Approximation in Reinforcement Learning

We propose a novel value approximation method, namely “Eigensubspace Regularized Critic (ERC)” for deep reinforcement learning (RL). ERC is motivated by an analysis of the dynamics of Q-value approximation error in the Temporal-Difference (TD) method, which follows a path defined by the 1-eigensubspace of the transition kernel associated with the Markov Decision Process (MDP). It reveals a fundamental property of TD learning that has remained unused in previous deep RL approaches. In ERC, we propose a regularizer that guides the approximation error tending towards the 1-eigensubspace, resulting in a more efficient and stable path of value approximation. Moreover, we theoretically prove the convergence of the ERC method. Besides, theoretical analysis and experiments demonstrate that ERC effectively reduces the variance of value functions. Among 26 tasks in the DMControl benchmark, ERC outperforms state-of-the-art methods for 20. Besides, it shows significant advantages in Q-value approximation and variance reduction. Our code is available at https://sites.google.com/view/erc-ecml23/ .
Contrastive Representation Through Angle and Distance Based Loss for Partial Label Learning

Partial label learning (PLL) is a form of weakly supervised learning which aims to train a deep network from training instances and its corresponding label set. The label set, also known as the candidate set, is a group of labels associated with each training instance, out of which only one label is the ground truth for the training instance. Contrastive learning is one of the popular techniques used to learn from a partially labeled dataset, intending to reduce intra-class while maximizing inter-class distance. In this paper, we suggest improving the contrastive technique used in PiCO. The proposed Contrastive Representation via Angle and Distance based Loss (CRADL) segregates the contrastive loss into two parts, the angle based loss and the distance based loss. The former angle based loss covers the angular separation between two contrastive vectors. However, we showcase a scenario where such angular loss prefers one contrastive vector over the other despite having the same angle. Thus, the second loss term built on distance fixes the issue. We show experiments on CIFAR-10 and CIFAR-100, where the corresponding PLL databases are generated using uniform noise. The experiments show that the PLL algorithms learn better using the proposed CRADL-based learning and generate distinguishing representations, as observed by compact cluster formation with CRADL. This eventually results in CRADL outperforming the current state-of-the-art studies in PLL setup at different uniform noise rates.
Offline Reinforcement Learning with On-Policy Q-Function Regularization

The core challenge of offline reinforcement learning (RL) is dealing with the (potentially catastrophic) extrapolation error induced by the distribution shift between the history dataset and the desired policy. A large portion of prior work tackles this challenge by implicitly/explicitly regularizing the learning policy towards the behavior policy, which is hard to estimate reliably in practice. In this work, we propose to regularize towards the Q-function of the behavior policy instead of the behavior policy itself, under the premise that the Q-function can be estimated more reliably and easily by a SARSA-style estimate and handles the extrapolation error more straightforwardly. We propose two algorithms taking advantage of the estimated Q-function through regularizations, and demonstrate they exhibit strong performance on the D4RL benchmarks.
Towards Understanding the Mechanism of Contrastive Learning via Similarity Structure: A Theoretical Analysis

Contrastive learning is an efficient approach to self-supervised representation learning. Although recent studies have made progress in the theoretical understanding of contrastive learning, the investigation of how to characterize the clusters of the learned representations is still limited. In this paper, we aim to elucidate the characterization from theoretical perspectives. To this end, we consider a kernel-based contrastive learning framework termed Kernel Contrastive Learning (KCL), where kernel functions play an important role when applying our theoretical results to other frameworks. We introduce a formulation of the similarity structure of learned representations by utilizing a statistical dependency viewpoint. We investigate the theoretical properties of the kernel-based contrastive loss via this formulation. We first prove that the formulation characterizes the structure of representations learned with the kernel-based contrastive learning framework. We show a new upper bound of the classification error of a downstream task, which explains that our theory is consistent with the empirical success of contrastive learning. We also establish a generalization error bound of KCL. Finally, we show a guarantee for the generalization ability of KCL to the downstream classification task via a surrogate bound.
Exploring and Exploiting Data-Free Model Stealing

Deep machine learning models, e.g., image classifier, are increasingly deployed in the wild to provide services to users. Adversaries are shown capable of stealing the knowledge of these models by sending inference queries and then training substitute models based on query results. The availability and quality of adversarial query inputs are undoubtedly crucial in the stealing process. The recent prior art demonstrates the feasibility of replacing real data by exploring the synthetic adversarial queries, so called data-free attacks, under strong adversarial assumptions, i.e., the deployed classier returns not only class labels but also class probabilities. In this paper, we consider a general adversarial model and propose an effective data-free stealing algorithm, TandemGAN, which not only explores synthetic queries but also explicitly exploits the high quality ones. The core of TandemGAN is composed of (i) substitute model which imitates the target model through synthetic queries and their inferred labels; and (ii) a tandem generator consisting of two networks, $$\mathcal {G}_x$$ and $$\mathcal {G}_e$$ , which first explores the synthetic data space via $$\mathcal {G}_x$$ and then exploits high-quality examples via $$\mathcal {G}_e$$ to maximize the knowledge transfer from the target to the substitute model. Our results on four datasets show that the accuracy of our trained substitute model ranges between 96–67% of the target model and outperforms the existing state-of-the-art data-free model stealing approach by up to 2.5X.
Boosting Object Representation Learning via Motion and Object Continuity

Recent unsupervised multi-object detection models have shown impressive performance improvements, largely attributed to novel architectural inductive biases. Unfortunately, despite their good object localization and segmentation capabilities, their object encodings may still be suboptimal for downstream reasoning tasks, such as reinforcement learning. To overcome this, we propose to exploit object motion and continuity (objects do not pop in and out of existence). This is accomplished through two mechanisms: (i) providing temporal loss-based priors on object locations, and (ii) a contrastive object continuity loss across consecutive frames. Rather than developing an explicit deep architecture, the resulting unsupervised Motion and Object Continuity (MOC) training scheme can be instantiated using any object detection model baseline. Our results show large improvements in the performances of variational and slot-based models in terms of object discovery, convergence speed and overall latent object representations, particularly for playing Atari games. Overall, we show clear benefits of integrating motion and object continuity for downstream reasoning tasks, moving beyond object representation learning based only on reconstruction as well as evaluation based only on instance segmentation quality.
ConGCN: Factorized Graph Convolutional Networks for Consensus Recommendation

An essential weakness of existing personalized recommender systems is that the learning is biased and dominated by popular items and users. Existing methods, particularly graph-based approaches, primarily focus on the “heterogeneous interaction” between user-item, leading to a disproportionately large influence of popular nodes during the graph learning process. Recently, popularity debiasing models have been proposed to address this issue, but they excessively concentrate on considering cause-effect or re-weighting the item/user popularity. These approaches artificially alter the nature of the data, inadvertently downplaying the representation learning of popular items/users. Consequently, balancing the trade-off between global recommendation accuracy and unpopular items/users exposure is challenging. In this paper, we advocate the concept of “homogeneous effect” from both user and item perspectives to explore the intrinsic correlation and alleviate the bias effects. Our core theme is to simultaneously factorize user-item interactions, user-user similarity, and item-item correlation, thereby learning balanced representations for items and users. To pursue well-balanced representations, we propose a Consensus factorized Graph Convolution neural Network (ConGCN), which leverages graph-based nonlinear representation learning and manifold constraints to regulate the embedding learning. An inherent advantage of ConGCN is its consensus optimization nature, where item-item and user-user relationships ensure that unpopular items are well preserved in the embedding space. The experiments on four real-world datasets demonstrate that ConGCN outperforms existing single-task-oriented methods on two typical tasks with opposite goals (global recommendation and popularity debiasing recommendation), indicating that our model can perform a balanced recommendation with both higher global and debiasing recommendation accuracy with greater long-tail item/user exposure.