OpenRaiser/Intern-Atlas huggingface dataset:
Search to Fine-tune Pre-trained Graph Neural Networks for Graph-level Tasks
Venue: arXiv cs.LG 2024
Year: 2024

Recently, graph neural networks (GNNs) have shown its unprecedented success in many graph-related tasks. However, GNNs face the label scarcity issue as other neural networks do. Thus, recent efforts try to pre-train GNNs on a large-scale unlabeled graph and adapt the knowledge from the unlabeled graph to the target downstream task. The adaptation is generally achieved by fine-tuning the pre-trained GNNs with a limited number of labeled data. Despite the importance of fine-tuning, current GNNs pre-training works often ignore designing a good fine-tuning strategy to better leverage transferred knowledge and improve the performance on downstream tasks. Only few works start to investigate a better fine-tuning strategy for pre-trained GNNs. But their designs either have strong assumptions or overlook the data-aware issue for various downstream datasets. Therefore, we aim to design a better fine-tuning strategy for pre-trained GNNs to improve the model performance in this paper. Given a pre-trained GNN, we propose to search to fine-tune pre-trained graph neural networks for graph-level tasks (S2PGNN), which adaptively design a suitable fine-tuning framework for the given labeled data on the downstream task. To ensure the improvement brought by searching fine-tuning strategy, we carefully summarize a proper search space of fine-tuning framework that is suitable for GNNs. The empirical studies show that S2PGNN can be implemented on the top of 10 famous pre-trained GNNs and consistently improve their performance. Besides, S2PGNN achieves better performance than existing fine-tuning strategies within and outside the GNN area. Our code is publicly available at \url{https://anonymous.4open.science/r/code_icde2024-A9CB/}.
Online Class Cover Problem
Venue: arXiv cs.CG 2024
Year: 2024

In this paper, we study the online class cover problem where a (finite or infinite) family $\cal F$ of geometric objects and a set ${\cal P}_r$ of red points in $\mathbb{R}^d$ are given a prior, and blue points from $\mathbb{R}^d$ arrives one after another. Upon the arrival of a blue point, the online algorithm must make an irreversible decision to cover it with objects from $\cal F$ that do not cover any points of ${\cal P}_r$. The objective of the problem is to place a minimum number of objects. When $\cal F$ consists of axis-parallel unit squares in $\mathbb{R}^2$, we prove that the competitive ratio of any deterministic online algorithm is $Ω(\log |{\cal P}_r|)$, and also propose an $O(\log |{\cal P}_r|)$-competitive deterministic algorithm for the problem.
Language is All a Graph Needs
Venue: arXiv cs.CL 2024
Year: 2024

The emergence of large-scale pre-trained language models has revolutionized various AI research domains. Transformers-based Large Language Models (LLMs) have gradually replaced CNNs and RNNs to unify fields of computer vision and natural language processing. Compared with independent data samples such as images, videos or texts, graphs usually contain rich structural and relational information. Meanwhile, language, especially natural language, being one of the most expressive mediums, excels in describing complex structures. However, existing work on incorporating graph problems into the generative language modeling framework remains very limited. Considering the rising prominence of LLMs, it becomes essential to explore whether LLMs can also replace GNNs as the foundation model for graphs. In this paper, we propose InstructGLM (Instruction-finetuned Graph Language Model) with highly scalable prompts based on natural language instructions. We use natural language to describe multi-scale geometric structure of the graph and then instruction finetune an LLM to perform graph tasks, which enables Generative Graph Learning. Our method surpasses all GNN baselines on ogbn-arxiv, Cora and PubMed datasets, underscoring its effectiveness and sheds light on generative LLMs as new foundation model for graph machine learning. Our code is open-sourced at https://github.com/agiresearch/InstructGLM.
MM-GEF: Multi-modal representation meet collaborative filtering
Venue: arXiv cs.IR 2024
Year: 2024

In modern e-commerce, item content features in various modalities offer accurate yet comprehensive information to recommender systems. The majority of previous work either focuses on learning effective item representation during modelling user-item interactions, or exploring item-item relationships by analysing multi-modal features. Those methods, however, fail to incorporate the collaborative item-user-item relationships into the multi-modal feature-based item structure. In this work, we propose a graph-based item structure enhancement method MM-GEF: Multi-Modal recommendation with Graph Early-Fusion, which effectively combines the latent item structure underlying multi-modal contents with the collaborative signals. Instead of processing the content feature in different modalities separately, we show that the early-fusion of multi-modal features provides significant improvement. MM-GEF learns refined item representations by injecting structural information obtained from both multi-modal and collaborative signals. Through extensive experiments on four publicly available datasets, we demonstrate systematical improvements of our method over state-of-the-art multi-modal recommendation methods.
EMID: An Emotional Aligned Dataset in Audio-Visual Modality
Venue: arXiv cs.MM 2024
Year: 2024

In this paper, we propose Emotionally paired Music and Image Dataset (EMID), a novel dataset designed for the emotional matching of music and images, to facilitate auditory-visual cross-modal tasks such as generation and retrieval. Unlike existing approaches that primarily focus on semantic correlations or roughly divided emotional relations, EMID emphasizes the significance of emotional consistency between music and images using an advanced 13-dimension emotional model. By incorporating emotional alignment into the dataset, it aims to establish pairs that closely align with human perceptual understanding, thereby raising the performance of auditory-visual cross-modal tasks. We also design a supplemental module named EMI-Adapter to optimize existing cross-modal alignment methods. To validate the effectiveness of the EMID, we conduct a psychological experiment, which has demonstrated that considering the emotional relationship between the two modalities effectively improves the accuracy of matching in abstract perspective. This research lays the foundation for future cross-modal research in domains such as psychotherapy and contributes to advancing the understanding and utilization of emotions in cross-modal alignment. The EMID dataset is available at https://github.com/ecnu-aigc/EMID.
Whale Detection Enhancement through Synthetic Satellite Images
Venue: arXiv cs.CV 2024
Year: 2024

With a number of marine populations in rapid decline, collecting and analyzing data about marine populations has become increasingly important to develop effective conservation policies for a wide range of marine animals, including whales. Modern computer vision algorithms allow us to detect whales in images in a wide range of domains, further speeding up and enhancing the monitoring process. However, these algorithms heavily rely on large training datasets, which are challenging and time-consuming to collect particularly in marine or aquatic environments. Recent advances in AI however have made it possible to synthetically create datasets for training machine learning algorithms, thus enabling new solutions that were not possible before. In this work, we present a solution - SeaDroneSim2 benchmark suite, which addresses this challenge by generating aerial, and satellite synthetic image datasets to improve the detection of whales and reduce the effort required for training data collection. We show that we can achieve a 15% performance boost on whale detection compared to using the real data alone for training, by augmenting a 10% real data. We open source both the code of the simulation platform SeaDroneSim2 and the dataset generated through it.
Expressivity of Graph Neural Networks Through the Lens of Adversarial Robustness
Venue: arXiv cs.LG 2024
Year: 2024

We perform the first adversarial robustness study into Graph Neural Networks (GNNs) that are provably more powerful than traditional Message Passing Neural Networks (MPNNs). In particular, we use adversarial robustness as a tool to uncover a significant gap between their theoretically possible and empirically achieved expressive power. To do so, we focus on the ability of GNNs to count specific subgraph patterns, which is an established measure of expressivity, and extend the concept of adversarial robustness to this task. Based on this, we develop efficient adversarial attacks for subgraph counting and show that more powerful GNNs fail to generalize even to small perturbations to the graph's structure. Expanding on this, we show that such architectures also fail to count substructures on out-of-distribution graphs.
On the Effectiveness of Log Representation for Log-based Anomaly Detection
Venue: arXiv cs.SE 2024
Year: 2024

Logs are an essential source of information for people to understand the running status of a software system. Due to the evolving modern software architecture and maintenance methods, more research efforts have been devoted to automated log analysis. In particular, machine learning (ML) has been widely used in log analysis tasks. In ML-based log analysis tasks, converting textual log data into numerical feature vectors is a critical and indispensable step. However, the impact of using different log representation techniques on the performance of the downstream models is not clear, which limits researchers and practitioners' opportunities of choosing the optimal log representation techniques in their automated log analysis workflows. Therefore, this work investigates and compares the commonly adopted log representation techniques from previous log analysis research. Particularly, we select six log representation techniques and evaluate them with seven ML models and four public log datasets (i.e., HDFS, BGL, Spirit and Thunderbird) in the context of log-based anomaly detection. We also examine the impacts of the log parsing process and the different feature aggregation approaches when they are employed with log representation techniques. From the experiments, we provide some heuristic guidelines for future researchers and developers to follow when designing an automated log analysis workflow. We believe our comprehensive comparison of log representation techniques can help researchers and practitioners better understand the characteristics of different log representation techniques and provide them with guidance for selecting the most suitable ones for their ML-based log analysis workflow.
Lifted Algorithms for Symmetric Weighted First-Order Model Sampling
Venue: arXiv cs.AI 2024
Year: 2024

Weighted model counting (WMC) is the task of computing the weighted sum of all satisfying assignments (i.e., models) of a propositional formula. Similarly, weighted model sampling (WMS) aims to randomly generate models with probability proportional to their respective weights. Both WMC and WMS are hard to solve exactly, falling under the $\#\mathsf{P}$-hard complexity class. However, it is known that the counting problem may sometimes be tractable, if the propositional formula can be compactly represented and expressed in first-order logic. In such cases, model counting problems can be solved in time polynomial in the domain size, and are known as domain-liftable. The following question then arises: Is it also the case for weighted model sampling? This paper addresses this question and answers it affirmatively. Specifically, we prove the domain-liftability under sampling for the two-variables fragment of first-order logic with counting quantifiers in this paper, by devising an efficient sampling algorithm for this fragment that runs in time polynomial in the domain size. We then further show that this result continues to hold even in the presence of cardinality constraints. To empirically verify our approach, we conduct experiments over various first-order formulas designed for the uniform generation of combinatorial structures and sampling in statistical-relational models. The results demonstrate that our algorithm outperforms a start-of-the-art WMS sampler by a substantial margin, confirming the theoretical results.
Pattern recognition using spiking antiferromagnetic neurons
Venue: arXiv cs.NE 2024
Year: 2024

Spintronic devices offer a promising avenue for the development of nanoscale, energy-efficient artificial neurons for neuromorphic computing. It has previously been shown that with antiferromagnetic (AFM) oscillators, ultra-fast spiking artificial neurons can be made that mimic many unique features of biological neurons. In this work, we train an artificial neural network of AFM neurons to perform pattern recognition. A simple machine learning algorithm called spike pattern association neuron (SPAN), which relies on the temporal position of neuron spikes, is used during training. In under a microsecond of physical time, the AFM neural network is trained to recognize symbols composed from a grid by producing a spike within a specified time window. We further achieve multi-symbol recognition with the addition of an output layer to suppress undesirable spikes. Through the utilization of AFM neurons and the SPAN algorithm, we create a neural network capable of high-accuracy recognition with overall power consumption on the order of picojoules.
Multi-Level Compositional Reasoning for Interactive Instruction Following
Venue: arXiv cs.RO 2024
Year: 2024

Robotic agents performing domestic chores by natural language directives are required to master the complex job of navigating environment and interacting with objects in the environments. The tasks given to the agents are often composite thus are challenging as completing them require to reason about multiple subtasks, e.g., bring a cup of coffee. To address the challenge, we propose to divide and conquer it by breaking the task into multiple subgoals and attend to them individually for better navigation and interaction. We call it Multi-level Compositional Reasoning Agent (MCR-Agent). Specifically, we learn a three-level action policy. At the highest level, we infer a sequence of human-interpretable subgoals to be executed based on language instructions by a high-level policy composition controller. At the middle level, we discriminatively control the agent's navigation by a master policy by alternating between a navigation policy and various independent interaction policies. Finally, at the lowest level, we infer manipulation actions with the corresponding object masks using the appropriate interaction policy. Our approach not only generates human interpretable subgoals but also achieves 2.03% absolute gain to comparable state of the arts in the efficiency metric (PLWSR in unseen set) without using rule-based planning or a semantic spatial memory.
STAR-RIS Aided MISO SWIPT-NOMA System with Energy Buffer: Performance Analysis and Optimization
Venue: arXiv cs.IT 2024
Year: 2024

In this paper, we propose a simultaneous transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) and energy buffer aided multiple-input single-output (MISO) simultaneous wireless information and power transfer (SWIPT) non-orthogonal multiple access (NOMA) system, which consists of a STAR-RIS, an access point (AP), and reflection users and transmission users with energy buffers. In the proposed system, the multi-antenna AP can transmit information and energy to several single-antenna reflection and transmission users simultaneously by the NOMA fashion in the downlink, where the power transfer and information transmission states of the users are modeled using Markov chains. The reflection and transmission users harvest and store the energy in energy buffers as additional power supplies, which are partially utilized for uplink information transmission. The power outage probability, information outage probability, sum throughput, and joint outage probability closed-form expressions of the proposed system are derived over Nakagami-m fading channels, which are validated via simulations. Results demonstrate that the proposed system achieves better performance as compared to the proposed system with discrete phase shifts, the STAR-RIS aided MISO SWIPT-NOMA buffer-less, conventional RIS and energy buffer aided MISO SWIPT-NOMA, and STAR-RIS and energy buffer aided MISO SWIPT-time-division multiple access (TDMA) systems. Furthermore, a particle swarm optimization-based power allocation (PSO-PA) algorithm is designed to maximize the uplink sum throughput with a constraint on the uplink joint outage probability and Jain's fairness index (JFI). Simulation results illustrate that the proposed PSO-PA algorithm can achieve an improved sum throughput performance of the proposed system.
Agreement theorems for high dimensional expanders in the small soundness regime: the role of covers
Venue: arXiv cs.CC 2024
Year: 2024

Given a family $X$ of subsets of $[n]$ and an ensemble of local functions $\{f_s:s\toΣ\; | \; s\in X\}$, an agreement test is a randomized property tester that is supposed to test whether there is some global function $G:[n]\toΣ$ such that $f_s=G|_s$ for many sets $s$. A "classical" small-soundness agreement theorem is a list-decoding $(LD)$ statement, saying that \[\tag{$LD$} Agree(\{f_s\}) > \varepsilon \quad \Longrightarrow \quad \exists G^1,\dots, G^\ell,\quad P_s[f_s\overset{0.99}{\approx}G^i|_s]\geq poly(\varepsilon),\;i=1,\dots,\ell. \] Such a statement is motivated by PCP questions and has been shown in the case where $X=\binom{[n]}k$, or where $X$ is a collection of low dimensional subspaces of a vector space. In this work we study small the case of on high dimensional expanders $X$. It has been an open challenge to analyze their small soundness behavior. Surprisingly, the small soundness behavior turns out to be governed by the topological covers of $X$.We show that: 1. If $X$ has no connected covers, then $(LD)$ holds, provided that $X$ satisfies an additional expansion property. 2. If $X$ has a connected cover, then $(LD)$ necessarily fails. 3. If $X$ has a connected cover (and assuming the additional expansion property), we replace the $(LD)$ by a weaker statement we call lift-decoding: \[ \tag{$LFD$} Agree(\{f_s\})> \varepsilon \Longrightarrow \quad \exists\text{ cover }ρ:Y\twoheadrightarrow X,\text{ and }G:Y(0)\toΣ,\text{ such that }\] \[P_{\tilde s\twoheadrightarrow s}[f_s \overset{0.99}{\approx} G|_{\tilde s}] \geq poly(\varepsilon),\] where ${\tilde s\twoheadrightarrow s}$ means that $ρ(\tilde s)=s$. The additional expansion property is cosystolic expansion of a complex derived from $X$ holds for the spherical building and for quotients of the Bruhat-Tits building.
When Stochastic Rewards Reduce to Deterministic Rewards in Online Bipartite Matching
Venue: arXiv cs.DS 2024
Year: 2024

We study the problem of vertex-weighted online bipartite matching with stochastic rewards where matches may fail with some known probability and the decision maker has to adapt to the sequential realization of these outcomes. Recent works have studied several special cases of this problem and it was known that the (randomized) Perturbed Greedy algorithm due to Aggarwal et al. (SODA, 2011) achieves the best possible competitive ratio guarantee of $(1-1/e)$ in some cases. We give a simple proof of these results by reducing (special cases of) the stochastic rewards problem to the deterministic setting of online bipartite matching (Karp, Vazirani, Vazirani (STOC, 1990)). More broadly, our approach gives conditions under which it suffices to analyze the competitive ratio of an algorithm for the simpler setting of deterministic rewards in order to obtain a competitive ratio guarantee for stochastic rewards. The simplicity of our approach reveals that the Perturbed Greedy algorithm has a competitive ratio of $(1-1/e)$ even in certain settings with correlated rewards, where no results were previously known. Finally, we show that without any special assumptions, the Perturbed Greedy algorithm has a competitive ratio strictly less than $(1-1/e)$ for vertex-weighted online matching with stochastic rewards.
StROL: Stabilized and Robust Online Learning from Humans
Venue: arXiv cs.RO 2024
Year: 2024

Robots often need to learn the human's reward function online, during the current interaction. This real-time learning requires fast but approximate learning rules: when the human's behavior is noisy or suboptimal, current approximations can result in unstable robot learning. Accordingly, in this paper we seek to enhance the robustness and convergence properties of gradient descent learning rules when inferring the human's reward parameters. We model the robot's learning algorithm as a dynamical system over the human preference parameters, where the human's true (but unknown) preferences are the equilibrium point. This enables us to perform Lyapunov stability analysis to derive the conditions under which the robot's learning dynamics converge. Our proposed algorithm (StROL) uses these conditions to learn robust-by-design learning rules: given the original learning dynamics, StROL outputs a modified learning rule that now converges to the human's true parameters under a larger set of human inputs. In practice, these autonomously generated learning rules can correctly infer what the human is trying to convey, even when the human is noisy, biased, and suboptimal. Across simulations and a user study we find that StROL results in a more accurate estimate and less regret than state-of-the-art approaches for online reward learning. See videos and code here: https://github.com/VT-Collab/StROL_RAL
Flamingo: Multi-Round Single-Server Secure Aggregation with Applications to Private Federated Learning
Venue: arXiv cs.CR 2024
Year: 2024

This paper introduces Flamingo, a system for secure aggregation of data across a large set of clients. In secure aggregation, a server sums up the private inputs of clients and obtains the result without learning anything about the individual inputs beyond what is implied by the final sum. Flamingo focuses on the multi-round setting found in federated learning in which many consecutive summations (averages) of model weights are performed to derive a good model. Previous protocols, such as Bell et al. (CCS '20), have been designed for a single round and are adapted to the federated learning setting by repeating the protocol multiple times. Flamingo eliminates the need for the per-round setup of previous protocols, and has a new lightweight dropout resilience protocol to ensure that if clients leave in the middle of a sum the server can still obtain a meaningful result. Furthermore, Flamingo introduces a new way to locally choose the so-called client neighborhood introduced by Bell et al. These techniques help Flamingo reduce the number of interactions between clients and the server, resulting in a significant reduction in the end-to-end runtime for a full training session over prior work. We implement and evaluate Flamingo and show that it can securely train a neural network on the (Extended) MNIST and CIFAR-100 datasets, and the model converges without a loss in accuracy, compared to a non-private federated learning system.