Title: Adjusted Count Quantification Learning on Graphs
Venue: NeurIPS 2025
Accepted: yes
Keywords: quantification learning

graph neural networks

distribution shift
Abstract: *Quantification learning* is the task of predicting the label distribution of a set of instances.
We study this problem in the context of graph-structured data, where the instances are vertices.
Previously, this problem has only been addressed via node clustering methods.
In this paper, we extend the popular *Adjusted Classify & Count* (ACC) method to graphs.
We show that the prior probability shift assumption upon which ACC relies is often not applicable to graph quantification problems.
To address this issue, we propose structural importance sampling (SIS), the first graph quantification method that is applicable under (structural) covariate shift.
Additionally, we propose Neighborhood-aware ACC, which improves quantification in the presence of non-homophilic edges.
We show the effectiveness of our techniques on multiple graph quantification tasks.

Adjusted Count Quantification Learning on Graphs

Abstract

Quantification learning is the task of predicting the label distribution of a set of instances. We study this problem in the context of graph-structured data, where the instances are vertices. Previously, this problem has only been addressed via node clustering methods. In this paper, we extend the popular Adjusted Classify & Count (ACC) method to graphs. We show that the prior probability shift assumption upon which ACC relies is often not applicable to graph quantification problems. To address this issue, we propose structural importance sampling (SIS), the first graph quantification method that is applicable under (structural) covariate shift. Additionally, we propose Neighborhood-aware ACC , which improves quantification in the presence of non-homophilic edges. We show the effectiveness of our techniques on multiple graph quantification tasks.

1 Introduction

We consider the task of quantification learning (QL) on graph-structured data. This term was first coined by Forman [8, 9, 10] and is used to describe the task of estimating label prevalences via supervised learning. A QL method receives a set of training instances with known labels, which is used to train a quantifier. The quantifier is then used to predict the label distribution of a set of test instances. Unlike standard instance-wise classification, QL does not concern itself with predicting an accurate label for each test instance but rather with predicting the overall prevalence of each label across all instances. QL can thus be seen as a dataset-level prediction task, where a single prediction is made for a population of instances.

Quantification problems naturally arise in polling and surveying, where the goal is to estimate the proportion of a population that has a certain property or holds a certain opinion. Examples include estimating the proportion of voters who support a certain political party or the proportion of customers who are satisfied with a product. Similarly, QL can be applied to epidemiology or ecological modelling to estimate the prevalence of diseases or species in a given population. We refer to Esuli et al. [5] for a comprehensive overview of the applications of quantification.

Typically, QL is studied in the context of tabular data, where each instance x ∈ X = R d is represented by a feature vector. In this setting, instances are assumed to be independent, i.e., the label distribution P ( Y | X = x ) is fully determined by the instance x . However, in many real-world applications, this independence assumption does not hold. Consider the example of estimating the proportion of voters supporting a certain party. Assume we have access to a social network where each node represents a voter and each edge represents a social connection. In this case, the label distribution of a voter, i.e., their political preferences, may depend not only on their own features but also on the features of their social connections. Incorporating this relational information into the quantification process can lead to more accurate estimates.

Generally speaking, QL methods can be divided into two categories: aggregative and nonaggregative. Aggregative quantifiers rely upon an instance-wise label estimator, i.e., a regular classifier; the instance-level label estimates are then aggregated to obtain dataset-level label prevalence

estimates. Non-aggregative quantifiers, on the other hand, directly estimate dataset-level label prevalences without first predicting labels for each instance. In this paper, we focus on aggregative quantification methods, which are more common and have been studied more extensively. An intuitively plausible aggregative method is to simply estimate the prevalence of a label as the fraction of test instances that are predicted to belong to that label by the classifier. This method is known as Classify &Count (CC) and, given a perfect classifier, it will yield perfect quantification results. However, in practice, classifiers are not perfect, and even a good but not perfect classifier can lead to poor quantification results. Conversely, even a bad classifier can yield good quantification results. The reason for this disconnect is that the optimization goals of classification and quantification are misaligned. More specifically, while a good binary classifier should minimize the total number of misclassifications, i.e., (FP + FN) , a good binary quantifier should minimize | FP -FN | . If FP = FN , even a classifier with a high misclassification rate will yield perfect quantification results.

This misalignment is commonly addressed by the family of Adjusted Classify & Count (ACC) methods, which use an estimate of the classifier's confusion matrix to adjust the predicted label prevalences [35, 29, 8]. ACC has been shown to estimate the true test label prevalences in expectation if the so-called prior probability shift (PPS) assumption holds [33].

In this paper, we investigate ACC in the context of graph-structured data and describe why it is oftentimes ill-suited to tackle graph quantification problems. To solve this problem, we propose two novel methods for graph quantification learning: Structural importance sampling (SIS) and Neighborhood-aware ACC (NACC). First, SIS generalizes ACC to the (structural) covariate shift setting; to our knowledge, this is the first quantification method that tackles covariate shift systematically. Second, NACC further improves the quantification performance of ACC in the graph domain by improving class identifiability in the presence of non-homophilic edges. We begin with a brief formal description of the general quantification problem in Section 2. In Section 3 we then consider graph quantification and introduce our novel SIS and NACC methods. In Section 4, the proposed methods are evaluated on a series of datasets under different shift assumptions. Last, we conclude with a brief outlook in Section 5.

2 Quantification Learning

Let X denote the instance space and Y = { 1 , . . . , K } the (finite) label space. In QL we assume to be given a training set of labeled instances D L ⊆ X × Y drawn from a distribution P with corresponding density p . Additionally, there is a set of labeled instances D U ⊆ X × Y drawn from a test distribution Q with corresponding density q . Let X and Y denote random variable s (RVs) that project the joint instance-label space to the instance and label spaces, respectively. The goal of QL is to estimate the pushforward measure Q ( Y ) given samples D L and X U := { x | ( x, y ) ∈ D U } . If P = Q , i.e., if the training and test data are drawn from the same distribution, the quantification problem is trivially solved via a maximum likelihood estimate of the label distribution on D L :

where 1 [ · ] denotes the indicator function. This Maximum Likelihood Prevalence Estimation (MLPE) approach [1, 5] is akin to the majority classifier in classification in the sense that it predicts the most likely distribution in the absence of test data X U . However, if the training and test data are not identically distributed, the quantification problem becomes more challenging. A quantification approach has to account for the distribution shift between P and Q to provide accurate estimates of Q ( Y ) . Depending on the nature of this distribution shift, different quantification methods may be more or less suitable.

2.1 Types of Distribution Shift

If the train and test distributions differ, one should ask whether learning from the training data is still feasible. Certainly, if P and Q are completely unrelated, any information learned from D L is useless for predicting Q ( Y ) . Quantification approaches, therefore, typically assume that P and Q are related in some way. The applicability of a quantification method then depends on whether those assumptions hold true for the given problem. First, note that Q can be expressed as

By fixing one of the factors in the two right-hand terms, we obtain three types of distribution shifts [5]:

̸

Concept Shift: The conditional label distribution changes, but the distribution of the instances remains the same, i.e., Q ( Y | X ) = P ( Y | X ) , while Q ( X ) = P ( X ) . This type of shift, also referred to as concept drift , can occur in domains with classes that are defined relative to some frame of reference.

̸

Covariate Shift: The distribution of the instances changes, but the conditional label distribution remains the same, i.e., Q ( X ) = P ( X ) , while Q ( Y | X ) = P ( Y | X ) . This is common in domain adaptation, where the training and test data are drawn from different but related domains. For example, assume the task is to predict the prevalence of a certain sentiment or opinion in social media posts. The training data may be drawn from one social media platform, while the test data is drawn from another. Given a post x , the probability of it expressing a certain sentiment y is likely the same on both platforms, but the distribution of posts may differ.

̸

Prior Probability Shift: The label distribution changes, but not the class-conditional instance distribution, i.e., Q ( Y ) = P ( Y ) , while Q ( X | Y ) = P ( X | Y ) . Similar to covariate shift, prior probability shift (PPS) occurs between domains that share the same label concepts. For example, consider the task of predicting the percentage of a population that has a certain disease. The training data may come from a case-control study consisting of an equal proportion of healthy and infected individuals, while the test data is drawn from the general population. Given y ∈ { infected , healthy } , the feature distribution of an individual x should be the same between training and test, whereas the prevalence of the disease will likely not be.

We do not consider the case where Q ( Y ) = P ( Y ) , as this would imply that the label distribution remains unchanged, in which case the quantification problem is trivially solved by MLPE. Note that the difference between covariate shift and PPS is subtle. Whether it is P ( X ) or P ( Y ) that changes between training and test is mostly a matter of the assumed causal relation between instances and labels, i.e., whether it is in the direction X → Y or Y → X [6, 30, 18]. In QL, PPS is commonly assumed, as there are many Y → X domains in which this is reasonable [15]. Generally speaking, quantification under concept or covariate shift is more challenging and often requires additional assumptions or domain knowledge. We will get back to the question of which shift assumptions are appropriate for a given domain in Section 3.

2.2 Adjusted Count

We will now describe the Adjusted Classify & Count (ACC) method, a popular approach to quantification under PPS [8]. As mentioned in the introduction, the naive Classify & Count method (incorrectly) assumes that the predicted label prevalences of a classifier h : X → Y equal the true label prevalences, i.e., that Q ( ˆ Y ) = Q ( Y ) , where ˆ Y = h ( X ) is a RV representing the predicted label. Under this assumption, the label prevalences can be estimated as

However, since h is trained on data drawn from P , the estimated propensity scores ˆ Q CC ( Y ) will be biased towards P ( Y ) in practice. ACC removes this bias by adjusting the predicted label prevalences based on an estimate of the classifier's confusion matrix. To understand ACC, note that the PPS assumption implies that for any measurable mapping ϕ : X → Z , we have P ( Z | Y ) = Q ( Z | Y ) where Z = ϕ ( X ) [19]. This allows us to factorize Q ( Z ) as follows:

Let ϕ = h , i.e., Z = ˆ Y ; then, we can plug the estimate of Q ( ˆ Y ) from Eq. (2) and the estimate

of Q ( ˆ Y | Y ) into Eq. (3). This yields a system of equations which can be solved to obtain estimates of Q ( Y ) [29]. Let ˆ C ∈ [0 , 1] K × K be the estimated confusion matrix of the classifier h on Q , i.e., ˆ C j,i = ˆ Q ( ˆ Y = j | Y = i ) . Then, the ACC estimates of Q ( Y ) are given by

While the binary version of ACC goes back at least to Gart and Buck [11], it was first described as a quantification method by Vucetic and Obradovic [35]. Tasche [33] showed that ACC is an unbiased estimator of the true test label prevalences if the PPS assumption holds.

Note that there are two practical problems with Eq. (5): First, if C is not invertible, there might be no or multiple solutions for ˆ Q ACC ( Y ) . Second, the adjusted label prevalences may not be a valid distribution over Y , i.e., they could lie outside [0 , 1] or not sum to one. Possible reasons for this are that the PPS assumption might not be (fully) satisfied or simply that the estimates ˆ C and ˆ Q ( ˆ Y ) are noisy, e.g., due to small sample sizes. A number of solutions to these problems have been proposed in the literature, including clipping and rescaling the estimates [10], adjusting the confusion matrix [19], using the pseudo-inverse of C , or replacing the system of equations with a constrained optimization problem [3]. In this work, we will use the latter approach, i.e., constrained optimization, to solve Eq. (5):

where ∆ K denotes the unit ( K -1) -simplex. This problem can be solved numerically, e.g., using a (quasi-)Newtonian method such as Sequential Least Squares Quadratic Programming . Bunse [3] has shown that this approach is a sensible default choice, as it generally performs well in practice.

In addition to the CC and ACC methods described above, which use a hard classifier h : X → Y , one can also use a probabilistic classifier h s : X → ∆ K [2]. Analogous to CC, Probabilistic Classify &Count (PCC) is defined as

Likewise, Probabilistic Adjusted Classify & Count (PACC) estimates Q ( ˆ Y ) and P ( ˆ Y | Y ) using predicted label probabilities. The motivation for using a soft classifier instead of a hard one is that predicted label probabilities can be more informative than hard labels. Whether this is truly the case is problem-dependent and depends on the quality of the predicted probabilities.

3 Graph Quantification Learning

We now turn to the problem of quantification learning on graph-structured data. In Section 2, we assumed that the instances in D L and D U are i.i.d. wrt. P and Q respectively. This assumption does not hold for graph-structured data, where the instances are the vertices of a graph and the labels are associated with the vertices. More specifically, let G = ( V , E ) be a graph with vertex set V and edge set E ⊆ V × V . Each vertex v i ∈ V is associated with a feature vector x i ∈ X and a label y i ∈ Y . We use N ( v i ) = { v j | ( v i , v j ) ∈ E} to denote the set of neighbors of v i . The edges in G are used to encode homophily between vertices, i.e., similar vertices are more likely to be connected. Formally, an edge ( v i , v j ) ∈ E should indicate that P ( y i = y j ) ≥ ε , with ε being either a graph-specific constant or a function of an edge weight w i,j ∈ R . Since homophily is symmetric by definition, G is undirected, i.e., ( v i , v j ) ∈ E ⇔ ( v j , v i ) ∈ E . Such homophilic graphs are commonly used to represent social networks, citation networks, co-purchase graphs, or the World Wide Web. Figure 1 shows one such graph, namely the Amazon Photos co-purchase graph [32], where vertices represent products, edges indicate that two products are frequently bought together, and labels represent product categories. Due to homophily, the product categories form separate densely connected clusters, while cross-category edges are sparse.

Analogous to the tabular case, in graph quantification learning (GQL) we are given a training set of labeled vertices D L ⊆ V × Y drawn from a distribution P and our goal is to estimate the label distribution of the vertices in a test set V U drawn from a distribution Q ( V ) , with V denoting a RV mapping the joint measure space V × Y to V . Given some vertex classifier h : V → Y , the GQL

2

3

4

5

8

problem is, in principle, amenable to standard aggregative quantification methods, such as ACC or PACC. As discussed in Section 2.2, those adjusted count methods assume PPS, which in turn assumes a Y → V domain. This means that, both, the training and the test data are assumed to be generated by sampling from some fixed distribution P ( V | Y = i ) for all i ∈ Y . We argue that this is often not realistic for graph-structured data.

Consider the example of estimating the proportion of users holding a certain opinion. Here, the training data D L may come from a social network where a (non-representative) local subset of users was sampled. The test data D U , on the other hand, may come from the entire social network or possibly some local subcluster of interest. In this setting, it is the instance distribution P ( V ) that changes, while P ( Y | V ) remains fixed, i.e., covariate shift. More generally, for a sampling process that is structure dependent, for example, by sampling local training or test neighborhoods, the covariate shift assumption is more appropriate than PPS. We will now discuss how such structural biases can be accounted for in the quantification process.

3.1 Structural Importance Sampling

ACC depends on being able to estimate the test confusion matrix C from training data. As described, this is trivial under PPS. We will now introduce structural importance sampling (SIS), a novel generalization of graph quantification learning to covariate shift. First, note that C j,i can be expressed as

with q V | Y = dQ ( V | Y ) dµ and p V | Y = dP ( V | Y ) dµ denoting the Radon-Nikodym derivatives of Q and P and µ a (counting) measure on V , i.e., their probability density function s (PDFs), and ρ V | Y denoting the ratio between those PDFs. Using the covariate shift assumption, we get

Thus, C can be obtained by reweighting the vertices:

Given D L , we can obtain an unbiased estimate of C j,i = Q ( ˆ Y = j | Y = i ) :

Note that this is essentially a weighted version of Eq. (4). The problem with this formulation is that it requires ρ V = q V p V , which cannot be computed since, both, P ( V ) and Q ( V ) are unknown. We do, however, have access to samples from both distributions, i.e., D L and V U . Using suitable vertex kernels k q , k p : V × V → R , we can thus obtain estimate of the PDFs via kernel density estimation:

The suitability of the kernels depends on the nature of the distribution shift. Intuitively, k q ( v, v ′ ) and k p ( v, v ′ ) should be proportional to the probability of sampling a vertex v from Q ( V ) and P ( V ) , respectively, given that v ′ has been sampled. For example, the constant kernel k 1 ( v, v ′ ) = 1 describes a sampling process where the probability of sampling v is independent of v ′ . Using k q = k p = k 1 , Eq. (11) simplifies to standard ACC (cf. Eq. (4)), i.e., SIS is a generalization of ACC.

Under (structural) covariate shift, k q and k p should be chosen in accordance with the sampling process. Since the family of structural shifts is broad, there is no single, generally applicable kernel. Nonetheless, if the shift is induced by a localized sampling process where vertices are sampled via random walk s (RWs), the probability of sampling v given v ′ is proportional to the number of RWs between both vertices. This probability can be computed via the personalized page-rank (PPR) algorithm [26]:

Here, ¯ A = AD -1 is the normalized adjacency matrix of the graph, α ∈ (0 , 1) is a teleportation parameter and L is the number of steps in the random walk. We found that k PPR is a good default for localized structural shifts. Nonetheless, depending on the problem domain, other choices might be more appropriate. A more in-depth discussion of the kernel selection can be found in Appendix B.1. Aformal analysis of the computational complexity of SIS under different kernel choices is provided in Appendices C.1 and C.2.

To summarize, SIS enables graph quantification under covariate shift by estimating Q ( ˆ Y | Y ) using kernel density estimates of the vertex distributions P ( V ) and Q ( V ) . Using these estimates, the adjusted label prevalences can be computed using Eq. (6).

3.2 Neighborhood-aware Adjusted Count

In the previous section, we extended GQL to covariate shift. Next, we will address another orthogonal problem of ACC: Class identifiability . Consider a classifier h that is unable to distinguish between two classes i and j , i.e., it predicts the same label for both. In this case, C : ,i = C : ,j ; thus, there is no unique solution for Eq. (5). This can lead to poor quantification results if the prediction vector Q ( ˆ Y ) has a large overlap with, both, C : ,i and C : ,j , since any distribution of probability mass between both classes may then be returned. To address this issue, we propose Neighborhood-aware ACC (NACC), which uses the neighborhood structure of the graph to improve class identifiability.

First, note that Eq. (6) can be understood as finding a mixture of the columns of C that best approximates Q ( ˆ Y ) . In the case of collinear columns, this mixture is not unique. A simple way to break such symmetries is to set Z = ˆ Y N in Eq. (3), where ˆ Y N is a RV representing a tuple of the predicted label of a vertex and the majority predicted label of its neighbors:

Using this decomposition of Q ( ˆ Y N ) , Q ( Y ) can be estimated using ACC and, possibly, SIS. Intuitively, this approach uses homophily information to improve class identifiability. Consider Fig. 1, where a vertex is highlighted if it is misclassified by an approximate personalized propagation of neural predictions (APPNP) classifier [12]. Note that the vertices with label 7 (dark green) are often confused with vertices of label 1 (blue) or 6 (orange) because there are many non-homophilic edges between those classes. Using ACC, this would imply that the row vectors of labels 7, 1 and 6 are collinear, i.e., C : , 7 ≈ α · C : , 1 +(1 -α ) · C : , 6 for some α ∈ [0 , 1] . Using the neighborhood structure, NACC can break this symmetry. For labels 1 and 6, the majority of predicted neighbors will

nearly always be of the same label due to homophily, whereas for label 7, both, ˆ Y N = (1 , 6) and ˆ Y N = (6 , 1) are common. With this information, NACC is able to distinguish the confusion profile of label 7 from those of labels 1 and 6.

In principle, one could extend NACC to use even more neighborhood information, e.g., by considering the majority label of the neighbors of neighbors or by considering the second-most predicted neighboring label. However, given a finite training set D L , by making the confusion profiles more fine-grained, the confusion estimate ˆ C will become noisier, counteracting the potential gains of additional information. We found that using the 1-hop majority label is a good trade-off between class identifiability and confusion estimate noise. An analysis of the computational complexity of NACC can be found in Appendix C.3.

4 Evaluation

We assess the performance of SIS and NACC on a series of graph quantification tasks using, both, PPS and covariate shift. The quantifiers are applied to the predictions of a set of node classifiers. As a baseline we compare our proposed GQL methods with MLPE, (P)CC and (P)ACC. We build upon the QuaPy Python library (BSD 3-Clause). Further details can be found in Appendix A.

4.1 Experimental Setup

Quantification Metrics There is a large number of metrics to evaluate quantification methods [5]. We use Absolute error (AE) and relative absolute error (RAE):

AE penalized all errors equally, whereas RAE [16] penalizes errors on rare labels more heavily.

Datasets Since the literature on GQL is scarce, there are no established benchmark datasets for this task. For this reason, we synthetically generate quantification tasks from the following five node classification datasets: 1. CoraML, 2. CiteSeer, 3. PubMed, 4. Amazon Photos and 5. Amazon Computers [21, 14, 13, 31, 25, 20, 32]. The first three datasets are citation networks, where the nodes are documents and the edges represent citations between them. The two Amazon datasets are product co-purchasing graphs, where the nodes are products and the edges represent that are often bought together. All nodes are labeled with the topic or product category they belong to. All reported results were obtained by averaging over 10 random splits of the node set into classifiertrain/quantifier-train/test, with sizes of 5% / 15% / 80% respectively.

Additionally, we conduct a study on the 'Twitch Gamers' dataset [28]; it consists of about 168k vertices and 6.8 million edges, where vertices represent Twitch accounts and edges represent followership relations. Each vertex is annotated with the language of the corresponding user, whether the user streams explicit content, and a number of other features. This social network dataset is a good source of real-world structural distribution shifts, since, for example, users from the same language community tend to form more densely connected subgraphs than users from different language communities (see Fig. 2). We select 10% of the users uniformly at random as classifier-train nodes with a binary target label indicating whether a given user streams explicit content. From the remaining nodes, another 10% are selected as quantifier-train nodes, i.e., as D L . Last, the then remaining nodes are partitioned into top-5 languages spoken by the users: English (74%), German (5%), French (4%), Spanish (3%) and Russian (2%); users outside of those languages are discarded. For each of the five partitions, the (binary) quantification task is to estimate the prevalence of explicit content streamers.

Distribution Shift For the synthetic quantification experiments, we introduce shifts in the test partitions, while the training data is sampled uniformly at random from the training split. We consider three types of test distribution shifts:

PPS: To simulate PPS, we first sample 10 · K target label distribution q ∈ ∆ K from a Zipf distribution over the labels [27]. For each sampled target distribution, we then sample 100 vertices such that the target label frequencies are reached.

DE

EN

other

other

FR

RU

ES

other

Structural covariate shift via breadth-first search (BFS): For each label, we select 10 corresponding vertices and starting at each of those vertices we sample 100 nodes via BFS.

Structural covariate shift via RWs: Analogous to the BFS setting, we sample 100 nodes via random walks of length 10 with teleportation parameter α = 0 . 1 .

For the real-world 'Twitch Gamers' dataset, no synthetic distribution shifts are introduced; instead we use the natural covariate shift occurring when partitioning users by language. This allows us to compare the applicability of the shift assumptions made by different quantification approaches.

Classifiers We use four different node classifiers to predict the labels of the vertices: A structureunaware Multilayer Perceptron (MLP) , a Graph Convolutional Network (GCN) [17], Graph Attention Network (GAT) [34], and APPNP [12]. All models are trained using the same training splits and hyperparameters, and two hidden layers/convolutions with widths of 64 and ReLU activations. Each model is trained ten times on each of the ten splits per dataset, totalling 100 models per dataset, with which each quantifier is evaluated. Additionally, we evaluate the previously proposed Ego-network Quantification (ENQ) GQL method by Milli et al. [22]. ENQ uses a simple neighborhood majority classifier combined with (standard) ACC; in addition, we evaluate it using NACC and SIS.

Quantifiers We evaluate SIS and NACC and compare them against standard (P)ACC and (P)CC. For SIS, we use k q = k λ , with k λ being an interpolated version of the PPR kernel from Eq. (13):

where λ ∈ [0 , 1] is a hyperparameter that controls the minimum weight that should be assigned to each vertex. In the following, we report results for λ = 0 . 9 ; evaluations with different λ parameterizations and an alternative shortest-path-based vertex kernel can be found in Appendix B. For the kernel density estimation (KDE) estimate of p V , we use the constant kernel k p = k 1 , since the training data is sampled uniformly at random without being subject to synthetic distribution shift in our setup. This implies ρ V = q V , simplifying the SIS estimation.

4.2 Discussion of Synthetic Distribution Shift Results

Table 1 compares the quantification performance of PCC, PACC and our extensions of the latter, i.e., SIS and neighborhood-aware PACC under synthetically induced distribution shifts. Additionally, the last block of columns shows the average rank of each quantifier across all datasets. Bold numbers indicate that there is no statistically significant difference between the reported mean and the best mean within a given block, determined by the 95th percentile of a one-sided t-test. Standard errors are not reported for visual clarity and space reasons, since they are mostly very close to zero.

In summary, considering the average ranks, our experiments show that both SIS and NACC are able to improve quantification results under PPS and covariate shift. The results are consistent across all classifiers, quantifiers, and types of distribution shifts, with either SIS or the combination of SIS and NACC performing best.

Influence of the Classifier Unsurprisingly, the choice of classifier has a significant impact on the quantification performance. Even though a good classifier h is not required by QL to obtain an unbiased estimate of the label prevalences, the quality of this estimate is still correlated with the classifier's accuracy. Overall, the naive neighborhood-based ENQ performs worst, followed by the structure-unaware MLP, while APPNP performs best.

Influence of the Type of Distribution Shift The SIS kernels used in the experiments are based on the assumption that the distribution shift is induced by sampling localized random walks. In the RW covariate setting, this assumption is satisfied by definition, while in the BFS setting, the PPR kernel does, at least in theory, not fully capture the underlying sampling behavior. Nonetheless, SIS is able to improve quantification results in both cases. Interestingly, even in the PPS setting, where SIS is not necessary to account for the shift, we observe a clear improvement over ACC.

Experimental results for different kernel choices and hyperparameter settings can be found in Appendix B. Additionally, in Appendix D, we demonstrate the importance of using a structural kernel by comparing against a feature-based variant of SIS that does not consider structural information.

4.3 Discussion of Real-world Covariate Shift Results

Table 2 shows the quantification results on the real-world 'Twitch Gamers' dataset. Here, the advantage of SIS and/or NACC over standard ACC and CC is less pronounced compared to the synthetic experiments. For the English language community, which also constitutes 74% of the training data, SIS performs worse than standard approaches - likely because there is only a small distribution shift compared to the training data for this community. However, for the other language communities, which are more strongly affected by distribution shift, SIS generally outperforms standard ACC and CC given an APPNP classifier. Using a weaker, structure-unaware MLP classifier, SIS is not able to significantly improve quantification results. This highlights the importance of a sufficiently accurate base classifier for effective aggregative quantification.

5 Conclusion

We have introduced two novel graph quantification methods, SIS and NACC; to our knowledge, this is the first work to investigate classifier-based graph quantification and the (structural) covariate shift problem. SIS enables quantification under covariate shift via kernel density estimates of the instance distributions. NACC uses the neighborhood structure of the graph to improve class identifiability. The effectiveness of our approach was demonstrated on multiple graph benchmark datasets.

We envision two lines of future research. First, in this work, we focused on extensions of ACC to the graph setting. Another family of methods are the so-called distribution matching quantifiers, e.g., DMy [16] or KDEy [24]. An extension of distribution matching approaches to GQL would be interesting. Second, there are no true graph quantification benchmark datasets currently, which is why we resorted to introducing synthetic dataset shifts to node classification benchmarks. Creating a true graph quantification dataset, e.g., using social media data, would be a valuable next step to assess the performance of SIS and NACC in the real-world.

Acknowledgments and Disclosure of Funding

We gratefully acknowledge the support of the Munich Center for Machine Learning (MCML), and the German Research Center for Artificial Intelligence (DFKI). We also thank the reviewers for their valuable and constructive feedback, which helped to improve our work.

References

Jose Barranquero, Pablo Gonz´ alez, Jorge D´ ıez, and Juan Jos´ e del Coz. On the study of nearest neighbor algorithms for prevalence estimation in binary problems. Pattern Recognition , 46(2): 472-482, 2013. ISSN 0031-3203. doi: 10.1016/j.patcog.2012.07.022.

Antonio Bella, Cesar Ferri, Jos´ e Hern´ andez-Orallo, and Mar´ ıa Jos´ e Ram´ ırez-Quintana. Quantification via Probability Estimators. In 2010 IEEE International Conference on Data Mining , pages 737-742, 2010. doi: 10.1109/ICDM.2010.75.

Mirko Bunse. On multi-class extensions of adjusted classify and count. In Proceedings of the 2nd International Workshop on Learning to Quantify (LQ 2022) , pages 43-50, 2022.

Clemens Damke and Eyke H¨ ullermeier. Linear Opinion Pooling for Uncertainty Quantification on Graphs. In The 40th Conference on Uncertainty in Artificial Intelligence , Barcelona, 2024.

Andrea Esuli, Alessandro Fabris, Alejandro Moreo, and Fabrizio Sebastiani. Learning to Quantify , volume 1 of The Information Retrieval Series . Springer, Cham, 2023. ISBN 978-3031-20467-8.

Tom Fawcett and Peter A. Flach. A Response to Webb and Ting's On the Application of ROC Analysis to Predict Classification Performance Under Varying Class Distributions. Machine Learning , 58(1):33-38, 2005. ISSN 1573-0565. doi: 10.1007/s10994-005-5256-4.

Matthias Fey and Jan Eric Lenssen. Fast Graph Representation Learning with PyTorch Geometric, 2019.

George Forman. Counting positives accurately despite inaccurate classification. In Proceedings of the 16th European Conference on Machine Learning , ECML'05, pages 564575, Berlin, Heidelberg, 2005. Springer-Verlag. ISBN 978-3-540-29243-2. doi: 10.1007/ 11564096 55.

George Forman. Quantifying trends accurately despite classifier error and class imbalance. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , KDD '06, pages 157-166, New York, NY, USA, 2006. Association for Computing Machinery. ISBN 978-1-59593-339-3. doi: 10.1145/1150402.1150423.

George Forman. Quantifying counts and costs via classification. Data Mining and Knowledge Discovery , 17(2):164-206, 2008. ISSN 1573-756X. doi: 10.1007/s10618-008-0097-y.

John J. Gart and Alfred A. Buck. Comparison of a screening test and a reference test in epidemiologic studies. II. A probabilistic model for the comparison of diagnostic tests. American Journal of Epidemiology , 83(3):593-602, 1966. ISSN 1476-6256, 0002-9262. doi: 10.1093/oxfordjournals.aje.a120610.

Johannes Gasteiger, Aleksandar Bojchevski, and Stephan G¨ unnemann. Predict then Propagate: Graph Neural Networks meet Personalized PageRank. In International Conference on Learning Representations , 2018.

Lise Getoor. Link-based Classification. In Sanghamitra Bandyopadhyay, Ujjwal Maulik, Lawrence B. Holder, and Diane J. Cook, editors, Advanced Methods for Knowledge Discovery from Complex Data , Advanced Information and Knowledge Processing, pages 189-207. Springer, London, 2005. ISBN 978-1-84628-284-3. doi: 10.1007/1-84628-284-5 7.

C. Lee Giles, Kurt D. Bollacker, and Steve Lawrence. CiteSeer: An automatic citation indexing system. In Proceedings of the Third ACM Conference on Digital Libraries , DL '98, pages 8998, New York, NY, USA, 1998. Association for Computing Machinery. ISBN 978-0-89791965-4. doi: 10.1145/276675.276685.

Pablo Gonz´ alez, Alejandro Moreo, and Fabrizio Sebastiani. Binary quantification and dataset shift: An experimental investigation. Data Mining and Knowledge Discovery , 38(4):16701712, 2024. ISSN 1573-756X. doi: 10.1007/s10618-024-01014-1.

V´ ıctor Gonz´ alez-Castro, Roc´ ıo Alaiz-Rodr´ ıguez, and Enrique Alegre. Class distribution estimation based on the Hellinger distance. Information Sciences , 218:146-164, 2013. ISSN 0020-0255. doi: 10.1016/j.ins.2012.05.028.

Thomas N. Kipf and Max Welling. Semi-Supervised Classification with Graph Convolutional Networks. In International Conference on Learning Representations , 2017.

Meelis Kull and Peter A. Flach. Patterns of dataset shift. 2014.

Zachary Lipton, Yu-Xiang Wang, and Alexander Smola. Detecting and Correcting for Label Shift with Black Box Predictors. In Proceedings of the 35th International Conference on Machine Learning , pages 3122-3130. PMLR, 2018.

Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton van den Hengel. Image-Based Recommendations on Styles and Substitutes. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval , SIGIR '15, pages 43-52, New York, NY, USA, 2015. Association for Computing Machinery. ISBN 978-1-45033621-5. doi: 10.1145/2766462.2767755.

Andrew Kachites McCallum, Kamal Nigam, Jason Rennie, and Kristie Seymore. Automating the Construction of Internet Portals with Machine Learning. Information Retrieval , 3(2):127163, 2000. ISSN 1573-7659. doi: 10.1023/A:1009953814988.

Letizia Milli, Anna Monreale, Giulio Rossetti, Dino Pedreschi, Fosca Giannotti, and Fabrizio Sebastiani. Quantification in social networks. In 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA) , pages 1-10, 2015. doi: 10.1109/DSAA.2015. 7344845.

Alejandro Moreo, Andrea Esuli, and Fabrizio Sebastiani. QuaPy: A Python-Based Framework for Quantification. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management , CIKM '21, pages 4534-4543, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 978-1-4503-8446-9. doi: 10.1145/3459637.3482015.

Alejandro Moreo, Pablo Gonz´ alez, and Juan Jos´ e del Coz. Kernel density estimation for multiclass quantification. Machine Learning , 114(4):1-38, 2025. ISSN 1573-0565. doi: 10.1007/s10994-024-06726-5.

Galileo Namata, Ben London, L. Getoor, and Bert Huang. Query-driven Active Surveying for Collective Classification. In Proceedings of the Workshop on Mining and Learning with Graphs (MLG-2012) , Edinburgh, Scotland, UK, 2012.

Lawrence Page, Sergey Brin, R. Motwani, and T. Winograd. The PageRank Citation Ranking : Bringing Order to the Web. In The Web Conference , 1999.

Lei Qi, Mohammed Khaleel, Wallapak Tavanapong, Adisak Sukul, and David Peterson. A Framework for Deep Quantification Learning. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2020, Ghent, Belgium, September 14-18, 2020, Proceedings, Part I , pages 232-248, Berlin, Heidelberg, 2020. Springer-Verlag. ISBN 978-3-030-67657-5. doi: 10.1007/978-3-030-67658-2 14.

Benedek Rozemberczki and Rik Sarkar. Twitch Gamers: A Dataset for Evaluating Proximity Preserving and Structural Role-based Node Embeddings, 2021.

Marco Saerens, Patrice Latinne, and Christine Decaestecker. Adjusting the Outputs of a Classifier to New a Priori Probabilities: A Simple Procedure. Neural Computation , 14(1):21-41, 2002. ISSN 0899-7667. doi: 10.1162/089976602753284446.

Bernhard Sch¨ olkopf, Dominik Janzing, Jonas Peters, Eleni Sgouritsa, Kun Zhang, and Joris Mooij. On causal and anticausal learning. In Proceedings of the 29th International Coference on International Conference on Machine Learning , ICML'12, pages 459-466, Madison, WI, USA, 2012. Omnipress. ISBN 978-1-4503-1285-1.

Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina EliassiRad. Collective Classification in Network Data. AI Magazine , 29(3):93-93, 2008. ISSN 2371-9621. doi: 10.1609/aimag.v29i3.2157.

Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan G¨ unnemann. Pitfalls of Graph Neural Network Evaluation, 2019.

Dirk Tasche. Fisher consistency for prior probability shift. J. Mach. Learn. Res. , 18(1):33383369, 2017. ISSN 1532-4435.

Petar Veliˇ ckovi´ c, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Li` o, and Yoshua Bengio. Graph Attention Networks. In International Conference on Learning Representations , 2018.

Slobodan Vucetic and Zoran Obradovic. Classification on Data with Biased Class Distribution. In Luc De Raedt and Peter Flach, editors, Machine Learning: ECML 2001 , pages 527-538, Berlin, Heidelberg, 2001. Springer. ISBN 978-3-540-44795-5. doi: 10.1007/3-540-44795-4 45.

Appendix for 'Adjusted Count Quantification Learning on Graphs'

A Further Details on the Implementation

In our implementation, we use torch-geometric [7] (MIT license) for the graph neural network (GNN) models, while the QuaPy Python library (BSD 3-Clause) was used for the quantification methods. Additionally, we used Nvidia's cuGraph library (Apache 2.0 license) for GPU-based graph traversal and distance computation, e.g., to create BFS-based covariate shift. All experiments were conducted on a single machine with an AMD Ryzen 9 5950X CPU, 64GB RAM and an Nvidia RTX 4090 GPU with 24GB VRAM. Our code is available at https://github.com/Cortys/ graph-quantification ; it includes a versioned list of all dependencies that were used.

B Kernel Selection & Influence of Kernel Hyperparameters

In Section 4.1, we defined the interpolated PPR kernel as

All reported results for SIS use this kernel with λ = 0 . 9 . In this section, we will explain why this particular kernel was chosen.

B.1 On Kernel Selection for SIS

As described in Section 3.1, SIS uses two vertex kernels, k p and k q , to estimate the densities p V and q V of the training and test vertices, respectively. Depending on the type of distribution shift and other available domain knowledge, different kernels can be used. Overall, we found that the PPR kernel is a good choice for structural covariate shift.

Recall that the kernels are used to estimate the density ratio ρ V ( v ) for each training vertex v . Those density ratios are then used to reweight the training vertices in the confusion matrix estimate ˆ C j,i , i.e., a high weight is assigned to training vertices from D L that 'look like' they have been sampled from the same distribution Q ( V ) as the unlabeled test vertices V U (see Eq. (11)). Assuming structural covariate shift between P and Q , the probability of sampling a vertex v given that v ′ was sampled should depend on some measure of the structural distance between v and v ′ . Since there are many different ways to define such a vertex distance ( shortest path , random walk, graph spectrum, etc.), there are many potential SIS kernels.

If we do not have any prior knowledge about the distribution shift, other than that it is structural and that the underlying graph is homophilic, the problem of estimating the probability of sampling a vertex v given that v ′ was sampled is ill-posed. Fortunately, we do not need to derive a perfect kernel to obtain good quantification results. Instead, note that:

The kernel-based estimates are used to reweight the training vertices in the confusion matrix estimate ˆ C j,i such that vertices that are close to the test distribution have a higher influence on the estimate than those that are not.

Standard (homophilic) GNN models consist of a stack of graph convolution layers, each of which essentially multiplies the vertex features with the graph adjacency matrix, thereby propagating the features of a vertex to its neighbors [17, 34, 12]. For APPNP this hold especially true since the embedding vector of a vertex is defined as a weighted sum over the embeddings of its neighbors, where the weights are defined to be PPR probabilities (cf. Eq. (13)). Overall, ignoring the nonlinearities between convolutions, for GNN models, k PPR ( v, v ′ ) can be interpreted as an approximation of how similar the predictions of a GNN h will be for two vertices v, v ′ .

Combining both points, the PPR kernel reweights vertices based on how similar the predictions of a GNN will be for them, i.e., SIS with PPR focuses on the region of the training data, where the predictive behavior of h is similar to the test data. Empirically, we found that this works well for different combinations of models, datasets and types of distribution shifts. However, in principle, given additional knowledge, one could also design domain specific kernels.

0.0

0.2

0.4

0.6

0.8

1.0

PPS

0.00

0.02

0.04

0.06

0.08

Quantification AE

MLP

APPNP

GCN

GAT

0.0

0.2

0.4

0.6

0.8

1.0

Covariate shift BFS

0.0

0.2

0.4

0.6

0.8

1.0

Covariate shift RW

CoraML

0.0

0.2

0.4

0.6

0.8

1.0

PPS

0.00

0.05

0.10

0.15

0.20

Quantification AE

0.0

0.2

0.4

0.6

0.8

1.0

Covariate shift BFS

0.0

0.2

0.4

0.6

0.8

1.0

Covariate shift RW

CiteSeer

0.0

0.2

0.4

0.6

0.8

1.0

PPS

0.000

0.005

0.010

0.015

0.020

0.025

0.030

0.035

Quantification AE

0.0

0.2

0.4

0.6

0.8

1.0

Covariate shift BFS

0.0

0.2

0.4

0.6

0.8

1.0

Covariate shift RW

Amazon Photos

0.0

0.2

0.4

0.6

0.8

1.0

PPS

0.00

0.01

0.02

0.03

0.04

0.05

Quantification AE

0.0

0.2

0.4

0.6

0.8

1.0

Covariate shift BFS

0.0

0.2

0.4

0.6

0.8

1.0

Covariate shift RW

Amazon Computers

B.2 Hyperparameter Evaluation of the PPR Kernel

As noted above, we use the interpolated PPR kernel from Eq. (15) with λ = 0 . 9 for all experiments. Intuitively, λ controls the influence of training vertices D L that are far-away from the test vertices V U . If λ = 1 , far-away vertices are effectively ignored. If λ < 1 , all vertices are considered at least to some degree, but the influence of far-away vertices is reduced. A large λ can have the advantage of reducing the influence of irrelevant or misleading vertices from different regions of the graph. However, if too many vertices are excluded, the effective sample size for the confusion matrix estimate is reduced, making it more noisy, which can, in turn, degrade performance.

Figure 3 shows the quantification performance of SIS with the PPR kernel for different values of λ . For CoraML and CiteSeer, λ < 1 clearly outperforms λ = 1 , with λ ≈ 0 . 9 performing very well. For the Amazon Photos and Computers datasets, λ = 1 performs best. Interestingly, for CoraML and CiteSeer, the quantification performance for λ = 1 is significantly worse than for λ < 1 , while for the Amazon datasets, λ = 1 performs better than λ < 1 . The reason for this discrepancy is that the CoraML and CiteSeer graphs contain multiple small components that are disconnected from the main connected component; for structurally shifted test distributions that lie within one of those small components, the PPR-based confusion matrix estimates are then based on very few vertices, making it very noisy and thereby degrading performance. By assigning at least a small weight to all vertices, e.g., via λ = 0 . 9 , the effective sample size is increased, which leads to better estimates on those datasets. For this reason, we found that a value of λ that is slightly smaller than 1 works well for most datasets, which is why we chose λ = 0 . 9 as a default.

B.3 Evaluation of the Shortest path Kernel

For comparison, we also evaluated an alternative to the PPR kernel. This alternative kernel is based on the shortest path (SP) distance between vertices instead of RW probabilities. We define this shortest-path kernel as

where d SP ( v, v ′ ) is the length of the shortest path length between v and v ′ and γ > 0 a tunable hyperparameter.

Figure 4 shows the quantification performance of SIS with the shortest-path kernel for different values of γ . Under PPS, increasing γ leads to a decrease in performance on all datasets except CiteSeer. Under covariate shift (both, BFS and RW), increasing γ generally improves the quantification performance.

This is plausible, since under PPS, all training vertices are equally important, while under covariate shift, the training vertices that are close to the test vertices are more important than those that are far away. By increasing γ , the kernel becomes more peaked around close vertices. Under covariate shift, where far-away vertices are less important, this leads to better quantification performance, while under PPS more aggressive reweighting decreases performance.

0.5

1.0

1.5

2.0

2.5

3.0

PPS

0.00

0.02

0.04

0.06

0.08

Quantification AE

MLP

APPNP

GCN

GAT

0.5

1.0

1.5

2.0

2.5

3.0

Covariate shift BFS

0.5

1.0

1.5

2.0

2.5

3.0

Covariate shift RW

CoraML

0.5

1.0

1.5

2.0

2.5

3.0

PPS

0.00

0.05

0.10

0.15

0.20

Quantification AE

0.5

1.0

1.5

2.0

2.5

3.0

Covariate shift BFS

0.5

1.0

1.5

2.0

2.5

3.0

Covariate shift RW

CiteSeer

0.5

1.0

1.5

2.0

2.5

3.0

PPS

0.000

0.005

0.010

0.015

0.020

0.025

0.030

0.035

0.040

Quantification AE

0.5

1.0

1.5

2.0

2.5

3.0

Covariate shift BFS

0.5

1.0

1.5

2.0

2.5

3.0

Covariate shift RW

Amazon Photos

0.5

1.0

1.5

2.0

2.5

3.0

PPS

0.00

0.01

0.02

0.03

0.04

0.05

Quantification AE

0.5

1.0

1.5

2.0

2.5

3.0

Covariate shift BFS

0.5

1.0

1.5

2.0

2.5

3.0

Covariate shift RW

Amazon Computers

Table 3 compares the quantification performance of SIS with the SP and the PPR kernel. Since large γ values generally lead to better performance under (structural) covariate shift, we used γ = 3 as a default for the reported SP kernel results. Overall, we find that the PPR kernel outperforms the SP kernel. Looking at the average ranks, we find that the SP kernel performs worst under PPS and best under BFS covariate shift. This is plausible, since the BFS-induced covariate shift samples vertices based on their distance to some root vertex; the test vertex density q V thus depends on SP distances. Nonetheless, the PPR kernel is still a better default choice.

C Runtime Analysis of SIS and NACC

Here, we analyze the runtime complexity of SIS for the PPR and SP kernels and provide additional information on their implementation. Additionally, we describe the complexity of NACC.

The main difference between standard ACC and ACC with SIS is the reweighting of training instances to account for covariate shift. Given a reweighted confusion matrix estimate, label prevalences are estimated using constrained optimization as described in Eq. (6), just like in standard ACC. The computational complexity of SIS thus is fully determined by the cost of computing ρ V ( v ) for all v ∈ D L (see Eq. (11)). As defined in Eq. (12), ρ V ( v ) is estimated via KDE.

As explained in Section 4.1, we use the constant kernel k p = k 1 to estimate p V , since the training data is sampled uniformly at random in our experiments and only the test data is subject to distribution shift. The computational complexity of SIS is thus determined by the time it takes to compute the kernel values k q . Let T k be the time it takes to compute the kernel values { k q ( v, v ′ ) | ( v, y ) ∈ D L , v ′ ∈ V U } . Based on those values, the time complexity of computing all weights ρ ( v ) thus is T = O ( T k + |D L | · |V U | ) . Here, T k depends on the choice of the kernel k .

C.1 Complexity of the PPR Kernel

For the PPR kernel from Eq. (13), the random walk probabilities Π L v ′ ,v have to be computed, where L is the length of the random walk. The matrix Π L ∈ R |V|×|V| is defined as

where ˆ A = D -1 A is the normalized (random-walk) adjacency matrix of the graph and α ∈ [0 , 1] is the probability of not moving in a given step of the random walk. Naively, Π L can be computed via standard dense matrix multiplication in O ( |V| 3 log 2 L ) , or O ( |V| 2 . 807 log 2 L ) , using Strassen's algorithm. Since matrix multiplication is very parallelizable, this naive strategy can be sufficient, even for medium to large graphs. The experiments on the PubMed dataset, which is the largest dataset with synthetic shifts, with roughly 20k vertices, were conducted using dense matrix multiplication on the described hardware (see Appendix A).

If dense multiplication is not feasible and the adjacency matrix is sparse (which is typically the case for large real-world graphs), the PPR kernel can still be computed using sparse matrix multiplications. Since the sparsity of Π L decreases with increasing L , simply applying sparse matrix multiplication L times is not feasible, as the resulting matrices quickly become dense. One way to tackle this 'densification' problem has been proposed by Damke and H¨ ullermeier [4]. They propose to ensure a certain level of sparsity after each multiplication by pruning entries below a threshold δ . This approximation allows for an efficient computation of Π L even for large graphs. We used this approach for the experiments with the 'Twitch Gamers' dataset (over 168k vertices).

C.2 Complexity of the Shortest path Kernel

For the SP kernel from Eq. (16), we need to compute the distances from the vertices in D L to the vertices in V U . Given an undirected graph without edge weights, the distance from any node v to all other nodes v ′ can computed via BFS in O ( |E| ) , where E is the edge set of the graph. Since node distances are symmetric in undirected graphs, we can thus simply run BFS for each node in D L or in V U . Starting the BFS traversals from the nodes in the smaller set, one then gets T k = O (min( |D L | , |V U | ) · |E| ) .

On a more practical note, when one wants to quantify label prevalences for many test sets sampled from the same graph, it is best to compute the entire all-pairs shortest path distance matrix D ∈ N |V|×|V| 0 in advance and to then use it as a lookup table for each quantification task. If the graph is sparse, i.e., |E| ≪ |V| 2 , D can be computed fairly efficiently. Additionally, distance computation in graphs lends itself well to parallelization. For our implementation, we used the Python bindings of Nvidia's libcugraph library, which provides a fast implementation of BFS for GPUs 1 .

C.3 Complexity of NACC

The primary difference between standard ACC and Neighborhood-aware ACC (NACC) is the confusion matrix estimate ˆ C and the distribution of predictions ˆ Q ( ˆ Y ) used in Eq. (6).

ACC uses a square K × K confusion matrix and a K -dimensional predictive distribution vector; both can be computed in O ( |V| + K 2 ) via a single pass over the vertices to count predicted label frequencies, followed by normalization. The objective of the constrained optimization problem from Eq. (6) can then be evaluated in O ( K 2 ) . The number of required objective evaluations depends on the used optimization algorithm; since the objective is quadratic, the number of optimizer evaluations can, however, be assumed to be in O (1) when using a (quasi-)Newtonian method. The number of parameters to the optimization problem is K , a quasi-Newtonian optimizer can thus compute and store estimates of the Hessian in O ( K 2 ) . Overall, the time complexity of ACC is thus O ( |V| + K 2 ) .

NACC uses an overdetermined system of equations with a rectangular K 2 × K confusion matrix estimate and a K 2 -dimensional predictive distribution vector. NACC computes the prediction frequencies of all class pairs ( j, k ) ∈ Y 2 , where the first class j is the predicted label of a class and k is the predicted majority label in its neighborhood. The majority label for all neighborhoods can be computed in O ( |E| ) . Combining this with the increased size of the confusion matrix, NACC thus has an overall time complexity of O ( |V| + |E| + K 3 ) . If the graph is sparse and the number of classes K is relatively small, NACC scales well even to large graphs.

D Feature-based Importance Sampling for Graph Quantification

In Section 3.1, we introduced SIS as a method to account for structural covariate shift in graph quantification. SIS is based on the assumption that the marginal densities p V and q V of the training and test vertices can be estimated using vertex kernels k p and k q that capture structural similarity between vertices. While this is a reasonable assumption under structural covariate shift, one could, in principle, use other types of kernels to estimate vertex similarity. One natural alternative to the structural kernels (Eqs. (15) and (16)) are non-structural kernels that only consider vertex features. In homophilic graphs, the features of neighboring vertices tend to be similar, indicating that feature similarity could be used as a proxy for structural similarity.

One advantage of a feature-based kernel is that it can be applied even in the absence of a graph structure, i.e., when only vertex features are available. Second, even if the graph structure is available, feature-based kernels can be more efficient to compute than structural kernels, especially on large graphs. To evaluate the suitability of feature-based kernels for graph quantification under structural covariate shift, we implemented a feature-based version of SIS using the inner product between vertex features as a kernel. Table 4 compares the quantification performance of this feature-based SIS variant with with SIS+PPR. Overall, we find that the feature-based SIS performs significantly worse than SIS with the PPR kernel across models and datasets. This demonstrates that structure-based vertex kernels are better suited to account for structural covariate shift in graph quantification than a purely feature-based kernel.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract claims that we propose two novel techniques for quantification learning on graphs, Structural importance sampling and Neighborhood-aware ACC, which improve quantification under covariate shift and in the presence of non-homophilic edges, respectively. Both approaches are described in Sections 3.1 and 3.2 and evaluated in Section 4. Our experiments demonstrate the claimed advantages of our methods over (standard) ACC and PACC on five graph datasets and using multiple base GNN classifiers.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: First, our work focuses, by design, on tackling structural covariate shift in graph quantification learning; to our knowledge, we are the first to consider this problem. We discuss other types of distribution shifts (e.g., concept shift) in Section 2.1 and explain why we focus on covariate shift in particular. Second, our proposed SIS method requires at least some knowledge about the data sampling process; in scenarios where little to no information about the origin of the data is available, SIS may not be applicable.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate 'Limitations' section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The theoretical derivation of SIS is provided in Section 3.1.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We describe all key aspects of our experimental setup and of our approach. Additionally, our code is available at https://github.com/Cortys/ graph-quantification . It contains instructions on how to reproduce our results.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: Our code is available at https://github.com/Cortys/ graph-quantification . It contains instructions on how to reproduce our results. The code is released under the MIT license.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: All relevant aspects of the experimental setup are described in Section 4.1. The supplement provides additional details. The provided code can be used to reproduce our results.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We do not report standard errors in Tables 1, 2, 3 and 4 because the errors are very close to zero and therefore not very informative. Instead, as described in Section 4.2, we conduct statistical significance tests to compare the performance different quantifiers.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer 'Yes' if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We describe our experimental setup, including the used compute resources, in the supplement.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We carefully read the Code of Ethics and hereby confirm that our research conforms to it.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: In our work, we consider the general GQL problem under covariate shift without a focus on a particular domain. While there are applications of GQL which could have a societal impact (e.g., voting outcome prediction), we do not evaluate our methods in such domains. Generally, since quantification learning is about predicting the distribution of classes in a population, the risks of quantification to the individual are limited. Additionally, we do not release any pretrained models that could be misused.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper does not include any data or models that have a high risk for misuse. The considered quantification domains are citation networks and co-purchasing graphs which do not contain any sensitive information. We do not release any pretrained models, but we provide the code to reproduce our results.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: For our implementation, we use parts of the QuaPy library (BSD 3Clause) [23], additionally, we use five common node classification datasets and cite the papers in which they were introduced. The datasets are publicly available and can, for example, be downloaded via the well-known torch-geometric library (MIT license).

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: We release the code for our new methods under the MIT license. The code is available at https://github.com/Cortys/graph-quantification with basic documentation.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: All experiments were conducted using pre-existing datasets. No new data was collected from human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: All experiments were conducted using pre-existing datasets. No new data was collected from human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: LLMs were only used to check and improve the writing of the paper. The core methods in this research and their development do not involve LLMs.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the problem of quantification learning (QL), that is, estimating label distributions instead of individual labels using the  graph-structured data. This problem is not explored for graph data yet. The authors identify limitations of existing approaches, particularly the adjusted classify & count (ACC) method under prior probability shift assumptions and this usually fails in graphs due to structural covariate shifts. Accordingly, the authors propose two key innovations:\n(a) Structural Importance Sampling (SIS), which generalizes ACC to handle covariate shifts using kernel density estimation, and\n(b) Neighborhood-aware ACC (NACC), which leverages neighborhood structure to improve class distinguishability in non-homophilic graphs. \nThe authors also conducted thorough experiments using 5 graph datasets and 3 types of distribution shift (PPS, BFS-based, and RW-based covariate shift) and demonstrated that their approaches outperform the existing baselines (measured using 2 metrics: absolute error (AE) and relative absolute error (RAE)).\n\nSTRENGTHS AND WEAKNESSES:\nBelow are my key comments and concerns on this paper:\n\n(a) SIS requires an estimate of the sampling distribution, which in practice assumes knowledge of the structural data generation or sampling process. This is a non-trivial requirement in real-world applications. How to address this challenge? With this, the potential impact of the proposed approach becomes limited. What are the alternatives to avoid this? Any thoughts?\n\n(b) SIS depends on graph kernels and density estimation. They are typically compute heavy tasks and may not scale to large graphs. How to address this computational scaling issue?\n\n(c) The authors work with synthetically induced distribution shifts on standard node classification datasets. These artificial or generated distribution shifts might not capture realistic scenarios as they are more complex in practice. How would you tackle this limitation in your experiments?\n\nQUESTIONS:\nPlease answer the following questions:\n\n(a) SIS requires an estimate of the sampling distribution, which in practice assumes knowledge of the structural data generation or sampling process. This is a non-trivial requirement in real-world applications. How to address this challenge? With this, the potential impact of the proposed approach becomes limited. What are the alternatives to avoid this? Any thoughts?\n\n(b) SIS depends on graph kernels and density estimation. They are typically compute heavy tasks and may not scale to large graphs. How to address this computational scaling issue?\n\n(c) The authors work with synthetically induced distribution shifts on standard node classification datasets. These artificial or generated distribution shifts might not capture realistic scenarios as they are more complex in practice. How would you tackle this limitation in your experiments?\n\nLIMITATIONS:\nBelow are my key comments and concerns on this paper:\n\n(a) SIS requires an estimate of the sampling distribution, which in practice assumes knowledge of the structural data generation or sampling process. This is a non-trivial requirement in real-world applications. How to address this challenge? With this, the potential impact of the proposed approach becomes limited. What are the alternatives to avoid this? Any thoughts?\n\n(b) SIS depends on graph kernels and density estimation. They are typically compute heavy tasks and may not scale to large graphs. How to address this computational scaling issue?\n\n(c) The authors work with synthetically induced distribution shifts on standard node classification datasets. These artificial or generated distribution shifts might not capture realistic scenarios as they are more complex in practice. How would you tackle this limitation in your experiments?", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

comments: SUMMARY:
This paper studies the quantification learning problem in the context of graph-structured data. The quantification learning aims to predict the label distribution of a set of instances. Compared to previous clustering based models, this paper extends the popular Adjusted Classify & Count (ACC) method to graphs, and proposes structural importance sampling (SIS) for covariate shift and Neighborhood-aware ACC to improve the quantification in the presence of non-homophilic edges.

STRENGTHS AND WEAKNESSES:
The strengths are as follows:  
1.Clear problem demonstration. The definition of quantification learning on graphs is well demonstrated, including its necessity, potential application, difference from traditional instance classification on graphs.  
2.Good background writing. The types of distribution shift, like concept shift, covarite shift and prior probability shift are well illustrated.  

The weakness are as follows:  
1.Although the quantification learning sounds a reasonable research prolem on graphs, I still think the instanece classification on graphs, together with aggregation models can well handle this problem. To be specific, the instance classification on graphs is well studied and many works can reach high performance on the used datasets, Citeseer, Pubmed and so on. Also, the results in Table 1 with AE and RAE seem not good metrics for researchers to understand how good the performance is. Any other metrics like classification acc?  
2.The used datasets are limited. Any true benchmark datasets for this quantification task? Not the generated datasets from common GNN datasets.  
3.The experiments are simple and monotonous, which is not convincing to illustrate the model's effectiveness.

QUESTIONS:
see above

LIMITATIONS:
see above
TITLE: Official Review
CLARITY: 2
IMPACT: 1
ORIGINALITY: 2
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper tackles the problem of estimating label prevalences on graph-structured data (graph quantification learning) where prior work was confined to simple node-clustering heuristics. It extends the Adjusted Classify & Count (ACC) family beyond the prior probability shift regime by introducing **Structural Importance Sampling (SIS)**, a covariate-shift-aware reweighting scheme that builds a kernelised confusion matrix using personalised PageRank densities so each training vertex votes in proportion to how likely it would have been sampled in the test graph. To resolve the class-identifiability failures that arise when non-homophilic edges blur decision boundaries, the authors add **Neighborhood-aware ACC (NACC)**: every prediction is paired with the majority label of its 1-hop neighbourhood, enlarging the observable confusion signature without retraining the base classifier. Both upgrades drop straight into the standard constrained least-squares ACC solver and gracefully collapse back to vanilla ACC when the shift is pure PPS.\n\nEmpirically, SIS and NACC are plugged on top of four backbones (MLP, GCN, GAT, APPNP) and evaluated on five benchmark graphs (CoraML, CiteSeer, PubMed, Amazon-Photos, Amazon-Computers) under three shift scenarios - prior-probability, BFS-induced covariate, and random-walk covariate. Across 36 model-shift combinations they achieve the best or tied-best average rank in 32 cases, cutting Absolute Error by up to 30-50% relative to Probabilistic ACC.\n\nOverall, by marrying structural reweighting with neighbourhood cues, the paper shows that ACC can be re-tooled to deliver reliable, shift-robust quantification on real-world graphs, establishing SIS + NACC as the first practical framework for graph quantification under covariate shift.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n**First to tackle covariate-shifted graph quantification:** The paper is the first to lift Adjusted Classify & Count (ACC) from the prior-probability-shift regime to full (structural) covariate shift on graphs, filling a clear gap in quantification research.\n\n**Elegant ACC generalisation (SIS):** Structural Importance Sampling rewrites the test-set confusion matrix as a covariate-shift re-weighting and estimates the density ratio with personalised-PageRank kernels, gracefully collapsing to vanilla ACC when constant kernels are used.\n\n**Neighbourhood cue for identifiability (NACC):** Appending the majority 1-hop label breaks class-confusion collinearity without retraining the base classifier, neatly exploiting homophily yet reverting to standard ACC in purely homophilic graphs.\n\n**Plug-and-play, classifier-agnostic design:** SIS + NACC drop straight into the standard constrained-least-squares ACC solver and work on four very different backbones (MLP, GCN, GAT, APPNP) without any model-specific tuning.\n\n**Consistent empirical gains across shifts:** On five benchmarks and three synthetic shift types, the new quantifiers attain the best or tied-best average rank in 32 / 36 model–shift combinations, cutting Absolute Error by up to 30–50 % over Probabilistic ACC.\n\n**Graceful degradation & interpretability:** Because both upgrades reduce to classic ACC when their extra signal is irrelevant, they inherit ACC’s transparent confusion-matrix interpretation and do not hurt performance under pure PPS.\n\n---\n\n## Weaknesses\n\n**Synthetic evaluation only:** All shifts are injected (Zipf PPS, BFS, RW) on node-classification datasets; the authors explicitly note the absence of real graph-quantification benchmarks, so external validity remains uncertain.\n\n**Kernel sensitivity and hyper-parameter exposure:** SIS assumes a local random-walk sampling process and relies on a PPR kernel with a user-set $\\lambda$ and walk length L; in BFS shifts the kernel is acknowledged to be a mismatch, hinting that performance may hinge on careful kernel engineering.\n\n**Homophily dependency and noise trade-off:** NACC presumes that neighbours share labels; on heterophilic or sparsely connected graphs its extra confusion features may be noisy, and the authors concede that adding deeper-hop information quickly degrades estimates.\n\n**Scalability not analysed:** Computing PPR densities for every training vertex can be $O(|V|^3)$ in the worst case; the paper gives no runtime or memory study, leaving open whether SIS is feasible for million-node graphs.\n\n**Baseline breadth limited:** Comparisons stop at CC/ACC and ENQ; distribution-matching quantifiers and recent deep quantification methods are only mentioned as future work, so the empirical case is incomplete.\n\nQUESTIONS:\n1. Could you (a) add at least one naturally shifted graph - e.g. a time-stamped social network split by month, or a sampled Web graph - and (b) report SIS + NACC performance there?\n2. Is it possible to add a sensitivity study ($\\lambda$, L, alternative kernels)?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper develops a method to estimate the label distribution in a node classification graph problem, under covariate shift.\nTwo adjustments that are tailored to graph data are given. One applies a density ratio correction based on a kernel density estimator, where the kernel is derived from the graph. The second applies an adjustment to label distributions based on homophily between neighboring vertices, in order to alleviate collinearity of label distributions. These adjustments are applied on top of standard adjustments applied for label shifts.\n\nExperiments on several node classification datasets using various classifiers are given, under label shift, and two types of covariate shifts induced by sampling using random walks on the graph. The methods show an overall improved performance over the baseline estimation method.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThe paper is nicely written, self contained and the methods are principled, simple, and seem effective. Experiments are compelling as they cover a nice variety of settings.\n\nWeaknesses:\n\nApplying a density ratio correction to adjust for covariate shift is a rather basic technique and even though the paper gives a nice presentation and a solid solution, I am not sure its novelty merits a publication at NeurIPS.\nAt the very least, I would expect a baseline that applies a density ratio correction not based on random walks, e.g. one that learns a probabilistic model to distinguish between the features of the source and target dataset. Even if the suggested method outperforms such a baseline, it wouldn't be surprising since the covariate shift induced in the experiments is tailored to the kernel in proposed method. A much more compelling experiment would consider a covariate shift that occurs naturally in graph data, e.g. by considering two different datasets. It is understandable that it may be difficult to find such data, but there should be a through discussion about it with experiments that examine baselines that handle covariate shift and datasets with shifts that do not adhere to the assumptions underlying the kernel.\n\nQUESTIONS:\nI've listed my suggestions in the strengths and weaknesses part. Otherwise, the paper is well written and I appreciate presentation.\nMore compelling datasets and baselines could make me consider raising my score, though it will still be a borderline case in my opinion.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}