Title: Scalable Fair Influence Maximization
Venue: NeurIPS 2023
Accepted: yes
Keywords: influence maximization

approximation algorithm

social fairness
Abstract: Given a graph $G$, a community structure $\mathcal{C}$, and a budget $k$, the fair influence maximization problem aims to select a seed set $S$ ($|S|\leq k$) that maximizes the influence spread while narrowing the influence gap between different communities. While various fairness notions exist, the welfare fairness notion, which balances fairness level and influence spread, has shown promising effectiveness. However, the lack of efficient algorithms for optimizing the welfare fairness objective function restricts its application to small-scale networks with only a few hundred nodes. In this paper, we adopt the objective function of welfare fairness to maximize the exponentially weighted summation over the influenced fraction of all communities. We first introduce an unbiased estimator for the fractional power of the arithmetic mean. Then, by adapting the reverse influence sampling (RIS) approach, we convert the optimization problem to a weighted maximum coverage problem. We also analyze the number of reverse reachable sets needed to approximate the fair influence at a high probability. Further, we present an efficient algorithm that guarantees $1-1/e - \varepsilon$ approximation.

Scalable Fair Influence Maximization

Abstract

Given a graph G , a community structure C , and a budget k , the fair influence maximization problem aims to select a seed set S ( | S | ≤ k ) that maximizes the influence spread while narrowing the influence gap between different communities. While various fairness notions exist, the welfare fairness notion, which balances fairness level and influence spread, has shown promising effectiveness. However, the lack of efficient algorithms for optimizing the welfare fairness objective function restricts its application to small-scale networks with only a few hundred nodes. In this paper, we adopt the objective function of welfare fairness to maximize the exponentially weighted summation over the influenced fraction of all communities. We first introduce an unbiased estimator for the fractional power of the arithmetic mean. Then, by adapting the reverse influence sampling (RIS) approach, we convert the optimization problem to a weighted maximum coverage problem. We also analyze the number of reverse reachable sets needed to approximate the fair influence at a high probability. Further, we present an efficient algorithm that guarantees 1 -1 /e -ε approximation.

1 Introduction

Influence maximization (IM) is a well-studied problem in the field of social network analysis. Given a graph G and a positive integer k , the problem asks to find a node set S ( | S | ≤ k ) which can spread certain information to trigger the largest expected number of remaining nodes. There have been various IM variants, such as adaptive influence maximization [1], multi-round influence maximization [2], competitive influence maximization [3], and time-critical influence maximization [4]. Influence maximization and these variants have important applications in viral marketing, rumor control, health interventions, etc [5].

Considering the situation that when disseminating public health interventions, for example, suicide/HIV prevention [6] or community preparedness against natural disasters, we can select individuals who act as peer-leaders to spread such information to maximize the outreach following influence

Lichao Sun

maximization. However, it may lead to discriminatory solutions as individuals from racial minorities or LGBTQ communities may be disproportionately excluded from the benefits of the intervention [7]. Therefore, derived from such significant social scenarios, fairness in influence maximization has become a focus of attention for many researchers [8, 9, 7, 10].

Generally, fair influence maximization aims to improve the influenced fraction inside some communities where the coverage may get unfairly low. Currently, a universally accepted definition of fair influence maximization remains elusive, and recent work has incorporated fairness into influence maximization by proposing various notions of fairness, such as maximin fairness [9], diversity constraints [8], demographic parity [11], and welfare fairness [7]. Among these notions, welfare fairness show several attractive features. Its objective function is the weighted sum (with community size as the weight) of the fractional power ( α fraction) of the expected proportion of activated nodes within every community. The fractional exponent α is the inequality aversion parameter, allowing one to balance between fairness and influence spread, with α tending to 1 for influence spread and α tending to 0 for fairness. The objective function enjoys both monotonicity and submodularity, enabling a greedy approach with 1 -1 /e -ε approximation.

However, even though Rahmattalabi et al. [9] has given a full analysis of welfare fairness, there is currently no efficient algorithm to optimize its objective function with provable guarantee, which restricts their applications to small-scale networks with only a few hundred nodes. In this paper, we propose an efficient algorithm for maximizing the welfare fairness based on reverse influence sampling (RIS) [12, 13, 14]. The main challenges in adapting the RIS approach to the welfare fairness objective include: (a) how to carry out the unbiased estimation of the fractional power of the expected proportion of activated nodes in each community, since simply obtaining an unbiased estimate of the expected proportion and then taking its fractional power is not an unbiased estimator; (b) how to integrate the unbiased estimation designed into the RIS framework. We address both challenges and propose a new scalable fair influence maximization with theoretical guarantees.

Our contributions can be summarized as follows:

We propose an unbiased estimator for the fractional power of the arithmetic mean by leveraging Taylor expansion techniques. The estimator enables us to accurately estimate the fair influence under welfare fairness.

Based on the above unbiased estimator, we adapt the RIS approach to approximate the fair influence with Reverse Reachable (RR) sets and propose the FIMM algorithm that works efficiently while guaranteeing the (1 -1 /e -ε ) -approximation solution. Our theoretical analysis needs to address the concentration of the unbiased estimator of the fractional power and thus is much more involved than the standard RIS analysis.

We carry out detailed experimental analysis on five real social networks to study the trade-off between fairness and total influence spread. We test different fairness parameters, influence probabilities, seed budget, and community structures to confirm the performance of our proposed algorithms.

Related Work Influence maximization (IM) is first studied as an algorithmic problem by Domingos and Richardson [15, 16]. Kempe et al. [17] mathematically formulate IM as a discrete optimization problem and prove it is NP-hard. They also provide a greedy algorithm with 1 -1 /e approximation based on the submodularity and monotonicity of the problem. Hence, many works have been proposed to improve the efficiency and scalability of influence maximization algorithms [18, 19, 20, 21, 22, 12, 13, 14]. Among these methods, the most recent and the state of the art is the reverse influence sampling (RIS) approach [12, 13, 14, 23], where the IMM algorithm [14] is one of the representative algorithms. The idea of RIS approaches is to generate an adequate number of reverse reachable sets ( a.k.a. RR sets), and then the influence spread can be approximated at a high probability based on these RR sets. Therefore, the greedy approach can be easily applied by iteratively selecting the node that could bring the maximal marginal gain in terms of influence spread as a seed node.

Recent work has incorporated fairness directly into the influence maximization framework by relying on Rawlsian theory [9], game theoretic principles [8], and equity-based notion [24]. Based on the Rawlsian theory, maximin fairness [9, 8] aims to maximize the influence fraction of the worst-off community. Inspired by the game theoretic notion of core, diversity constraints [8] require that every community obtains an influenced fraction higher than when it receives resources proportional to its size and allocates them internally. Equity-based notion [24] strives for equal influenced fraction

across all communities. However, these notions can hardly balance fairness and total influence and usually lead to a high influence reduction. Especially, strict equity [24] is rather hard to achieve in influence maximization. To address these shortcomings, Rahmattalabi et al. [7] propose the welfare fairness that can control the trade-off between fairness and total influence by an inequality aversion parameter. Based on the cardinal welfare theory [25], the objective function of welfare fairness is to maximize the weighted summation over the exponential influenced fraction of all communities. Fish et al. [26] also follow welfare functions and propose ϕ -mean fairness, where the objective function becomes MMF when ϕ = -∞ . However, they do not address the challenge of unbiased estimation of the fractional power. In addition, none of the above studies address the scalability of the algorithms. Thus, to the best of our knowledge, we are the first to study scalability in the context of fair influence maximization.

2 Model and Problem Definition

Information Diffusion Model In this paper, we adopt the well-studied Independent Cascade (IC) model as the basic information diffusion model. Under IC model, a social network is modeled as a directed influence graph G = ( V, E, p ) , where V is the set of vertices (nodes) and E ⊆ V × V is the set of directed edges that connect pairs of nodes. For an edge ( v i , v j ) ∈ E , p ( v i , v j ) indicates the probability that v i influences v j . The diffusion of information or influence proceeds in discrete time steps. At time t = 0 , the seed set S is selected to be active, denoted as A 0 . At each time t ≥ 1 , all nodes in A t -1 try to influence their inactive neighbors following influence probability p . The set of activated nodes at step t is denoted as A t . The diffusion process ends when there is no more node activated in a time step. An important metric in influence maximization is the influence spread , denoted as σ ( S ) , which is defined as the expected number of active nodes when the propagation from the given seed set S ends. For the IC model, σ ( S ) = E [ | A 0 ∪ A 1 ∪ A 2 ∪ . . . | ] . We use ap ( v, S ) to represent the probability that node v is activated given the seed set S . Then we have σ ( S ) = ∑ v ∈ V ap ( v, S ) .

Live-edge Graph Given the influence probability p , we can construct the live-edge graph L = ( V, E ( L )) , where each edge ( v i , v j ) is selected independently to be a live edge with the probability p ( v i , v j ) . The influence diffusion in the IC model is equivalent to the deterministic propagation via bread-first traversal in a random live-edge graph L . Let Γ( G,S ) denote the set of nodes in graph G that can be reached from the node set S . By the above live-edge graph model, we have σ ( S ) = E L [ | Γ( L, S ) | ] = ∑ L Pr [ L | G ] ·| Γ( L, S ) | , where the expectation is taken over the distribution of live-edge graphs, and Pr [ L | G ] is the probability of sampling a live-edge graph L in graph G .

Approximation Solution A set function f : V → R is called submodular if for all S ⊆ T ⊆ V and u ∈ V \ T , f ( S ∪{ u } ) -f ( S ) ≥ f ( T ∪{ u } ) -f ( T ) . Intuitively, submodularity characterizes the diminishing return property often occurring in economics and operation research. Moreover, a set function f is called monotone if for all S ⊆ T ⊆ V , f ( S ) ≤ f ( T ) . It is shown in [17] that influence spread σ ( · ) for the independent cascade model is a monotone submodular function. A non-negative monotone submodular function allows a greedy solution to its maximization problem with 1 -1 /e approximation [27], which provides the technical foundation for most influence maximization tasks.

Fair Influence Maximization For a given graph G with n G nodes, the classic influence maximization problem is to choose a seed set S consisting of at most k seeds to maximize the influence spread σ ( S, G ) . Assuming each node belongs to one of the disjoint communities c ∈ C := { 1 , 2 , . . . , C } , such that V 1 ∪ V 2 ∪ · · · ∪ V C = V where V c ( n c = | V c | ) denotes the set of nodes that belongs to community c . Generally, fair influence maximization (FIM) aims to narrow the influence gap between different communities while maintaining the total influence spread as much as possible. In this paper, we adopt the fair notion proposed by Rahmattalabi et al. [7], where the welfare function is used to aggregate the cardinal utilities of different communities. The goal is to select at most k seed nodes, such that the objective function F α ( S ) (also referred to as fair influence in this paper) is maximized, where F α ( S ) = ∑ c ∈C n c u c ( S ) α , 0 < α < 1 . The utility u c ( S ) denotes the expected proportion of influenced nodes in the community c with the seed set S . Exponent α is the inequality aversion parameter that controls the trade-off between fairness and total influence, with α tending to 1 for influence spread and α tending to 0 for fairness. We thus define the fair influence maximization problem in this paper as follows:

Definition 1. The Fair Influence Maximization (FIM) under the independent cascade model is the optimization task where the input includes the directed influence graph G = ( V, E, p ) , the nonoverlapping community structure C , and the budget k . The goal is to find a seed set S ∗ to maximize the fair influence, i.e. , S ∗ = argmax S : | S |≤ k F α ( S ) .

According to [7], the fair influence F α ( S ) is both monotone and submodular, which provides the theoretical basis for our efficient algorithm design, to be presented in the next section.

3 Method

The monotonicity and submodularity of the fair influence objective function enable a greedy approach for the maximization task. However, as commonly reported in influence maximization studies, naively implementing a greedy approach directly on the objective function will suffer a long running time. The reason is that accurate function evaluation requires a large number of Monte-Carlo simulations. In this section, we aim to significantly speed up the greedy approach by adapting the reverse influence sampling (RIS) [12, 13, 14], which provides both theoretical guarantee and high efficiency. We propose the FIMM algorithm that is efficient when the number of communities is small, which is hopefully a common situation such as gender and ethnicity.

For the convenience of reading, we list most important symbols featured in this paper in Table 1.

3.1 Unbiased Fair Influence

To estimate the influence spread, we may generate a number of live-edge graphs L = { L 1 , L 2 , · · · , L t } as samples. Then, for a given seed set S , ˆ σ ( L , S ) = 1 t ∑ t i =1 | Γ( L i , S ) | is an unbiased estimator of σ ( S ) . However, situations are completely different for fair influence. For each community c , its fair influence is actually n c u α c . If we still generate a number of live-edge graphs and estimate u c by ˆ u c ( L , S ) = 1 t ∑ t i =1 | Γ( L i , S ) ∩ V c | / | V c | , then ˆ u c ( L , S ) is an unbiased estimator for u c , but ˆ u c ( L , S ) α is actually a biased estimator of u α c for 0 < α < 1 . In fact, the value of u α c is generally higher than the true value, which is revealed by Jensen's Inequality.

Fact 1. (Jensen's Inequality) If X is a random variable and ϕ is a concave function, then

Therefore, our first challenge in dealing with the welfare fairness objective is to provide an unbiased estimator for the fractional power value of u α c . We meet this challenge by incorporating Taylor expansion as in Lemma 1.

Lemma 1. For a given seed set S and an inequality aversion parameter α , the fair influence

Proof. By Taylor expansion of binomial series, we have

By definition of fair influence in Definition 1, we have

where

Thus concludes the proof.

Lemma 1 demonstrates that the calculation of fair influence with fractional powers can be transformed into the summation of integral powers. Further, we can get an unbiased estimator for integral powers of arithmetic mean as given in Lemma 2.

Lemma 2. [28] Suppose that a simple random sample of size m is to be drawn, with replacement, in order to estimate µ n . An unbiased estimator for µ n ( n ≤ m ) is

̸

̸

̸

where the summation extends over all permutations of all sets of n observations in a sample subject only to the restriction noted.

3.2 Unbiased Fair Influence with RR sets

Many efficient influence maximization algorithms such as IMM [14] are based on the RIS approach, which generates a suitable number of reverse-reachable (RR) sets for influence estimation.

̸

RR set An RR set RR ( v ) (rooted at node v ∈ V ) can be generated by reversely simulating the influence diffusion process starting from the root v , and then adding all nodes reached by reversed simulation into this RR set. In this way, RR ( v ) is equivalent to collecting all nodes that can reach v in the random live-edge graph L , denoted by Γ ′ ( L, v ) . Intuitively, each node u ∈ RR ( v ) if selected as a seed would activate v in this random diffusion instance. We say that S covers a RR set RR ( v ) if S ∩ RR ( v ) = ∅ . The expected activated probability ap ( v, S ) is thus equivalent to the probability that S covers a randomly generated v -rooted RR set. In the following, we use RR ( v ) to represent a randomly generated RR set when v is not specified, i.e. , RR ( v ) = Γ ′ L ∼U ( P L ) ( L, v ) where P L is the space of all live-edge graphs and U ( · ) denotes the uniform distribution.

̸

Let X c be the random event that indicates whether a randomly selected node in community c would be influenced in a diffusion instance by the given seed set S . As mentioned above, an RR set maps to a random diffusion instance. Assuming we generate R consisting of θ RR sets in total and each community c gets θ c RR sets. Let R c be the set of RR sets that are rooted in the community c , then |R c | = θ c . Let X i c ( i ∈ [ θ c ] ) be a random variable for each RR set R i ∈ R c , such that X i c = 1 if R i c ∩ S = ∅ , and X i c = 0 otherwise. Then, we have E [ X c ] = u c and E [ X c ] = 1 -u c .

Based on Lemma 1 and Lemma 2, we can get the unbiased estimator of E [ X c ] α through RR sets as

Further, we can get the unbiased estimator of the fair influence F α ( S ) as

where π c = θ c -∑ i ∈ [ θ c ] X i c , and

In the following, we consider Eq. 4 as our objective function to deal with the fair IM problem.

3.3 FIMM

For a given seed set S , let φ [ c ] denote the number of all u -rooted ( u ∈ V c ) RR sets covered by S , and κ [ v ][ c ] denote the number of all u -rooted ( u ∈ V c ) RR sets that covered by v ( v ∈ V \ S ) but not by S , then the marginal fair influence gain of v is

Therefore, when generating RR sets, we have to count κ [ v ][ c ] which indicates the community-wise coverage for v and record η [ v ] which indicates the linked-list from v to all its covered RR sets, as shown in Algorithm 1. As shown in lines 6 ∼ 9, when generating a random v -rooted RR set RR ( v ) , we count all nodes u ∈ RR ( v ) and raise all κ [ u ][ c ( v )] by 1 , where c ( v ) indicates v 's community label. It should be noted that modifying κ [ u ][ c ( v )] can be accomplished simultaneously when generating RR ( v ) by the reverse influence sampling.

Based on the RR sets generated by Algorithm 1, we present our FIMM algorithm (Algorithm 2) to select k seed nodes that maximize Eq. 4 through a greedy approach, i.e. , iteratively selecting a node with the maximum alternative marginal fair influence gain as presented in Eq. 5. Apparently, it costs O ( C ) to calculate ˆ F ( v | S ) for any v where C is the number of communities. When C is small ( i.e. , a constant), it would be efficient to compute ˆ F ( v | S ) for all v ∈ V in O ( Cn G ) . Besides, since

Algorithm 1: RR-Generate : Generate RR sets

ˆ F α ( S, R ) is submodular and monotone, we can adopt a lazy-update strategy [18] that selects v with the maximal ˆ F α ( v | S ) as a seed node if ˆ F α ( v | S ) is still the maximum after updating. This lazy-update strategy (lines 10 ∼ 12) can cut down a great amount of redundant time cost that can be empirically up to 700 times faster than a simple greedy algorithm [18].

There are two vital counting arrays in Algorithm 2, i.e. , φ [ c ] and κ [ v ][ c ] . φ [ c ] records and updates the number of RR sets covered by S in community-wise. By lines 20 ∼ 24, κ [ v ][ c ] keeps updating and always indicates the extra coverage of v on all u -rooted ( u ∈ V c ) RR sets besides S . It establishes a

Algorithm 2: FIMM : Fair Influence Maximization

̸

convenient way for updating φ ( c ) that only needs to increase φ ( c ) by κ [ v ][ c ] where v is the newly selected node for all c ∈ C . If we denote the original community-wise coverage as κ ′ , which means ( ∼ , κ ′ , ∼ ) = RR-Generate ( G, C , k, θ c ) , then it holds κ ′ [ v ][ c ] = κ [ v ][ c ] + φ [ c ] for all v ∈ V and c ∈ C in Algorithm 2.

3.4 Number of RR sets

In this subsection, we discuss the number of RR sets needed to approximate the fair influence with high probability. Let OPT F denote the optimal solution and S ∗ denote the corresponding optimal seed set for the fair influence maximization problem defined in this paper, i.e. , OPT F = F α ( S ∗ ) = ∑ c ∈C n c u c ( S ∗ ) α = ∑ c ∈C n c ( 1 -α ∑ ∞ n =1 η ( n, α ) ( 1 -u c ( S ∗ ) ) n ) . Since this paper deals with the fair influence maximization problem, we thus assume that the maximal community utility max c ∈C u c ( S # ) ≥ max c ∈C u c ( S ∗ ) of an arbitrary seed set S # would not be too big.

Lemma 3. Let δ 1 ∈ (0 , 1) , ε 1 ∈ (0 , 1) , and θ 1 = 12 Q 2 ln( C/δ 1 ) ε 2 1 (1 -b ) where Q is the approximation parameter, b = max ( u c ( S ∗ )) , ∀ c ∈ C , and S ∗ = argmax S : | S |≤ k F α ( S ) denotes the optimal solution for the FIM problem based on R , then ˆ F α ( S ∗ , R ) ≥ (1 -ε 1 ) · OPT F holds at least 1 -δ 1 probability if θ ≥ Cθ 1 .

Lemma 4. Let δ 2 ∈ (0 , 1) , ε 2 = ( e e -1 ) ε -ε 1 , and θ 2 = 8 Q 2 ln( C ( n G k ) /δ 2 ) ε 2 2 (1 -b 0 ) where Q is the approximation parameter, b 0 = max ( u c ( S # )) , ∀ c ∈ C where S # could be an arbitrary fair solution. For each bad S (which indicates F α ( S ) < (1 -1 /e -ε ) · OPT F , ˆ F α ( S, R ) ≥ (1 -1 /e )(1 -ε 1 ) · OPT F holds at most δ 2 / ( n G k ) probability if θ ≥ Cθ 2 .

Please refer to Appendix for the detailed proof of Lemma 3 and Lemma 4.

Theorem 1. For every ε > 0 , ℓ > 0 , 0 < α < 1 , and Q ≥ 2 , by setting δ 1 = δ 2 = 1 / 2 n ℓ G and θ ≥ C · max ( θ 1 , θ 2 ) , the output S of FIMM satisfies F α ( S ) ≥ (1 -1 /e -ε ) F α ( S ∗ ) , where S ∗ denotes the optimal solution with probability at least 1 -1 /n ℓ G .

Proof. Combining Lemma 3 and Lemma 4, we have ˆ F α ( S, R ) ≥ (1 -1 /e -ε ) · OPT F at least 1 -δ 1 -δ 2 probability based on the union bound. If we set δ 1 = δ 2 = 1 / 2 n ℓ G , then, following the standard analysis of IMM, our FIMM algorithm provides ( 1 -1 /e -ε )-approximation with probability at least 1 -1 /n ℓ G .

If we set δ 1 = δ 2 = 1 2 n ℓ G and ε 1 = ε · e e -1 · √ 3 τ 1 √ 3 τ 1 + √ 2 τ 2 where τ 1 = √ ln C + ℓ ln n G +ln2 and τ 2 2 = τ 2 1 +ln ( n G k ) , then a possible setting of θ could be θ = ( e -1 e ) 2 · 4 CQ 2 ( √ 3 τ 1 + √ 2 τ 2 ) 2 ε 2 (1 -b 0 ) .

4 Experiments

4.1 Dataset

Email The Email dataset [29] is generated using email data from a large European research institution, where every node is a member of the research institution and an directed edge ( v, u ) indicates that v has sent u at least one email. It contains 1,005 nodes and 25,571 directed edges. Moreover, this dataset also contains "ground-truth" community memberships of nodes, where each member belongs to exactly one of 42 departments at the research institute.

Flixster The Flixster dataset [30] is a network of American social movie discovery services. To transform the dataset into a weighted graph, each user is represented by a node, and a directed edge from node u to v is formed if v rates one movie shortly after u does so on the same movie. It contains 29,357 nodes and 212,614 directed edges. It also provides the learned influence probability between each node pair, which can be incorporated into the IC model. Since it has no community information, we construct the biased community structure by categorizing individuals according to their susceptibility of being influenced to highlight the level of inequality and get 100 communities. Moreover, this dataset contains the learned influence probability between each node pair, which can be incorporated into the IC model.

Amazon The Amazon dataset [31] is collected based on Customers Who Bought This Item Also Bought feature of the Amazon website. If a product i is frequently co-purchased with product j , the graph contains an undirected edge between i to j . The dataset also provides the ground-truth community structure which indicates the product categories. The original network has 334,863 nodes and 925,872 undirected edges. After Pruning low-quality communities (whose size is no more than 10 nodes), the Amazon network tested in our experiments 12,698 nodes, 40,096 edges, and 509 communities.

Youtube The Youtube dataset [31] is a network of the video-sharing web site that includes social relationships. Users form friendship each other and users can create groups which other users can join. The friendship between users is regarded as undirected edges and the user-defined groups are considered as ground-truth communities. The original network has 1,134,890 nodes and 2,987,624 undirected edges. After screening high-quality communities, it remains 30,696 nodes, 198,867 edges, and 1,157 communities.

DBLP The DBLP dataset [31] is the co-authorship network where two authors are connected if they ever published a paper together. Publication venues, such as journals or conferences, defines an individual ground-truth community and authors who published to a certain journal or conference form a community. The original network has 717,080 nodes and 1,049,866 undirected edges. We also perform the network pruning and finally obtain 72,875 nodes, 268,346 edges, and 1,352 communities.

4.2 Evaluation

Let S I denote the seed set returned by IMM [14], S F denote the seed set returned by FIMM , the performance of S F towards fairness can be evaluated via the Price of Fairness (PoF) and the Effect of Fairness (EoF) as

where | S I | = | S F | = k , σ ( · ) denotes the influence spread and F α ( · ) denotes the fair influence.

Intuitively, PoF implies how much price it cost to access fairness and EoF implies to what extent it steps towards fairness.

4.3 Results

We test IMM and our proposed FIMM algorithm in the experiment. In all tests, we run 10,000 Monte-Carlo simulations to evaluate both the influence spread and the fair influence under IC model. We also test influence probability p , inequality aversion parameter α and the seed budget k .

4.3.1 Email & Flixster

For the Email network, we set α = 0 . 5 . Since the network is small, we apply the Uniformed IC model where the influence probability is the same across all edges. We test different probabilities that range

0.001

0.003

0.005

0.007

0.009

p

0

10

20

30

40

50

60

%

PoF

EoF

(a) Email

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0

20

40

60

80

%

PoF

EoF

from 0.001 to 0.01 with the step of 0.001. For the Flixster network, we test the inequality aversion parameter α which ranges from 0.1 to 0.9 with the step of 0.1. We set k = 50 for both networks and the results are shown in Figure 1.

As the influence probability p increases, both PoF and EoF show a downward trend. This may be attributed to the increased challenges faced by disadvantaged communities in being influenced when p is small. Similarly, both PoF and EoF also show a downward trend with the increase of the aversion parameter α . The reason lies that communities experience greater promotions in fair influence when the aversion parameter is smaller, resulting in higher EoF and PoF . Moreover, there is hardly any fairness when α ≥ 0 . 7 where the gap between u α and u is just too small.

4.3.2 Amazon, Youtube & DBLP

For Amazon, Youtube, and DBLP networks, we set α = 0 . 5 and p ( v i , v j ) = 1 /d in ( v j ) where d in denotes the in-degree as the influence probability following the weighted IC model [17]. We test different seed budget k that ranges from 5 to 50 with the step of 5. Results are shown in Figure 2.

5

10

15

20

25

30

35

40

45

50

k

0

20

40

60

80

%

PoF

EoF

(a) Amazon

5

10

15

20

25

30

35

40

45

50

k

0

4

8

12

%

PoF

EoF

5

10

15

20

25

30

35

40

45

50

k

0

5

10

15

20

25

30

%

PoF

EoF

(c) DBLP

Generally, FIMM tends to produce a noticeably fairer output when k is small. It reflects the idea that enforcing fairness as a constraint becomes easier when there is an abundance of resources available. However, there are also some exceptions where smaller k leads to a lower EoF, e.g. , k = 20 Figure 2(b) and k = 5 , 15 in Figure 2(c). This may be attributed to the fact that the seed selection in FIMM follows a pattern of remedying the previously fair solutions in each round.

5 Conclusion

This paper focuses on the fair influence maximization problem with efficient algorithms. We first tackle the challenge of carrying out the unbiased estimation of the fractional power of the expected proportion of activated nodes in each community. Then, we deal with the challenge of integrating unbiased estimation into the RIS framework and propose an ( 1 -1 /e -ε ) approximation algorithm FIMM . We further give a theoretical analysis that addresses the concentration of the unbiased estimator of the fractional power. The experiments validate that our algorithm is both scalable and effective, which is consistent with our theoretical analysis.

There are several future directions from this research. One direction is to find some other unbiased estimators for the fair influence that would be easier to calculate through RIS. Another direction is to explore a more efficient seed selection strategy. The fairness bound is also a meaningful research direction.

Limitation The limitations of our work are mainly two points. The first limitation is that our algorithm can only be efficient when the number of communities is small (e.g., a constant). The second limitation is that our algorithm is based on the assumption that the minimal community utility of an arbitrary seed set would not be too big.

Acknowledgments and Disclosure of Funding

This work was supported by the Fundamental Research Funds for the Central Universities (No.2022QN1093).

References

Binghui Peng and Wei Chen. Adaptive influence maximization with myopic feedback. In NeurIPS , pages 5575-5584, 2019.

Lichao Sun, Weiran Huang, Philip S. Yu, and Wei Chen. Multi-round influence maximization. In KDD , pages 2249-2258, 2018.

Khurshed Ali, Chih-Yu Wang, and Yi-Shin Chen. Leveraging transfer learning in reinforcement learning to tackle competitive influence maximization. Knowl. Inf. Syst. , 64(8):2059-2090, 2022.

Junaid Ali, Mahmoudreza Babaei, Abhijnan Chakraborty, Baharan Mirzasoleiman, Krishna P. Gummadi, and Adish Singla. On the fairness of time-critical influence maximization in social networks. In ICDE , pages 1541-1542, 2022.

Yuchen Li, Ju Fan, Yanhao Wang, and Kian-Lee Tan. Influence maximization on social graphs: A survey. IEEE Trans. Knowl. Data Eng. , 30(10):1852-1872, 2018.

Bryan Wilder, Laura Onasch-Vera, Juliana Hudson, Jose Luna, Nicole Wilson, Robin Petering, Darlene Woo, Milind Tambe, and Eric Rice. End-to-end influence maximization in the field. In AAMAS , pages 1414-1422, 2018.

Aida Rahmattalabi, Shahin Jabbari, Himabindu Lakkaraju, Phebe Vayanos, Max Izenberg, Ryan Brown, Eric Rice, and Milind Tambe. Fair influence maximization: A welfare optimization approach. In AAAI , volume 35, pages 11630-11638, 2021.

Alan Tsang, Bryan Wilder, Eric Rice, Milind Tambe, and Yair Zick. Group-fairness in influence maximization. In IJCAI , pages 5997-6005, 2019.

Aida Rahmattalabi, Phebe Vayanos, Anthony Fulginiti, Eric Rice, Bryan Wilder, Amulya Yadav, and Milind Tambe. Exploring algorithmic fairness in robust graph covering problems. In NeurIPS , pages 15750-15761, 2019.

Ruben Becker, Gianlorenzo D'Angelo, Sajjad Ghobadi, and Hugo Gilbert. Fairness in influence maximization through randomization. In AAAI , pages 14684-14692, 2021.

Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez-Rodriguez, and Krishna P. Gummadi. Fairness constraints: Mechanisms for fair classification. In AISTATS , volume 54, pages 962-970, 2017.

Christian Borgs, Michael Brautbar, Jennifer Chayes, and Brendan Lucier. Maximizing social influence in nearly optimal time. In SIAM , 2014.

Youze Tang, Xiaokui Xiao, and Yanchen Shi. Influence maximization: near-optimal time complexity meets practical efficiency. In SIGMOD , 2014.

Youze Tang, Yanchen Shi, and Xiaokui Xiao. Influence maximization in near-linear time: a martingale approach. In SIGMOD , 2015.

Pedro Domingos and Matthew Richardson. Mining the network value of customers. In KDD , 2001.

Matthew Richardson and Pedro Domingos. Mining knowledge-sharing sites for viral marketing. In KDD , 2002.

David Kempe, Jon M. Kleinberg, and Éva Tardos. Maximizing the spread of influence through a social network. In KDD , 2003.

Jure Leskovec, Andreas Krause, Carlos Guestrin, Christos Faloutsos, Jeanne M. VanBriesen, and Natalie S. Glance. Cost-effective outbreak detection in networks. In KDD , pages 420-429, 2007.

Wei Chen, Yajun Wang, and Siyu Yang. Efficient influence maximization in social networks. In KDD , 2009.

Wei Chen, Yifei Yuan, and Li Zhang. Scalable influence maximization in social networks under the linear threshold model. In ICDM , 2010.

Chi Wang, Wei Chen, and Yajun Wang. Scalable influence maximization for independent cascade model in large-scale social networks. DMKD , 2012.

Kyomin Jung, Wooram Heo, and Wei Chen. Irie: Scalable and robust influence maximization in social networks. In ICDM , 2012.

Jing Tang, Xueyan Tang, Xiaokui Xiao, and Junsong Yuan. Online processing algorithms for influence maximization. In Proc. ACM SIGMOD , pages 991-1005, 2018.

Ana-Andreea Stoica, Jessy Xinyi Han, and Augustin Chaintreau. Seeding network influence in biased networks and the benefits of diversity. In WWW , pages 2089-2098, 2020.

Hervé Moulin. Fair division and collective welfare . 2004.

Benjamin Fish, Ashkan Bashardoust, danah boyd, Sorelle A. Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. Gaps in information access in social networks? In WWW , pages 480-490, 2019.

G. L. Nemhauser, L. A. Wolsey, and M. L. Fisher. An analysis of approximations for maximizing submodular set functions. Mathematical Programming , 14(1):265-294, 1978.

Gerald J Glasser. An unbiased estimator for powers of the arithmetic mean. Journal of the Royal Statistical Society: Series B (Methodological) , 23(1):154-159, 1961.

Hao Yin, Austin R. Benson, Jure Leskovec, and David F. Gleich. Local higher-order graph clustering. In KDD , pages 555-564, 2017.

Nicola Barbieri, Francesco Bonchi, and Giuseppe Manco. Topic-aware social influence propagation models. In ICDM , pages 81-90, 2012.

Jaewon Yang and Jure Leskovec. Defining and evaluating network communities based on ground-truth. In ICDM , pages 745-754, 2012.

Appendix

A Proofs

Fact 2. (Chernoff bound) Let X 1 , X 2 , . . . , X R be R independent random variables with X i having range [0, 1], and there exists µ ∈ [0 , 1] making E [ X i ] = µ for any i ∈ [ R ] . Let Y = ∑ R i =1 X i , for any γ > 0 ,

Lemma 3. Let δ 1 ∈ (0 , 1) , ε 1 ∈ (0 , 1) , and θ 1 = 12 Q 2 ln( C/δ 1 ) ε 2 1 (1 -b ) where Q is the approximation parameter, b = max ( u c ( S ∗ )) , ∀ c ∈ C , and S ∗ = argmax S : | S |≤ k F α ( S ) denotes the optimal solution for the FIM problem based on R , then ˆ F α ( S ∗ , R ) ≥ (1 -ε 1 ) · OPT F holds at least 1 -δ 1 probability if θ ≥ Cθ 1 .

̸

Proof. Let X i c be the random variable for each R i ∈ R ( R i rooted in c ), such that X i c = 1 if S ∗ ∩ R c ( i ) = ∅ , and X i c = 0 otherwise. Let π c = θ c -∑ i ∈ [ θ c ] X i c .

For each community c , let ε ′ 1 = 1 -α ∑ ∞ n =1 η ( n,α ) ( 1 -u c ( S ∗ ) ) n α ∑ ∞ n =1 η ( n,α ) ( 1 -u c ( S ∗ ) ) n · ε 1 , thus ε ′ 1 ≥ ε 1 when α ≤ 1 / 2 , and

For any 0 < γ < 1 ,

To limit Eq.(7) to the first Q ( Q ≥ 2 ) terms, Eq.(7) becomes

Therefore,

It indicates

Therefore,

Pr

{

ˆ

F

α

(

S

∗

,

R

)

<

(1

-

ε

1

)

·

OPT

}

1

-

δ

, thus concludes the proof.

F

1

Lemma 4. Let δ 2 ∈ (0 , 1) , ε 2 = ( e e -1 ) ε -ε 1 , and θ 2 = 8 Q 2 ln( C ( n G k ) /δ 2 ) ε 2 2 (1 -b 0 ) where Q is the approximation parameter, b 0 = max ( u c ( S # )) , ∀ c ∈ C where S # could be an arbitrary fair solution. For each bad S (which indicates F α ( S ) < (1 -1 /e -ε ) · OPT F , ˆ F α ( S, R ) ≥ (1 -1 /e )(1 -ε 1 ) · OPT F holds at most δ 2 / ( n G k ) probability if θ ≥ Cθ 2 .

̸

Proof. Let X i c be the random variable for each R i ∈ R ( R i rooted in c ), such that X i c = 1 if S ∩ R c ( i ) = ∅ , and X i c = 0 otherwise. Let π c = θ c -∑ i ∈ [ θ c ] X i c .

(1 -u c ( S )) Q +1 yθ 2

Therefore,

It indicates Pr { ˆ F α ( S, R ) ≥ (1 -1 /e )(1 -ε 1 ) · OPT F } ≤ δ 2 / ( n G k ) , thus concludes the proof.

B Additional Experiments

B.1 Running Time

The number of RR sets is mainly determined by both the size of the network and the number of communities. In the following, we exhibit the runtime of our algorithm with the scale of networks. Note that our algorithm is currently implemented in Matlab 2022a, thus it costs more time to generate RR sets (generating RR sets in C++ could be at least 100 times faster). RRsets refers to the time (seconds) used to generate RR sets, IMM and FIMM denote the time used to select seeds based on the generated RR sets for IMM and FIMM, respectively.

Note that the proposed algorithm is more suitable and could be much more efficient under scenarios where the number of communities is rather limited ( e.g. , a constant).

B.2 Comparison with Equality-based methods

Our proposed algorithm is based on the notion of welfare fairness, which is in favor of fair result-aware seeding. In this subsection, we aim to explore the difference between such results and communityaware seeding, which is based on the notion of equality. The Equality asks to divide the budget k proportionally to the cluster sizes, i.e. , | S ∩ V c | ≈ k · n c /n G . We adopt two strategies to select seeds under Equality: picking nodes with the highest degree per community and running IMM inside each community.

The dataset tested is the Email network (since the number of communities of other datasets is more than 50 and k is set to 50). The experiment settings are the same as the setting for Email in our Section 4 ( α = 0 . 5 and p ranges from 0.001 to 0.01 with the step of 0.001). The Table 3 below exhibits the results, where S ours , S C-HD and S C-IMM refer to seeds selected by our method, seeds selected by community-aware highest degree, and community-aware IMM, respectively. PoF and EoF are calculated w.r.t IMM in the classic influence maximization problem.

As can be seen from the results, both PoF (the lower the better) and RoF of S ours (the higher the better) are always better than that of both S C-HD and S C-IMM. In other words, S C-HD pays more price of fairness yet achieves a lower degree of fairness. Compared with S C-HD, S C-IMM yields a better performance, Moreover, as p increases, the fair influence of both S C-HD and S C-IMM is even lower (leading to N/A of EoF ) than that of IMM, which does not even contribute to fairness at all. The reason is that community-aware seeding highlights fairness in the process of seed allocation but not selection, while welfare fairness (also, MMF and DC) highlights fairness in the spreading results. The former could have an explicit fair distribution in seeding, but may still lead to unfair results.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes an algorithm for fair influence maximization. The objective is defined as a sum of powers of a utility function for each cluster, indicating that each cluster should receive some influence. Reducing the exponent will bring the objective to a fairer distribution of influence. An algorithm based on max cover on reverse reachable sets is presented. Optimal number of sets are derived to guarantee $(1-1/e - \\epsilon)$-approximation\n\nSTRENGTHS:\n- The key idea to convert the powers into sums using the binomial theorem is elegant, as it helps with modifying the existing algorithm on influence maximization for the proposed objective.\n- Detailed derivations are presented.\n\nWEAKNESSES:\n- Baselines: While the theoretical guarantee is the key contribution, the experiment part of the paper can be strengthened. How much do we gain from this algorithm compared to some simple heuristics? One may consider simple heuristics like degree. Better heuristics may also be generated, e.g., (1) picking high-degree nodes from each cluster, and (2) using the RR sets in Algorithm 1 but performing max-cover without Eq (5) [maybe similar to performing Influence Maximization independently in each cluster]\n\n- The derivation assumes that a fair solution exists, in the form of $b = \\max \\mathbf{u_c} (S^{\\\\#})$. The usage of this appears in the proofs, but some indication of how this would impact the main results could be discussed. \n\n- The symbol $n$ is overloaded. It is used as an index for the summation as well as the number of nodes. I think the paper will benefit from a table of symbols.\n\nQUESTIONS:\n- Please see the issues raised in \"Weaknesses\"\n- Do we need the constraint $\\alpha > 0$? My intuition is that the method will work with negative $\\alpha$ as well, but it needs to be carefully analyzed.\n\nLIMITATIONS:\nMinor issue: A discussion on the impact of the assumptions could be included.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors apply fairness to the influence maximization problem (IM). On a social network, IM models which node subset should be used to trigger the spread of information to maximize its effects. A motivating application is the selection of leaders for natural disaster preparedness, where in existing methods, minority leaders are disproportionately underrepresented. They are not the first to do this, however they are the first to introduce efficient algorithms for solving IM with welfare fairness, which allows the user to parametrize an objection on sliding scale between total fairness and no fairness at all (i.e., maximum influence spread).\n\nThey work in the Independent Cascade model of influence spreading, where each node has a given probability of influencing an adjacent node, and influence is randomly spread according to these probabilities across a number of timesteps. The influence spread, i.e. the objective, is the number of influenced nodes when the process converges. By submodularity, the unfair problem can be solved greedily to a (1-1/e) approximation factor. They assume the given graph is separated into a small number of communities to apply their fairness notion to. While this also admits a greedy approach via Monte Carlo methods, it requires too long to run. \n\nTheir main contribution is a (1-1/e-epsilon) approximate and efficient solution to IM under welfare fairness. En route, they provide an unbiased estimator of the fair objective under a number of random seeds for which edges spread influence when. This is harder than the vanilla problem because the objective is raised to a power alpha in [0,1] as opposed to just 1. They show how to use a Taylor series to provide an unbiased estimator. This is used to then compute the marginal expected gain in influence of adding a vertex to their set and thus greedily select the best one.\n\nThey also experimentally validate their results on five different, semi-synthetic data (e.g., for their Flixster data, they had to construct community information). They show that the price of fairness and effect of fairness both decrease as the alpha parameter increases, which makes sense since that decreases the fairness impact on the objective. They also show that adding more influence budget generally decreases the price and effect of fairness.\n\nSTRENGTHS:\nTo my knowledge, the theoretical results of this paper are substantial and original. It’s always interesting and practical to see how notions of fairness apply to different problems. They are the first to efficiently solve IM under welfare fairness, and their proposal of an unbiased estimator for the marginal influence spread of a vertex seems notable, though I am not particularly familiar with influence literature (e.g., it is possible these methods are pretty standard for influence spreading strategies). From my standpoint, it is a nice result with a solid place in fairness literature.\n\nWEAKNESSES:\nWhile I think the paper is reasonably clear, it can be quite dense at times with its heavy use of notation. I found it particularly hard to follow in Section 3.4, when they pretty much state two lemmas and a theorem, deferring most proofs to the appendix. I think it’s better to explain these things at a high level more and defer formal statements to the end. It was hard to make much of it. In general, I would prefer the paper to undergo more edits for clarity, particularly in Section 3.4, before publication.\n\nQUESTIONS:\n1. A big value of your work is that your algorithm is efficient. Do you have any experiments that evaluate the efficiency?\n2. Can you give a high level discussion of the results from Section 3.4?\n3. For someone not in the area of influence spreading, can you discuss how natural the welfare fairness metric is? It's somewhat unintuitive at a glance.\n4. Big question: what is the runtime of your algorithm?\n\nLIMITATIONS:\nI do not see a limitations section. Most papers have limitations of one kind or another, even highly theoretical ones like this. However, I don’t think it is the most necessary in this case, as no subjects are involved and they are only suggesting a general tool.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors study a variant of the popular influence maximization problem\nthat incorporates fairness constraints. Given a partition of the node set\nof the graph into communities, the goal is to select seeds subject to\nbudget constraints that maximizes influence as well as reduces the\ninfluence gap between communities. The authors consider a particular notion\nof fairness called welfare fairness, where a single parameterized objective\nfunction can be used to achieve the desired balance between maximizing\ninfluence and maintaining fairness. The objective function is monotone\nsubmodular, and therefore, the greedy algorithm guarantees a constant\napproximation factor as in the case of the traditional influence\nmaximization problem. Since the greedy algorithm is not scalable to large\nnetworks, this paper focuses on developing an efficient algorithm using the\nreverse influence sampling (RIS) approach. The main contributions are an\nunbiased estimator for the fractional power of the arithmetic mean using\nTaylor expansion, solving a main hurdle for using RIS approach for welfare\nfairness objective function, and an analysis of the greedy algorithm based\non the RIS approach. Experiments are carried out on five social networks to\nstudy the trade-off between fairness and influence spread.\n\nSTRENGTHS:\nThe paper addresses an important topic of scalability in influence\nmaximization with fairness. The unbiased estimator for the fractional power\nof the arithmetic mean is a significant contribution of this paper, which\ncould be applicable to other problems as well. The paper addresses\nimportant computational and approximation related aspects.\n\nThe paper is generally well written.\n\nWEAKNESSES:\nThe importance of/need for the unbiased estimator is not discussed.\nFirstly, in Section 3.1, it is pointed out that (\\hat{u}_c)^\\alpha is a\nbiased estimator of u_c^\\alpha. Suppose we were to use it, how bad would\nthe outcome be? \n\nA second related issue is that the paper does not comment on the complexity\nof computing the unbiased estimate. When the number of RR sets is large,\nupdating according to equation (5) can be compute intensive (Algorithm 2\nline 11).\n\nAlso, some approximation of equation (5) must be involved as the summation\nover n goes from 1 to infinity. The authors do not seem to discuss about\nthe truncation of the Taylor series and its effect on the accuracy.\n\nIn Theorem 1, the role of \\epsilon is not clear. It does not seem to\nfeature in the probability expression or have anything to do with the size\nof the RR set. In Lemma 4, it is mentioned, but again, its role is not at\nclear.\n\nExperiments section could be much stronger: Considering that the key problem\naddressed by the paper is scalability, the experiments section does\nnot provide any analysis of the computation time of the algorithm and how\nit scales with the size of the networks. It also does not provide any\nanalysis regarding how truncation of higher order terms in the Taylor\nseries affect the performance. Thirdly, there is no analysis of how it\nscales with the size of the RR set. \n\nMinor comments:\nLine 82: \"However, these notions can hardly ...\" It would be better if this\nsentence was supported by references.\n\nLine 197: that covered ... -> that are covered ...\n\nLine 219: then it holds ... -> then the following holds: ...\n\nLine 231: ... holds at least ... -> holds with probability at least ...\n\nQUESTIONS:\nThere are some questions in the Weaknesses that need to be addressed.\n\nIn algorithm 1, line 6, is it sample with replacement?\n\nIn algorithm 1, line 9, what if c(v) is same as c(u)?\n\nIn algorithm 2, line 20, what is \\eta(v)?\n\nWhat is \\Theta in Lemma 3? It is not defined anywhere.\n\nLIMITATIONS:\nLimitations not adequately addressed.\n\nNo negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to study the problem of fairness-aware influence maximization over a community structure under a welfare fairness notion that balances fairness level and influence spread using an exponentially-weighted sum over communities objective with an fractional exponent parameter α expressing inequality aversion. Given an unbiased estimator for the fractional powers from [27], a reverse-influence-sampling approach, adapted from previous works, is applied to the problem that turns the problem into a weighted maximum coverage problem. The number of reverse reachable (RR) samples needed to achieve a good approximation and an algorithm is given that achieves a 1 − 1/e − ε approximation.\n\nSTRENGTHS:\nS1. Revisiting an important problem with an aim to efficiency.\nS2. Lucrative application of reverse influence sampling approach.\nS3. Approximation guarantees.\n\nWEAKNESSES:\nW1. Limited novelty with respect to previous work in [7].\nW2. Lack of discussion of the most basic problem case of full fairness.\nW3. Lack of discussion of and comparison to most recent related work.\n\nQUESTIONS:\nThe solution relies on a submodularity of the objective function and the accompanying analysis in [7]. However, there is no discussion of what happens in the most interesting case from the point of view of fairness, namely as α tends to 0. According to previous works, the objective is not submodular in exactly that case. This matter deserves discussion yet it is not addressed. The work pays no attention to more recents works in the area, such as [A, B]. There is no experimental comparison to the algorithms developed in those works, which, at least the ones in [A], also claim efficiency, and no discussion in terms of the framework presented in [B].\n\n[A] Fish et al.: Gaps in information access in social networks? WWW 2019.\n[B] Farnadi et al.: A unifying framework for fairness-aware influence maximization. Companion of The Web Conference 2020.\n\nLIMITATIONS:\nThe paper offers a discussion on the ethical side of the objective aimed for.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-08"}