Question: "It seems there is no strong motivation behind the poisson factorization model, can we replace it with other types of matrix factorization or neural networks?"
Evidence: The first step of PIF is to construct substitutes for the per-person latent variables ρ i and per-item latent variables τ k by fitting latent variable models to the social network and yesterday's purchases. The idea is that the per-person variables ρ i and z i drive each observed connection a ij in the social network. If we fit a latent variable model to the observed social network that has this same structure, the inferred latent variables will contain some of the same information contained by the confounder ρ i . The same idea applies to the observed data of purchases from yesterday and the confounder τ k . To implement this strategy, we fit models of networks and matrix factorization methods.
The idea behind substitutes was developed in Wang and Blei (2019a), in the deconfounder algorithm for multivariate treatments. Here, we extend these ideas to the social influence setting, where the social network and yesterday's purchases ("treatments" in our setting) are both needed to capture the information in the confounders.
We illustrate the technical ideas behind substitutes by defining them for the per-person variable ρ i . We fit a latent variable model to the observed network a. Standard generative models for networks model the likelihood of observing a connection between person i and person j using Kdimensional latent variables c i and c j on nodes, often called community membership. For concreteness, we will focus on a generative model with a Poisson likelihood, which has been well-studied for modeling social networks . The model assumes that each connection in the network is conditionally independent given the latent variables,
We perform inference in this model over the latent variables c 1:n . Each inferred variable ĉi is a substitute for some of the per-person confounders ρ i . We call the variables ĉ1:n per-person substitutes.
Figure: (Left) The per-person confounder ρ i can be separated into traits u i that affect more than one connection in a i and in traits v i that affect a single connection in a i . (Right) The latent variable model posits the same conditional independence structure over a i as the causal model on the left.
Why are substitutes valid? Figure illustrates the justification behind per-person substitutes. Consider the connections a i made by person i, and for ease of explanation, suppose that there are no traits z i , which only affect the connections a i . Then, Figure shows that the perperson confounder ρ i can be separated into the traits u i (outlined in green) that affect more than one connection in the set a i and traits v i (outlined in red), which affect only one connection. The assumed causal model on the left in Figure implies that each connection a ij ∈ a i is conditionally independent given the per-person variable u i . That is,
Consider the generative model on the right in Figure. We see that it posits the same conditional independence of observed variables a i given the latent variables c i . Assume that the generative model well-approximates the empirical distribution of the network, captured by each row a i of the adjacency matrix. Then, the per-person variables c i that render each observation a ij conditionally independent of the others must coincide with the variables u i posited in the causal model in Figure.
We make this idea concrete with a formal result and proof in the appendix.
When does this not work? The per-person substitute ĉi cannot capture the confounder v i , which only affects one of the connection a i . This fact is because a per-person substitute ĉi can still render the observed connections a i conditionally independent without capturing the trait that affects a single connection. Put differently, such traits leave no observable implications in the data.
What information is captured? The per-person substitute ĉi contains information about both variables ρ i and z i , since both drive the connections a i and render them conditionally independent (Figure). However, only the per-person variables ρ i are confounders. To target ρ-specific information, we can instead fit a joint factor model,
which uses the community membership c to jointly model the network and yesterday's purchases.
We can construct per-person substitutes with inferred variables ĉi as before. We will compare both methods for constructing per-person substitutes in the empirical studies.
We will use the same ideas to find substitutes for the per-person confounders τ k that capture the attributes of an item k that cause people to purchase it (or not). As we noted, finding substitutes relies on models that posit the conditional independence of observed data given latent variables (e.g., Eq. ()). For heterogeneous data such as people's purchases of multiple products, one such a model is Poisson matrix factorization . Thus, to construct substitutes for some of the latent confounders τ k , we fit Poisson matrix factorization to yesterday's purchases,
where Q is the number of factors. After performing inference over the latent variables d 1:n and w 1:m , each inferred variable ŵk is a substitute for the subset of per-item variables τ k that affect more than one purchase in x k = {x 1k , . . . , x nk }. We call each variable ŵk a per-item substitute.
In this section, we will formally state a result about valid substitutes for per-person and per-item confounders.
Proposition 7.1 Let u j ⊂ {ρ j , z j } be the traits of person j that affect more than one connection in a j . Suppose there exists a variable c j so that the distribution p(a j | c j ) factorizes as,
for parameters η a j . Then, the variable c j contains the information that is contained in the set of user traits u j . Let τ k be the attributes of item k that affect more than one purchase in x k . Suppose there exists a variable w k so that the distribution p(x k | w k ) factorizes as,
for parameters η x k . Then, the variable w k contains the information contained in the variable τ k .
Proof Assume that a variable w k which satisfies the factorization above exists. Then, by definition, each pair of variables (x ik , x i ′ k ) in the set x k are d-separated by w k . Suppose that w k does not contain all the information contained in the confounders τ k , common causes of multiple variables in x k . That means, there exists some subset U ⊂ τ k that is not contained in w k but which are parents of at least two variables, (x ak , x bk ) in x k . But, we assumed that w k d-seperates all pairs of variables in x k . By contradiction, w k contains all the information contained in τ k . The same proof applies for the variable c j .
Poisson influence factorization (PIF) estimates social influence from observed social network data and and data from a behavior of interest, such as users purchasing items. PIF uses latent variable models of such data to construct substitutes for unobserved confounders: per-person variables that affect both connections and purchases, and per-item variables that drive purchases. PIF then uses substitutes as drop-in replacements when estimating social influence. We demonstrated the accuracy of PIF for estimating social influence on semi-synthetic datasets. In Section 7.6, we analyzed the sensitivity of PIF to assumption violations. Finally, we applied PIF to study real data from Last.fm.
Future work. In this paper, we made the assumption that influence only spreads behavior in a network from one time period to the next. One avenue of future work is modeling time-series influence. We also focused on Poisson factor models and generalized linear model of outcomes. Another avenue of future work is developing more flexible models of outcomes, networks and purchases. For example, it is worthwhile to study heterogeneous influence effects with outcome models that capture this variation. Finally, we only studied the estimation error of PIF empirically. It is important future work to establish technical results about estimation with finite samples and potentially mis-measured substitutes.
The first step of PIF is to construct substitutes for the per-person latent variables ρi and per-item latent variables τk by fitting latent variable models to the social network and yesterday’s purchases. The idea is that the per-person variables ρi and zi drive each observed connection aij in the social network. If we fit a latent variable model to the observed social network that has this same structure, the inferred latent variables will contain some of the same information contained by the confounder ρi . The same idea applies to the observed data of purchases from yesterday and the confounder τk. To implement this strategy, we fit models of networks (Holland et al., 1983; De Bacco et al., 2017; Ball et al., 2011; Contisciani et al., 2020; Peixoto, 2019; Gopalan and Blei, 2013; Hoff, 2008) and matrix factorization methods (Lee and Seung, 2001; Gopalan et al., 2013, 2015). The idea behind substitutes was developed in Wang and Blei (2019a), in the deconfounder algorithm for multivariate treatments. Here, we extend these ideas to the social influence setting,  where the social network and yesterday’s purchases (“treatments” in our setting) are both needed to capture the information in the confounders. ****We also focused on Poisson factor models and generalized linear model of outcomes.Another avenue of future work is developing more flexible models of outcomes, networks and purchases. For example, it is worthwhile to study heterogeneous influence effects with outcome models that capture this variation. Finally, we only studied the estimation error of PIF empirically. It is important future work to establish technical results about estimation with finite samples and potentially mis-measured substitutes.
Answer: The motivation behind using Poisson community models and Poisson matrix factorization is that they are well-studied, have been successful choices for modeling network and heterogeneous data such as purchases, and admit efficient inference algorithms [1, 2, 3]. We discussed this in section 3.1. We agree that it would be an interesting area of future research to fit neural models of outcomes; we highlighted it as an area of future work in our discussion in section 5.
References:
[1] Ball, B., Karrer, B. and Newman, M.E., 2011. Efficient and principled method for detecting communities in networks. Physical Review E, 84(3), p.036103.
[2] Gopalan, P.K. and Blei, D.M., 2013. Efficient discovery of overlapping communities in massive networks. Proceedings of the National Academy of Sciences, 110(36), pp.14534-14539.
[3] Gopalan, P., Hofman, J.M. and Blei, D.M., 2015, July. Scalable Recommendation with Hierarchical Poisson Factorization. In UAI.