Causal attribution datasets
In this work we compare causal attribution networks derived from three datasets. A causal attribution dataset is a collection of text pairs that reflect cause-effect relationships proposed by humans (for example, “virus causes sickness”). These written statements identify the nodes of the network (see also our graph fusion algorithm for dealing with semantically equivalent statements) while cause-effect relationships form the directed edges (“virus” $\rightarrow $ “sickness”) of the causal attribution network.
We collected causal attribution networks from three sources of data: English Wikidata BIBREF11 , English ConceptNet BIBREF10 , and IPRnet BIBREF12 . Wikidata and ConceptNet, are large knowledge graphs that contain semantic links denoting many types of interactions, one of which is causal attribution, while IPRnet comes from an Amazon Mechanical Turk study in which crowd workers were prompted to provide causal relationships. Wikidata relations were gathered by running four search queries on the Wikidata API (query.wikidata.org). These queries searched for relations with the properties: "has immediate cause", "has effect", "has cause", or "immediate cause of". The first and third searches reverse the order of the cause and effect which we reversed back. We discarded any Wikidata relations where the cause or effect were blank, as well as one ambiguous relation where the cause was "NaN". ConceptNet attributions were gathered by searching the English ConceptNet version 5.6.0 assertions for “/r/Causes/” relations. Lastly, IPRnet was developed in BIBREF12 which we use directly.
The three networks together contain $23\,239$ causal links and $19\,096$ unique terms, of which there are $4\,265$ and $14\,831$ unique causes and effects, respectively.
Text processing and analysis
Each node in our causal attribution networks consists of an English sentence, a short written description of an associated cause and/or effect. Text analysis of these sentences was performed using CoreNLP v3.9.2 and NLTK v3.2.2 BIBREF16 , BIBREF17 . We computed Part-of-Speech (POS) tags and identified (but did not remove) stop words for these sentences. We used the standard Brown corpus as a text baseline for comparison. Text processing procedures such as lemmatization or removal of casing were not performed in order to retain information for subsequent operations. A small number of ConceptNet sentences contained `/n' and `/v' codes within the text denoting parts-of-speech tags; we removed these before applying our own POS tagger. POS tagging of the causal sentences and the baseline dataset was performed using CoreNLP by tokenizing each input using the Penn Treebank tokenizer then applying the Stanford POS tagger. This tagger uses Penn Treebank tags. We aggregated these 36 tags into NLTK's universal tagset which consists of a simpler set of 12 tags including NOUN, VERB, ADJ, and more. To simplify presentation, we chose to further collect all non-verb, non-noun, and non-adjective tags into an “Other” tag. Stop words were identified using NLTK's English stop words corpus.
Word vectors, or embeddings, are modern computational linguistics tools that project words into a learned vector space where context-based semantics of text are preserved, enabling computational understanding of text via mathematical operations on the corresponding vectors BIBREF18 . Many different procedures exist for learning these vector spaces from text corpora BIBREF18 , BIBREF19 , BIBREF20 , BIBREF21 . Document embeddings, or “sentence vectors,” extend word vectors, representing more complex multi-word expressions in a vector space of their own BIBREF22 . Given two nodes $i$ and $j$ with corresponding sentences $s_i$ and $s_j$ and sentence vector representations $\mathbf {v}_i$ and $\mathbf {v}_j$ , respectively, the vector cosine similarity $\frac{ \mathbf {v}_i \cdot \mathbf {v}_j }{ \Vert  \mathbf {v}_i \Vert  \Vert  \mathbf {v}_j \Vert  }$ is a useful metric for estimating the semantic association between the nodes. High vector similarity implies that textual pairs are approximately semantically equivalent and sentence vectors can better compare nodes at a semantic level than more basic approaches such as measuring shared words or n-grams.
We computed sentence vectors using TensorFlow BIBREF23 v1.8.0 using the Universal Sentence Encoder v2, a recently developed embedding model that maps English text into a 512-dimensional vector space and achieves competitive performance at a number of natural language tasks BIBREF24 . This model was pretrained on a variety of text corpora BIBREF24 . The Universal Sentence Encoder was tested on several baseline NLP tasks including sentiment classification and semantic textual similarity, for each of which it performs with the highest accuracy. Given the higher performance of the Universal Sentence Encoder with respect to textual similarity tasks, we elected to utilize it instead of other sentence encoding models including the character level CNN architecture used in Google's billion word baseline BIBREF25 , and weighted averaging of word vector representations BIBREF26 .
Graph fusion
Graph fusion takes two graphs $G_1=(V_1, E_1)$ and $G_2=(V_2,E_2)$ and computes a fused graph $G = (V,E)$ by identifying and combining semantically equivalent nodes (according to some measure of similarity) within and between $V_1$ and $V_2$ . Graph fusion is closely related to graph alignment and (inexact) graph matching BIBREF27 , although fusion assumes the need to identify node equivalents both within and between the networks being fused, unlike alignment and matching which generally focus on uncovering relations between $V_1$ and $V_2$ . Graph fusion is particularly important when a canonical representation for nodes, such as an ID number, is lacking, and thus equivalent nodes may appear and need to be combined. This is exactly the case in this work, where each node is a written description of a concept, and the same concept can be equivalently described in many different ways.
Here we describe Network FUsion with SEmantic Similarity (NetFUSES). This algorithm computes the fused graph $G$ given a node similarity function $f: V \times V \rightarrow \lbrace 0,1\rbrace $ . This $f$ should encode the semantic closeness between nodes $u$ and $v$ , with $f(u,v) = 1$ for semantically equivalent $u$ and $v$ and $f(u,v) = 0$ for semantically non-equivalent $u$ and $f: V \times V \rightarrow \lbrace 0,1\rbrace $0 . We assume $f: V \times V \rightarrow \lbrace 0,1\rbrace $1 and $f: V \times V \rightarrow \lbrace 0,1\rbrace $2 .
To fuse $G_1$ and $G_2$ into $G$ , first compute $F = \lbrace f(u,v) \mid u,v \in V_1 \cup V_2 \rbrace $ . One can interpret $F$ as (the edges of) a fusion indicator graph defined over the combined node sets of $G_1$ and $G_2$ . Each connected component in $F$ then corresponds to a subset of $V_1 \cup V_2$ that should be combined into a single node in $V$ . (One can also take a stricter view and combine nodes corresponding to completely dense connected components of $G_2$0 instead of any connected components, but this strictness can also be incorporated by making $G_2$1 more strict.) Let $G_2$2 indicate the connected component of $G_2$3 containing node $G_2$4 . Abusing notation, one can also consider $G_2$5 as representing the node in $G_2$6 that the unfused node $G_2$7 maps onto. Lastly, we define the edges $G_2$8 of the fused graph based on the neighborhoods of nodes in $G_2$9 . The neighborhood $G$0 of each node $G$1 in the fused graph is the union of the neighborhoods of the nodes connected to $G$2 in $G$3 : for any node $G$4 , let $G$5 and $G$6 Then the neighborhood $G$7 defines the edges incident on $G$8 in the fused graph and $G$9 may now be computed. Notice by this procedure that if an edge already exists in $F = \lbrace f(u,v) \mid u,v \in V_1 \cup V_2 \rbrace $0 and/or $F = \lbrace f(u,v) \mid u,v \in V_1 \cup V_2 \rbrace $1 between two nodes $F = \lbrace f(u,v) \mid u,v \in V_1 \cup V_2 \rbrace $2 and $F = \lbrace f(u,v) \mid u,v \in V_1 \cup V_2 \rbrace $3 that share a connected component in $F = \lbrace f(u,v) \mid u,v \in V_1 \cup V_2 \rbrace $4 , then a self-loop is created in $F = \lbrace f(u,v) \mid u,v \in V_1 \cup V_2 \rbrace $5 when $F = \lbrace f(u,v) \mid u,v \in V_1 \cup V_2 \rbrace $6 and $F = \lbrace f(u,v) \mid u,v \in V_1 \cup V_2 \rbrace $7 are combined. For our purposes these self-loops are meaningful, but otherwise they can be discarded.
Semantic similarity In this work, each node $i$ is represented only by a short written sentence $s_i$ , and two sentences $s_i \ne s_j$ may in fact be different descriptions of the same underlying concept. Hence the need for NetFUSES. To relate two sentences $s_i$ and $s_j$ semantically, we rely upon recent advances in natural language processing that can embed words and multiword expressions into a semantically-meaningful vector space (see Sec. "Discussion" ). Let $\mathbf {v}_i$ be the “sentence vector” corresponding to $s_i$ . Then define $f(i,j) = 1$ if $\frac{ \mathbf {v}_i \cdot \mathbf {v}_j }{ \Vert  \mathbf {v}_i \Vert  \Vert  \mathbf {v}_j \Vert  } > t$ and zero otherwise, for some parameter $t$ . In other words, we consider nodes $s_i$0 and $s_i$1 to be semantically equivalent when the cosine similarity between their vectors exceeds a given threshold $s_i$2 . Our procedure in the main text determined $s_i$3 as an approach threshold.
Capture-recapture
Capture-recapture (also known as mark-and-recapture and recapture sampling) methods are statistical techniques for estimating the size of an unobserved population by examining the intersection of two or more independent samples of that population BIBREF28 , BIBREF29 . For example, biologists wishing to understand how many individuals of a species exist in an environment may capture $n_1$ individuals, tag and release them, then later gather another sample by capturing $n_2$ individuals. The more individuals in the second sample that carry tags, the more likely it is that the overall population $N$ is small; conversely, if the overlap in the samples is small, then it is likely that $N$ is large. Capture-recapture is commonly used by biologists and ecologists for exactly this purpose, but it has been applied to many other problems as well, including estimating the number of software faults in a large codebase BIBREF28 and estimating the number of relevant academic articles covering a specific topic of interest BIBREF30 .
The simplest estimator for the unknown population size $N$ is the Lincoln-Petersen estimator. Assuming the samples generated are unbiased, meaning that each member of the population is equally likely to be captured, then the proportion of captured individuals in the second sample who were tagged should be approximately equal to the overall capture probability for the first sample, $n_1 / N \approx n_{12} / n_2$ . Solving for $N$ gives the intuitive Lincoln-Petersen estimator $\hat{N} = {n_1 n_2}/{ n_{12}}$ , for $n_{12} > 0$ . While a good starting point, this estimator is known to be biased for small samples BIBREF29 , and much work has been performed to determine improved estimators, such as the well-known Chapman estimator BIBREF31 .
In this work we use the recently developed Webster-Kemp estimator BIBREF30 :
$$\hat{N} = \frac{\left(n_1-n_{12}+1\right)\left(n_2-n_{12}+1\right)}{n_{12}} + n_1 + n_2 - n_{12},$$   (Eq. 6)
which assumes (i) that one tried to capture as many items as possible (as opposed to predetermining $n_1$ and $n_2$ and capturing until reaching those numbers) and (ii) the total number of items found $n_1 + n_2 - n_{12} \gg 1$ . Webster and Kemp also derive the variance of this estimator:
$$\sigma ^{2}_{\hat{N}} = \frac{(n_1-n_{12}+1)(n_2-n_{12}+1)(n_1+1)(n_2+1)}{n_{12}^{2}(n_{12}-1)},$$   (Eq. 7)
with $n_{12} > 1$ , allowing us to assess our estimate uncertainty. Equations ( 6 ) and ( 7 ) are approximations when assuming a flat prior on $N$ but are exact when assuming an almost-flat prior on $N$ that slightly favors larger populations $N$ over smaller BIBREF30 .
Results
Here we use network and text analysis tools to compare causal attribution networks (Sec. "Comparing causal networks" ). Crucially, nodes in these networks are defined only by their written descriptions, and multiple written descriptions can represent the same conceptual entity. Thus, to understand how causal attribution networks can be combined, we introduce and analyze a method for fusing networks (Sec. "Fusing causal networks" ) that builds off both the network structure and associated text information and explicitly incorporates conceptual equivalencies. Lastly, in Sec. "Inferring the size of the causal attribution network" we use the degree of overlap in these networks as a means to infer the total size of the one underlying causal attribution network being explored by these data collection efforts, allowing us to better understand the size of collective space of cause-effect relationships held by humans.
Comparing causal networks
We perform a descriptive analysis of the three datasets, comparing and contrasting their features and properties. We focus on two aspects, the network structure and the text information (the written descriptions associated with each node in the network). Understanding these data at these levels can inform efforts to combine different causal attribution networks (Sec. "Fusing causal networks" ).
Table 1 and Fig. 2 summarize network characteristics for the three causal attribution networks. We focus on standard measures of network structure, measuring the sizes, densities, motif structure, and connectedness of the three networks. Both Wikidata and ConceptNet, the two larger networks, are highly disconnected, amounting to collections of small components with low density. In contrast, IPRnet is smaller but comparatively more dense and connected, with higher average degree, fewer disconnected components, and more clustering (Table 1 ). All three networks are degree dissortative, meaning that high-degree nodes are more likely to connect to low-degree nodes. For connectedness and path lengths, we consider both directed and undirected versions of the network allowing us to measure strong and weak connectivity, respectively. All three networks are well connected when ignoring link directionality, but few directed paths exist between disparate nodes in Wikidata and ConceptNet, as shown by the large number of strong connected components and small size of the strong giant components for those networks.
To examine motifs, we focus on feedback loops and feedforward loops, both of which play important roles in causal relationships BIBREF32 , BIBREF33 . The sparse Wikidata network has neither loops, while ConceptNet has 87 feedforward loops and 1 feedback loop (Table 1 ). In contrast, IPRnet has far more loops, 986 feedback and 3541 feedforward loops.
Complementing the statistics shown in Table 1 , Fig. 2 shows the degree distributions ( 2 A), distributions of component sizes ( 2 B), and distributions of two centrality measures ( 2 C). All three networks display a skewed or heavy-tailed degree distribution. We again see that Wikidata and ConceptNet appear similar to one another while IPRnet is quite different, especially in terms of centrality. One difference between ConceptNet and Wikidata visible in 2 A is a mode of nodes with degree $\sim 30$ within ConceptNet that is not present in Wikidata.
Understanding the network structure of each dataset only accounts for part of the information. Each node $i$ in these networks is associated with a sentence $s_i$ , a written word or phrase that describes the cause or effect that $i$ represents. Investigating the textual characteristics of these sentences can then reveal important similarities and differences between the networks.
To study these sentences, we apply standard tools from natural language processing and computational linguistics (see Sec. "Data and Methods" ). In Table 2 and Fig. 3 we present summary statistics including the total text size, average length of sentences, and so forth, across the three networks. We identify several interesting features. One, IPRnet, the smallest and densest network, has the shortest sentences on average, while ConceptNet has the longest sentences (Table 2 and Fig. 3 A). Two, ConceptNet sentences often contain stop words (`the,' `that,' `which,', etc.; see Sec. "Data and Methods" ) which are less likely to carry semantic information (Fig. 3 B). Three, Wikidata contains a large number of capitalized sentences and sentences containing numerical digits. This is likely due to an abundance of proper nouns, names of chemicals, events, and so forth. These textual differences may make it challenging to combine these data into a single causal attribution network.
We next applied a Part-of-Speech (POS) tagger to the sentences (Sec. "Data and Methods" ). POS tags allow us to better understand and compare the grammatical features of causal sentences across the three networks, for example, if one network's text is more heavily focused on nouns while another network's text contains more verbs. Additionally, POS tagging provides insight into the general language of causal attribution and its characteristics. As a baseline for comparison, we also present in Fig. 3 C the POS frequencies for a standard text corpus (Sec. "Data and Methods" ). As causal sentences tend to be short, often incomplete statements, it is plausible for grammatical differences to exist compared with formally written statements as in the baseline corpus. For conciseness, we focus on nouns, verbs, and adjectives (Sec. "Data and Methods" ). Nouns are the most common Part-of-Speech in these data, especially for Wikidata and IPRnet that have a higher proportion of nouns than the baseline corpus (Fig. 3 C). Wikidata and IPRnet have correspondingly lower proportions of verbs than the baseline. These proportions imply that causal attributions contain a higher frequency of objects committing actions than general speech. However, ConceptNet differs, with proportions of nouns and verbs closer to the baseline. The baseline also contains more adjectives than ConceptNet and IPRnet. Overall, shorter, noun-heavy sentences may either help or harm the ability to combine causal attribution networks, depending on their ambiguity relative to longer, typical written statements.
Fusing causal networks
These causal attributions networks are separate efforts to map out the underlying or latent causal attribution network held collectively by humans. It is natural to then ask if these different efforts can be combined in an effective way. Fusing these networks together can provide a single causal attribution network for researchers to study.
At the most basic level, one can fuse these networks together simply by taking their union, defining a single network containing all the unique nodes and edges of the original networks. Unfortunately, nodes in these networks are identified by their sentences, and this graph union assumes that two nodes $i$ and $j$ are equivalent iff $s_i = s_j$ . This is overly restrictive as these sentences serve as descriptions of associated concepts, and we ideally want to combine nodes that represent the same concept even when their written descriptions differ. Indeed, even within a single network it can be necessary to identify and combine nodes in this way. We identify this problem as graph fusion. Graph fusion is a type of record linkage problem and is closely related to graph alignment and (inexact) graph matching BIBREF27 , but unlike those problems, graph fusion assumes the need to identify node equivalencies both within and between the networks being fused.
We introduce a fusion algorithm, NetFUSES (Network FUsion with SEmantic Similarity) that allows us to combine networks using a measure of similarity between nodes (Sec. "Data and Methods" ). Crucially, NetFUSES can handle networks where nodes may need to be combined even within a single network. Here we compare nodes by focusing on the corresponding sentences $s_i$ and $s_j$ of the nodes $i$ and $j$ , respectively, in two networks. We use recent advances in computational linguistics to define a semantic similarity $S(s_i,s_j)$ between $s_i$ and $s_j$ and consider $i$ and $j$ as equivalent when $S(s_i,s_j) \ge t$ for some semantic threshold $s_j$0 . See Sec. "Data and Methods" for details.
To apply NetFUSES with our semantic similarity function (Sec. "Data and Methods" ) requires determining a single parameter, the similarity threshold $t$ . One can identify a value of $t$ using an independent analysis of text, but we argue for a simple indicator of its value given the networks: growth in the number of self-loops as $t$ is varied. If two nodes $i$ and $j$ that are connected before fusion are combined into a single node $u$ by NetFUSES, then the edge $i\rightarrow j$ becomes the self-loop $u \rightarrow u$ . Yet the presence of the original edge $i \rightarrow j$ generally implies that those nodes are not equivalent, and so it is more plausible that combining them is a case of over-fusion than it would have been if $i$ and $t$0 were not connected. Of course, in networks such as the causal attribution networks we study, a self-loop is potentially meaningful, representing a positive feedback where a cause is its own effect. But these self-loops are quite rare (Table 1 ) and we argue that creating additional self-loops via NetFUSES is more likely to be over-fusion than the identification of such feedback. Thus we can study the growth in the number of self-loops as we vary the threshold $t$1 to determine as an approximate value for $t$2 the point at which new self-loops start to form.
Figure 4 identifies a clear value of the similarity threshold $t\approx 0.95$ . We track as a function of threshold the number of nodes, edges, and self-loops of the fusion of Wikidata and ConceptNet, the two largest and most similar networks we study. The number of self-loops remains nearly unchanged until the level of $t = 0.95$ , indicating that as the likely onset point of over-fusion. Further lowering the similarity threshold leads to growth in the number of self-loops, until eventually the number of self-loops begins to decrease as nodes that each have self-loops are themselves combined. Thus, with a clear onset of self-loop creation, we identify $t = 0.95$ to fuse these two networks together.
Inferring the size of the causal attribution network
These three networks represent separate attempts to map out and record the collective causal attribution network held by humans. Of the three, IPRnet is most distinct from the other two, being smaller in size, denser, and generated by a unique experimental protocol. In contrast, Wikidata and ConceptNet networks are more similar in terms of how they were constructed and their overall sizes and densities.
Treating Wikidata and ConceptNet as two independent “draws” from a single underlying network allows us to estimate the total size of this latent network based on their overlap. (We exclude IPRnet as this network is generated using a very different mechanism than the others.) High overlap between these samples implies a smaller total size than low overlap. This estimation technique of comparing overlapping samples is commonly used in wildlife ecology and is known as capture-recapture or mark-and-recapture (see Sec. "Capture-recapture" ). Here we use the Webster-Kemp estimator (Eqs. ( 6 ) and ( 7 )), but given the size of the samples this estimator will be in close agreement with the simpler Lincoln-Petersen estimator.
We first begin with the strictest measure of overlap, exact matching of sentences: node $i$ in one network overlaps with node $j$ in the other network only when $s_i = s_j$ . We then relax this strict assumption by applying NetFUSES as presented in Sec. "Fusing causal networks" .
Wikidata and ConceptNet contain 12 741 and 5 316 nodes, respectively, and the overlap in these sets (when strictly equating sentences) is 208. Substituting these quantities into the Webster-Kemp estimator gives a total number of nodes of the underlying causal attribution network of $\hat{N} = 325\,715.4 \pm 43\,139.2$ ( $\pm $ 95% CI). Comparing $\hat{N}$ to the size of the union of Wikidata and ConceptNet indicates that these two experiments have explored approximately 5.48% $\pm $ 0.726% of causes and effects.
However, this estimate is overly strict in that it assumes any difference in the written descriptions of two nodes means the nodes are different. Yet, written descriptions can easily represent the same conceptual entity in a variety of ways, leading to equivalent nodes that do not have equal written descriptions. Therefore we repeated the above estimation procedure using Wikidata and ConceptNet networks after applying NetFUSES (Sec. "Fusing causal networks" ). NetFUSES incorporates natural language information directly into the semantic similarity, allowing us to incorporate, to some extent, natural language information into our node comparison.
Applying the fusion analysis of Sec. "Fusing causal networks" and combining equivalent nodes within the fused Wikidata and ConceptNet, networks, then determining whether fused nodes contain nodes from both original networks to compute the overlap in the two networks, we obtain a new estimate of the underlying causal attribution network size of $\hat{N} = 293\,819.0 \pm 39\,727.3$ . This estimate is smaller than our previous, stricter estimate, as expected due to the fusion procedure, but within the previous estimate's margin of error. Again, comparing this estimate to the size of the union of the fused Wikidata and ConceptNet networks implies that the experiments have explored approximately 5.77% $\pm $ 0.781% of the underlying or latent causal attribution network.
Finally, capture-recapture can also be used to measure the number of links in the underlying causal attribution network by determining if link $i\rightarrow j$ appears in two networks. Performing the same analysis as above, after incorporating NetFUSES, provides an estimate of $\hat{M} = 10\,235\,150 \pm 8\,962\,595.9$ links. This estimate possesses a relatively large confidence interval due to low observed overlap in the sets of edges. According to this estimate, $0.198\% \pm 0.174\%$ of links have been explored.
Discussion
The construction of causal attribution networks generates important knowledge networks that may inform causal inference research and even help future AI systems to perform causal reasoning, but these networks are time-consuming and costly to generate, and to date no efforts have been made to combine different networks. Our work not only studies the potential for fusing different networks together, but also infers the overall size of the total causal attribution network being explored.
We used capture-recapture estimators to infer the number of nodes and links in the underlying causal attribution network, given the Wikidata and ConceptNet networks and using NetFUSES and a semantic similarity function to help account for semantically equivalent nodes within and between Wikidata and ConceptNet. The validity of these estimates depends on Wikidata and ConceptNet being independent samples of the underlying network. As with many practical applications of capture-recapture in wildlife ecology and other areas, here we must question how well this independence assumption holds. The best way to sharpen these estimates is to introduce a new causal attribution survey specifically designed to capture either nodes or links independently (it is unlikely that a single survey protocol can sample independently both nodes and links), and then perform this same survey multiple times.
NetFUSES is a simple approach to graph fusion, in this case building off advances made in semantic representations of natural language, although any similarity function can be used to identify semantically equivalent nodes as appropriate. We anticipate that more accurate and more computationally efficient methods for graph fusion can be developed, but even the current method may be useful in a number of other problem domains.
Acknowledgments
This material is based upon work supported by the National Science Foundation under Grant No. IIS-1447634.

Question: What are causal attribution networks?
Answer: networks where nodes represent causes and effects, and directed edges represent cause-effect relationships proposed by humans