Experimental assessment of AI-based interactome mapping

Domain: neuroscience
Published: 2026-04-04
Editorial outcome: minor_revision

Abstract
Genotype-phenotype relationships are mediated through intricate networks of physical and functional interactions among macromolecules. Knowledge of the interactome is vital to understand and model genetics and cellular biology. Recent advances in accurately predicting tertiary protein structures using artificial intelligence (AI) approaches such as AlphaFold1 have revived the vision that the protein-protein interactome might be fully predictable through computational modeling of quaternary structures. Here we present a comprehensive experimental framework to systematically assess the impact of AI-driven interactome predictions for yeast2 and human3. We find that the quality of high-confidence predictions is on par with established experimental approaches. However, in proteome-wide screening, the tested AI approaches underperform in the discovery of strictly novel protein-protein interactions (PPIs) compared to experimental reference interactome maps. In particular, the yeast interactome map describe here identifies >40-fold more novel PPIs than its AI counterpart. Strikingly, AlphaFold provides structural models for a substantial number of experimentally identified PPIs miss by the virtual screens. Our results suggest that, at this stage, the main contribution of AI predictions is to provide quaternary structure models for experimentally identified PPIs.

Introduction
Physical interactions among proteins and other biomolecules are the basis of the functional organization of cells. Knowledge about ‘the interactome’ is indispensable for biological discovery from small-scale hypothesis-driven approaches to the development of large-scale models of cellular systems. Consequently, numerous approaches to map the protein-protein interactome of different organisms have been developed4. Despite these efforts, a complete and coherent representation of any physical interactome remains missing. This is due to technical challenges in measuring physical interactions but also the result of different interpretations of ‘physical interactions’, partly driven by the results obtained by different technologies. Thus, interactions may be interpreted as ‘membership of a macromolecular complex’ and then include indirect associations or, alternatively, as direct molecular contacts between polypeptide chains that form an interface with specific bonding. These contacts can be directly affected by sequence changes due to genetic variation or mutation, and can be targeted by small molecules as in drug-target therapeutics.

Recently, the prediction of accurate tertiary structures by AlphaFold1 has revolutionised structural biology. The subsequent development of deep-learning methods to predict quaternary structures of protein complexes, primarily AlphaFold-Multimer5, and subsequently, AlphaFold36, have revitalised the hope of complete and precise computational protein-protein interaction (PPI) discovery without the costs of experiments. Moreover, the availability of models for all quaternary structures, hence of the complete ‘contactome’7,8, would naturally reveal interfaces and render the question of indirect versus direct associations obsolete while facilitating drug discovery and interpretation of genetic variation. However, despite high profile attempts to ‘compute the interactome’2,3 it is unclear how accurate and how sensitive current protein interaction predictions are.

Here, we conduct a comprehensive experimental assessment of structurally resolved interactome predictions for both S. cerevisiae and H. sapiens. Focusing on S. cerevisiae, we first assessed the state of structurally resolved contactome- and interactome-mapping, centering the latter on direct contact interactions (‘binary interactions’, Fig. 1a). Based on these baselines we evaluated the quality and contribution of state-of-the art artificial intelligence (AI) derived predictions to the yeast and human interactome maps. At their current stage of development, the most useful application of AI predictions is to complement existing experimental methods by providing quaternary structure models for novel interaction pairs.

Results
Over half the yeast interactome undiscovered
We started our study by assessing the state of interactome mapping in yeast. Protein interactions with experimentally resolved structures represent contactome maps with the highest level of quality and resolution, however such data exists for only 2,576 heteromeric yeast PPIs between 1,408 proteins (I3D-exp-24)9 (Supplementary Data 1). This corresponds to about 8% of the interactome and with the current growth trajectory it would take more than one hundred years before a structural map of the full contactome becomes available (Supplementary Fig. 1a). Therefore, we next examined the interactome at the level of PPIs, which we define as direct physical contacts between two proteins4. We found that public repositories list ~200,000 protein pairs (Figs. 1b, c), vastly in excess of the estimated interactome size of 20,000 to 40,000 PPIs10–13. Likely, this discrepancy is primarily driven by the inclusion of indirect associations between proteins in large complexes that are not in direct contact14. 83% of the reported pairs were obtained using experimental methods related to affinity chromatography, such as affinity purification mass spectrometry (AP/MS), which, in general, cannot distinguish between co-complex indirect associations and direct PPIs15 (Supplementary Fig. 1b). A second factor possibly inflating the number of reported interactions is low data quality. Despite the perception of literature-curated interaction pairs coming from small-scale studies in which multiple lines of evidence support each interaction14, a majority of the curated yeast protein pairs were supported by only a single piece of evidence, coming from experiments that yielded many pairs (Supplementary Fig. 1c, d). It’s unclear if and which of these studies applied the quality standards necessary for high-throughput approaches to ensure the release of high-quality datasets, or if they simply produced many hits, of which only a few were followed up in the published study and the rest left unvalidated.

To comprehensively and experimentally assess current interactome information and reduce biases, we developed a yeast two-hybrid (Y2H) version, Y2H-v4, engineered to exhibit orthogonal detection characteristics comparable to previous Y2H versions11. This was achieved by using higher copy-number vectors to elevate expression levels, and a longer linker to the activation domain to reduce steric constraints (Supplementary Fig. 1e). We benchmarked this assay against a widely used Y2H version, Y2H-v1, along with the Gaussia princeps protein complementation assay (GPCA)16, and the mammalian protein-protein interaction trap (MAPPIT)17 assay using second-generation positive and random reference sets for S. cerevisiae (scPRS-v2/scRRS-v2). Y2H-v4 yielded an assay sensitivity of 22%, with none of the negative control pairs scored positive (Fig. 1d, Supplementary Fig. 1f, and Supplementary Data 2-4). This is a substantial improvement over Y2H-v1, with 24% of detected pairs undetected by any of the other three assays, demonstrating a substantial level of orthogonality for Y2H-v4 (Fig. 1e) and is therefore well suited to assess the quality of PPI datasets with minimal risk of circularity and bias.

We then used these three assays to systematically evaluate the quality of major yeast interaction datasets representing the full range of mapping approaches, by testing representative samples and comparing the recovery rates against scPRRS-v2 and scRRS-v2 (Figs. 1f-h, Supplementary Figs. 1g-k, 2a-c, 3, Supplementary Note 1, Supplementary Tables 1-3, Supplementary Data 5-12). Specifically, we tested 8,999 pairs in Y2H-v4, which included all protein pairs with experimental 3D structures curated in 2017 (I3D-exp-17), literature pairs with multiple evidence of which at least one can detect binary PPIs (Lit-BM-17), and three systematically generated binary maps (Y2H-union-08) (Fig. 1f and Supplementary Data 10). Since most of the systematic binary approaches and many of the studies that comprise Lit-BM used the Y2H assay, we also tested Lit-BM-13, Y2H-union-08, and DHFR-PCA in GPCA and MAPPIT (Fig. 1g, h and Supplementary Data 11). Of all datasets tested, we found that I3D-exp-17, Lit-BM-13/-17, and Y2H-union-08 tested comparably to the positive control set, whereas the other datasets, which make up the majority of the published pairs, tested at rates closer to the random pairs.

Taken together, our rigorous experimental evaluation of existing yeast interactome datasets shows that only a few represent predominantly direct contacts between proteins. Importantly, interactions in I3D-exp-24 and Lit-BM-24 are concentrated around few highly-studied proteins. In contrast, Y2H-union-08 covers the search space more uniformly, although some of the proteins that were missing from previously used clone collections also have fewer interactions here (Fig. 1i, Supplementary Data 13). We then combined I3D-exp-24, Lit-BM-24, and Y2H-union-08, into a high-quality dataset representing validated binary yeast interactome data in 2024 (ValBin-24), containing a total of 9,817 PPIs (Supplementary Data 14). Using four different estimates of the size of the yeast interactome10–13, ValBin-24 covers between 25% and 50% of the complete PPI network (Fig. 1j, Supplementary Fig. 4a).

In summary, only a small proportion of interactions in public repositories can be assigned to datasets that pass benchmarked experimental validation for binary interactions. Interactions in these datasets cover only a small proportion of the estimated direct interactome in yeast, and so there are a majority of interactions remaining to be discovered by either new experimental or computational approaches. Having established a validated combined dataset of existing PPIs, it is now possible to compare computational and experimental interactome mapping approaches using yeast as a testbed. Notably, of all the experimental methods, systematic high-throughput Y2H-based mapping has proven to be a particularly practical and scalable approach, having produced multiple large interactome maps of reliable and primarily direct PPIs.

A reference map adds novel PPIs
Given the limitations of current maps, we first aimed to investigate how much a experimental effort can add to our current knowledge of the yeast interactome. We used Y2H-v4 as the primary screening assay, as its orthogonal detection profile maximises the potential for discovery (Fig. 1d, e, Supplementary Fig. 1f, Supplementary Data 4). A limitation of previous mapping efforts (e.g. Y2H-union-08)11,18,19 was the incompleteness of available ORFeome collections, each covering only 69-77% of the search space of pairwise protein combinations. To generate a more complete map, we compiled a high-quality ORFeome collection covering 99.5% of the 5,883 yeast protein-coding genes by starting with the existing FLEXGene collection covering 4,933 ORFs20, and cloning an additional 921 ORFs thus generating a collection of 5,854 sequence-validated ORFs (Fig. 2a and Supplementary Data 15). This ORFeome can be readily employed in different assays21 and thus forms a foundation for the systematic completion of the S. cerevisiae interactome map using assays with complementary detection profiles22.

We performed three independent screens of the full search space using Y2H-v4, followed by verification of all detected pairs in two independent pairwise tests. Only pairs that were sequence-confirmed and scored positive in both verification tests were included. Thus, we generated a high-quality map of 1,880 heteromeric PPIs between 1,346 proteins, and 30 homodimeric PPIs, which we refer to as the “Yeast Reference Interactome” (YeRI) dataset (Supplementary Data 16). Dataset quality was then assessed by testing every pair, alongside the scPRS-v2/scRRS-v2 reference sets, in the orthogonal MAPPIT17 and GPCA16 assays: in both assays, the pairs in our dataset were recovered at rates similar to Lit-BM-13 (Fig. 2a, Supplementary Fig. 4b, Supplementary Data 11) demonstrating that the biophysical quality of the generated dataset is on par with high-quality interactions from the literature. We found significant enrichment for interactions between proteins that share annotations for cellular compartments, pathways, or protein complexes (Supplementary Fig. 4c). For a deeper analysis, we made use of profile similarity networks (PSNs) from three systematic and complementary genome-wide functional datasets: i) genetic interaction similarity23; ii) similarity of growth phenotypes of single-gene knock-out (KO) strains in multiple conditions24; and iii) co-expression correlations25 (Fig. 2b). PPI pairs in our dataset were significantly enriched in each of the three PSNs (Fig. 2c), with higher enrichment among more strongly correlated pairs (Supplementary Fig. 4d).

Interactome networks have a modular organization, knowledge of which facilitates systems understanding and functional annotation of unknown genes26. To find functional network modules, higher density maps are advantageous27, while systematic maps avoid the confounding effects of study bias11. Therefore, we combined YeRI with the three previous systematic Y2H maps, Uetz-screen18, Ito-core19, and CCSB-YI111, resulting in Y2H-union-25 comprising 4,307 PPIs (Supplementary Data 17). We found that the number of clusters of interacting proteins enriched for shared Gene Ontology (GO)28 increased substantially with the addition of each map, with new clusters including: Cell Cycle, Kinase Activity, and Transcription (Fig. 2d and Supplementary Fig. 4e). Even though S. cerevisiae is one of the most extensively-studied organisms, to this day the functions of almost one thousand yeast proteins remain unknown29. Y2H-union-25 contains substantially more interactions involving proteins encoded by genes of unknown function compared to the literature-derived map (Supplementary Figs. 4f-h) highlighting the value of Y2H-union-25 in functionally characterising proteins. Altogether, 33% of uncharacterised proteins have at least one interaction in Y2H-union-25. To explore whether these interactions could inform function, we applied a guilt-by-association approach to Y2H-union-25 and found that it could successfully predict gene function (Supplementary Note 2, Supplementary Fig. 4i, Supplementary Data 18).

Notably, the overall sensitivity of YeRI is lower than the 22% scPRS-v2 recovery (Fig. 1d) due to the sampling sensitivity30 of the screen. Adding YeRI to the datasets we previously identified as high-quality binary interactions (ValBin-24) results in a total of 11,349 PPIs. Thus a substantial fraction of the interactome remains to be discovered. Next, we therefore investigated the potential of computational approaches to achieve completion.

Assessing an AI predicted yeast interactome
As an alternative to experiments, computational prediction of PPIs has long offered the promise of a cheaper and faster route to complete contactome maps. Recently, success in protein structure prediction5 has raised the hope that using the same algorithms to predict interacting proteins could map the contactome at a fraction of the cost of experimental approaches. This computational mapping has been performed for the yeast contactome using AlphaFold/RoseTRTAFold (AF/RF)2. To assess the accuracy of these predictions we experimentally evaluated the yeast AF/RF dataset, performing Y2H-v4 pairwise tests of all 1,505 pairs. When binning protein pairs according to their “contact probability” – the confidence score assigned to each predicted structure – recovery rates in the experimental assay improve substantially with increasing contact probability. Specifically, pairs with a contact probability of 0.95 or higher tested positive at rates statistically indistinguishable to scPRS-v2, thus indicating high data quality (P = 0.09, one-sided Fisher’s exact test, Fig. 3a and Supplementary Data 10, 19). We thus restricted subsequent analyses to the 969 out of the 1,505 AF/RF pairs (64%) with contact probability > 0.95, which we refer to as “AF/RF-core” (Fig. 3b). Of these, 325 PPIs (34%) had no previous experimental structure available.

Visualizing the network, we realized that the AF/RF-core formed a highly fragmented network (Fig. 3c). Exploring if this could be due to the smaller size of the map we used random edge removal simulations to compare this to I3D-exp-24, Lit-BM-24, and Y2H-union-08. This analysis revealed a similarity of the two structural networks, and showed that it is not the smaller size giving rise to the different topology (Fig. 3d). Upon closer inspection we noticed that both networks lack highly connected hub proteins (Fig. 3e, Supplementary Fig. 5a), which are important for interconnectivity. The underlying reason is likely that hubs commonly interact with many partners via intrinsically disordered regions31, which pose a challenge for structural approaches, and so might be more amenable to different computational approaches32. Additionally, experimental PPI structures are biased towards stronger interactions, potentially because transient interaction pairs are more difficult to crystallize33. Computationally predicting such PPI structures could offer a way to overcome this bias in experimental data generation. However, as previously observed2, AlphaFold also generates more confident predictions for PPIs with larger interfaces (Fig. 3f and Supplementary Data 20). Together, these observations suggest that AF/RF-core not only recapitulates the bias towards more stable PPIs in its training data but actually increases this bias through the PPIs for which it can generate confident structure predictions. It also recapitulates the bias towards better studied proteins from its training data, as shown by the search space coverage ranked according to the number of publications per gene (Fig. 3g).

Finally, we assessed the novelty of the predicted interaction pairs in the AF/RF-core dataset based on their presence or absence in pre-existing datasets (Fig. 3h). Of the 969 AF/RF-core pairs, 65% correspond to pre-existing experimental structures published elsewhere with either the exact same yeast protein pairs (407 pairs) or homologous protein pairs, referred to as “interolog structures”34 (225 pairs). Of the remaining 337 AF/RF-core pairs, 167 had previously been reported in high-quality binary datasets. Thus, of 969 high-quality pairs in AF/RF-core, 799 (82%) had been identified previously. Among the 170 remaining pairs, 140 reside in previously described co-complex association datasets. Even though these are not strictly novel interactions the protein pairs are informative since AF/RF-core upgraded them from co-complex association status to genuinely contacting interaction pairs. Finally, 30 pairs (3%) of the AF/RF-core dataset can be considered strictly novel PPIs that have not been described in any dataset so far.

This surprising finding raised the question: does such a low rate of true novelty also apply to our experimental dataset? Therefore, we conducted the same analysis for all interaction pairs in YeRI (Fig. 3h). In contrast to AF/RF-core, 1,382 out of 1,880 (74%) of YeRI pairs correspond to strictly novel PPIs not observed previously, whereas 26% were previously observed in experimental structures (135 pairs), interolog structures (93 pairs), high quality binary PPI datasets (169 pairs), and co-complex associations (101 pairs). Thus, the experimental YeRI provides 46-fold more strictly novel PPIs than the AI-based AF/RF-core dataset (1,382 versus 30 interactions).

To provide a time-controlled side-by-side comparison of how computational and experimental approaches perform in finding novel interactions relative to what was known at the time of their release, we plotted the growth over time of the ValBin-25 dataset, comprised of Lit-BM, I3D-exp, AF/RF-core, and Y2H-union-25 (Fig. 3i, Supplementary Data 21). Beginning with the first PPIs reported in yeast several decades ago, the total had increased to 494 PPIs by 2000, the time the first systematic binary dataset (Uetz-screen)11,18 was published. Thus, at that time, with its 577 novel pairs, Uetz-screen provided a delta beyond I3D-exp and Lit-BM of 117%, doubling the number of known yeast PPIs. Subsequent systematic binary maps, Ito-core11,19 in 2001 and CCSB-YI111 in 2008, provided a delta of 44% and 35%, respectively. In subsequent years, Lit-BM and I3D-exp continued their linear growth with over 8,000 PPI pairs reported by the time AF/RF-core and YeRI were released. In this context, AF/RF-core and YeRI provided a 4% and 15% delta of pairs beyond previously known binary S. cerevisiae PPIs, respectively.

Testing an updated AI human interactome
A recent AF/RF predicted human protein interactome map3 – with improvements including a deeper multiple sequence alignment, an updated screening algorithm, and multiple optimised selection cutoffs – computationally screened ~200 million protein pairs and predicted 18,316 interaction pairs with high confidence. This has the potential to outperform the previous approach in yeast. To assess the overall success rate of this approach, we tested a random sample of 3,222 of the predicted pairs in Y2H-v1 alongside hsPRS-v2, hsRRS-v2, and Lit-BM-24 controls. The predicted dataset was generated by integrating multiple different strategies, each with a corresponding threshold that depended on whether the protein pair came from the de novo screen or had prior supporting information, e.g. from genetic interaction datasets or PPI databases. Analysing the Y2H results by these strategies found that, analogous to the yeast predictions, the majority of the data was of high-quality (Fig. 4a, Supplementary Fig. 5b, Supplementary Data 22). Filtering the dataset by increasing the thresholds to obtain recovery rates not significantly lower than Lit-BM-24 in each strategy resulted in an AF/RF-core dataset of 15,799, corresponding to 89% of the predicted PPIs.

Next we assessed the novelty of the interaction pairs. Similar to yeast, the human pairs (12,930 PPIs / 82%) had largely been observed previously in experiments (Fig. 4b). Again, this contrasts to the experimental Human Reference Interactome (HuRI)15 dataset, where only 8,358 PPIs / 16% had been previously observed. Notably, the lower-confidence pairs that we excluded from AF/RF-core were disproportionately strictly novel (Fig. 4c). By plotting human literature-curated binary pairs with multiple evidence, curated experimental structures, and systematic binary maps15,35–38, over time (Fig. 4d), we observe that our three empirically validated systematic maps currently available of the human interactome (CCSB-HI130,35, HI-II-1437, and HuRI15) provided a delta of 39%, 61%, and 100%, respectively, at the time of their release. In contrast, AF/RF-core provided a 10% delta.

Due to the observations that AF/RF-core had lower success with smaller PPI interfaces (Fig. 3f) and missed highly-connected hubs in the yeast network (Fig. 3e), we hypothesised that AI approaches could struggle with domain-motif mediated PPIs, which have smaller interaction interfaces and potentially underlie many hubs in the direct PPI network. We tested this by separating experimental structures into domain-domain and domain-motif subsets, and found that the recovery of domain-motif interactions was less than half of domain-domain PPIs (Fig. 4e, Supplementary Fig. 5c). Since a large proportion of the interactome may be mediated through short linear motifs39, especially among undiscovered pairs, this could offer a partial explanation for why AlphaFold-based approaches have had limited success in discovering novel PPIs.

Integrated contactome mapping strategy
While systematic experimental approaches currently outperform AlphaFold in identifying previously unknown contacting protein pairs, the PPI data does not provide structural resolution (Fig. 1a). We wondered for what proportion of novel YeRI pairs could AlphaFold generate a confident structural prediction. To address this, we first optimised and benchmarked algorithmic assay performance using scPRS-v2/scRRS-v2, removing pairs with existing experimental structures, to avoid training bias. Firstly, we evaluated if AlphaFold36 offered a significant improvement over AlphaFold2 for PPI prediction. Surprisingly, we observed a degradation in performance at low false positive rates due to several highly confident predictions of interactions between random pairs (Fig. 5a and Supplementary Fig. 6a). Therefore we continued this analysis using AlphaFold2. Secondly, we investigated the performance of different scoring metrics, which aggregate the output of the structural prediction to estimate the likelihood that a pair of proteins interacts. Several scoring metrics have been used in different studies, including model confidence5, contact probability2, pDockQ40, and iPAE41. To address a key limitation of existing metrics – their sensitivity to disordered regions outside of the true interaction interface – we also designed and then evaluated the “Confident Contacts Count” (CCC), defined as the number of residue-residue contacts9 with an associated predicted aligned error (PAE) ≤ 4 Å. Indeed, we find that our CCC outperforms the existing metrics at differentiating between real interactions and random pairs (Fig. 5b and Supplementary Data 23). Using CCC with a cutoff set to match the precision of the experimental assays, i.e. to avoid false positive predictions, we found that confident structures could be generated for 37% of the scPRS-v2 pairs (Fig. 5c, Supplementary Fig. 6b). Interestingly, the detection profile of AlphaFold2 complements the evaluated experimental assays, highlighting the benefits of, and likely the need for, combining experimental and AI approaches (Fig. 5d).

Following this optimization we used AlphaFold2 with a CCC cutoff of at least 5 residue-residue contacts with PAE < 4 Å to attempt predicting structures for the novel pairs in YeRI. High-confidence structures could be predicted for 237 of the 1,637 PPIs (14%) without previous structural models in YeRI (Fig. 5e, Supplementary Fig. 6c, and Supplementary Data 24). The lower proportion of successful AlphaFold2 predictions in the novel YeRI pairs relative to scPRS-v2 could be due to a greater proportion of transient interactions, interactions involving disordered regions, and interactions between less evolutionarily conserved proteins detected in YeRI compared to the reference set42, in addition we noticed a bias in the scPRS-v2, which contains PPIs with significantly larger interface sizes on average (P = 1.7 x 10-6 , two-sided Mann-Whitney U test). While the number of new structural PPI models from YeRI (237) is similar to AF/RF-core (325) (Fig. 5f), the modeled YeRI pairs involve more unnamed genes (Fig. 5g) possibly due to the more uniform coverage of the search space (Supplementary Fig. 6d).

Newly discovered PPIs in YeRI, for which high-confidence structural models could be predicted, offer a rich source of mechanistic hypotheses. One example involves the chromatin assembly complex subunit Rlf2, known to deactivate the DNA damage checkpoint following DNA repair43 through an unknown mechanism. The integrated approach reveals a novel structurally modeled interaction between Rlf2 and the DNA damage checkpoint protein Mec3 (Fig. 5h), suggesting a direct regulatory role. Another PPI that could add to our understanding of DNA repair was between the nuclease Pso2 and the sliding replication clamp for DNA polymerase delta Pol30 (Fig. 5i). In addition to illuminating known pathways, several structurally modeled novel YeRI PPIs point to putative functions for uncharacterized proteins. Yjr015w, whose deletion strongly perturbs lipidomics profiles44, interacts with the sterol biosynthesis protein Cyb5 (cytochrome b5) (Fig. 5j). Similarly, Ypl068c, localized to the nucleus45 and expressed in response to the DNA damaging agent MMS46, interacts with Tbf1, a telomeric repeat-binding factor that is also involved in DNA double-strand break repair47 (Fig. 5k).

In total, these 237 novel PPIs with high-confidence predicted structures provide a valuable resource of potential mechanistic insight. The combined experimental and computational approach of discovering PPIs using systematic high-throughput experiments followed by AI structural prediction offers a state-of-the-art method for generating a high-coverage and high-resolution map of the contactome; however, further improvements are needed on both sides to generate a completed map of the protein contactome.

Methods
Yeast strains
Yeast haploid strains MATα Y8930 and MATa Y8800, derived from PJ69-454 and used previously11,55 harbor the following genotype: leu2-3,112 trp1-901 his3Δ200 ura3-52 gal4Δ gal80Δ GAL2::ADE2 GAL1::HIS3@LYS2 GAL7::lacZ@MET2 cyh2R. Yeast cells, parental strains or transformants, were cultured either in YEPD or synthetic drop out media, supplemented as needed and incubated at 30°C.

Bacterial strains
Chemically competent DH5α or DB3.1 E. coli cells were used for all bacterial transformations in this study. Transformed cells were cultured in Luria Broth or Terrific Broth, supplemented with antibiotics (50 µg/ml of ampicillin, spectinomycin or kanamycin) as needed and incubated at 37°C.

Human cell lines
Human embryonic kidney HEK293T cells were cultured in Dulbecco’s Modified Eagle Medium (DMEM) supplemented with 10% fetal bovine serum, 2mM L-glutamine, 100 I.U./ml penicillin, and 100 μg/ml streptomycin. Cells were incubated at 37°C with 5% CO2 and 95% humidity.

Yeast open reading frames
The list of yeast ORFs was downloaded from the Saccharomyces Genome Database (SGD) (https://www.yeastgenome.org/) on January 14th, 2017. Four ORFs (YCR097W/HMRA1, YCR096C/HMRA2, YCL066W/HMLALPHA1, YCL067C/HMLALPHA2) annotated in SGD as “silenced gene” were removed. Only SGD-annotated “Verified” and “Uncharacterized” ORFs were included whereas ORFs annotated as “Dubious” were excluded, leaving a total of 5,883 ORFs with 5,155 and 728 ORFs classified as Verified and Uncharacterized, respectively. All datasets analyzed have been restricted to these 5,883 ORFs and previous ORF names that appear as aliases for one of these ORFs have been mapped to their corresponding new names.

Assembly and description of yeast biological network datasets
Y2H-union-08: Uetz-screen, Ito-core and CCSB-YI1
As described previously11, Uetz-screen is a subset of PPIs from Uetz et al.18,19 that was obtained from a proteome-scale systematic Y2H screen, excluding a smaller-scale, targeted experiment with a smaller number of well-studied bait proteins. Ito-core is a subset of PPIs found three times or more in Ito et al.19, excluding unreliable pairs of proteins found only once or twice. CCSB-YI1 is a proteome-scale dataset of Y2H PPIs validated using two orthogonal assays MAPPIT and yPCA11. After restricting to PPIs involving the 5,883 ORFs (described above) the PPI dataset sizes are as follows: Uetz-screen: 645 PPIs; Ito-core: 816 PPIs; CCSB-YI: 1,772 PPIs. The union of these maps (Y2H-union-08) contains 1,933 proteins and 2,833 PPIs.

Literature-curated biophysical datasets (Lit-CC, Lit-BS, Lit-BM)
Literature-curated pairs were obtained from the databases MINT56, IntAct57, DIP58, and BioGRID59. The data files used were the 2024-11-29 release from IntAct (containing data from IntAct, MINT, and DIP) and BioGRID release 4.4.240 (from 2024-11-25). We excluded evidence corresponding to the systematic, proteome-scale binary and co-complex association datasets11,18,19,60–64. Data was filtered to ensure valid IDs for UniProt accession numbers, Pubmed IDs, and PSI-MI terms. Each piece of evidence for a protein pair had to consist of a Pubmed ID and an interaction detection method term in the PSI-MI controlled vocabulary (http://www.psidev.info/groups/molecular-interactions). Duplicated evidence can arise in cases where different source databases curate the same paper. We merged duplicated entries for each pair, which we identified as multiple pieces of evidence with the same Pubmed ID and experimental interaction detection codes which are either identical or have an ancestor-descendent relationship in the MI ontology. In the latter case, the more specific descendent term was assigned to the merged evidence. In order to select the subset of protein pairs corresponding to binary interactions (as opposed to co-complex associations), we developed a manual classification of the PSI-MI interaction detection method terms37. Our classification has since been updated to cover new experimental methods which have been added to the controlled vocabulary in the intervening time. The methods are classified into three categories; ‘invalid’, ‘binary’ and ‘co-complex’. Where ‘invalid’ corresponds to PSI-MI terms that are not considered valid experimental protein-protein interaction detection methods, ‘binary’ corresponds to terms that detect binary protein-protein interactions and ‘co-complex’ corresponds to terms that detect potentially indirect associations. An example term in the “invalid” category is “colocalization”. All protein pairs annotated with “invalid” terms were excluded. ‘Binary’ versus ‘co-complex’ evidence was used to categorize protein pairs in the literature-curated dataset as follows. Pairs with no binary experimental evidence were classified as “Lit-CC”, corresponding to 158,843 pairs. Pairs with a single piece of binary evidence and no other evidence were classified as “Lit-BS”, corresponding to 15,090 pairs. Finally, pairs with two or more pieces of evidence including at least one binary evidence were classified as “Lit-BM”, corresponding to 6,309 pairs.

Previous literature-curated datasets generated in 2017 and 2013 were used as a source dataset for pairs experimentally tested in GPCA, MAPPIT and Y2H-v4 (see Engineering of Y2H destination vectors) experiments. These were generated and processed as above with small differences. Lit-BM-17 and Lit-BS-17 were obtained via the Mentha resource using the data file dated August 28th 201765. Lit-BM-13/Lit-BS-13/Lit-CC-13 were generated as described previously37. Yeast PPIs annotated through December 2013 from six source databases: BIND66, BioGRID59, DIP58, MINT56, IntAct67 and PDB68 were extracted and processed using the same protocol.

One note: we excluded the evidence from the systematic, proteome-scale binary and co-complex association datasets based on the PubMed ID, but not all pairs associated to those PubMed IDs are included in our definition of the systematic datasets. For example for Ito et al. we just take the Ito-core subset. As a result, the bars in Fig. 1c are a few percent short of the total number of pairs reported in literature-curated databases.

Direct PPIs with experimental structures
The most definitive proof that two interacting proteins are in physical direct contact is the availability of a three-dimensional (3D) structure of their interface. We used the subset of Interactome3D9 restricted to experimental structures, excluding homology models. We re-ran the Interactome3D pipeline specifically on S. cerevisiae experimental structures, ignoring homology models (Supplementary Data 1) to generate the dataset referred to as “I3D-exp-24”, used for most computational analyses. The dataset from the May 2020 release of Interactome3D, referred to as “I3D-exp-20”, was used for some computational analyses, specifically to define the homology models of PPI structures, and to define direct vs indirect contacts within complexes (see the section “Direct or indirect contact in a complex structure”). The dataset from the June 2017 release, “I3D-exp-17”, was experimentally tested in its entirety using Y2H-v4 (see Engineering of Y2H destination vectors). The date assigned to PPIs was obtained from the PDB database taking their earliest release date for all PDB structures from the “complete” Interactome3D dataset.

Note on the overlap between I3D-exp-24 and Lit-BM-24 PPIs
There were a surprisingly large number of pairs in I3D-exp-24 that were absent from Lit-BM-24 (1,414 pairs in I3D-exp-24 that are not in Lit-BM-24, 1,162 pairs in the intersection). Other than those due to the difference between the exact date of generation of the two datasets, the pairs were mostly cryo-EM structures of larger complexes. The reason for this is that in the generation of the literature-curated datasets (see section Literature-curated biophysical datasets), we didn’t use the structural data for direct contacts. Instead, we base the binary vs co-complex distinction on the experimental method used and we classify cryo-EM as co-complex since the information of whether the reported pairs are in direct contact or not is not contained in the data files from the PPI repositories.

AF/RF predicted structures
The list of PPIs for AF/RF came from the excel file captioned “Descriptions of all predicted protein-protein interactions” 2. Six PPIs with missing gene names were discarded. The predicted structures, which were available only for pairs with contact probability ≥ 0.9 were downloaded from https://modelarchive.org/doi/10.5452/ma-bak-cepc.

Systematic AP/MS
AP/MS-06 is made up of Gavin et al. 200261, Gavin et al. 200662, Krogan et al. 200663. We didn’t include Ho et al. 200264, since it was generated with a smaller, more focussed selection of bait proteins and thus was not considered proteome-scale.

Functional profile similarity networks (PSNs)
Genetic interaction similarity profile data (GI-PSN) were extracted from Costanzo et al. 201623. The average PCC of a pair was used if multiple PCCs were available. Pairs with PCCs ranked in the top 1% were used to generate the GI-PSN. Condition-sensitivity data (CS-PSN) was extracted from Hillenmeyer et al. 200824. The log of growth ratios from the homozygous deletion data were used to calculate PCC for each pair of genes. Pairs with PCCs ranked in the top 1% were used to generate the CS-PSN. Gene expression data (GE-PSN) was downloaded from https://coxpresdb.jp25. PCCs were extracted from the union datasets (Sce-m.c3-0 and Sce-r.c1-0, 2018.11.07) and pairs with PCCs ranked in the top 1% were used to generate the GE-PSN. See Supplementary Table 4 for details of the network size and thresholds.

Generation of scPRS-v2 and scRRS-v2
Due to the changes in yeast ORFeome version used here, we updated our positive reference set (PRS) and random reference set (RRS) from our original set11. We named the updated Saccharomyces cerevisiae positive and random reference sets “scPRS-v2” and “scRRS-v2” respectively). In Yu et al., 188 PPIs with five or more papers describing each were finalized as PRS candidates for which 116 protein pairs corresponded to cases where both ORFs were available in the ORFeome collection at the time. Of the 188 PPIs, we filtered for pairs that were also present in Lit-BM-20, and for which both ORFs are available the FLEXGene collection20, resulting in a final scPRS-v2 of 108 PPIs. Of 188 RRS pairs in Yu et al., we removed all ORFs annotated as dubious, then required they have both ORFs in the FLEXGene collection. We then increased the size of the RRS by adding additional pairs randomly selected from the space of all possible pairwise combinations of ORFs in the FLEXGene collection. Since the RRS is used as a negative control, we then filtered out any pairs that appeared in any of the experimental PPI or co-complex association datasets, which resulted in removing one pair that appeared in Lit-CC-20 resulting in a final scRRS-v2 of 198 pairs.

Assembly and description of human biological network datasets
AF/RF predictions for human
Originally, data corresponding to the Zhang et al. biorxiv preprint v169 downloaded from http://prodata.swmed.edu/humanPPI/bulk_download/GoodPairs.txt on 2024-10-05. The information about the GI and PPI-DB sources was obtained from personal communication with the authors (Qian Cong and Jing Zhang). The strategies were recreated using the information in the methods section and validated by comparing to the numbers in Fig 3A of the preprint, finding only small disagreements of 1-3% with the values. These were used to generate the random sample of pairs to test in Y2H. These were subsequently filtered for pairs in the published Zhang et al. Science3 version of the dataset, which was also used for all computational analyses.

Lit-BM-24
Data was generated as for yeast, above.

Engineering of Y2H destination vectors
Gateway compatible 2μ high-copy destination vectors pVV212 and pVV21370 with a Gal4 DNA binding domain (DB) and a Gal4 activation domain (AD), respectively, were modified to be compatible with our standard Y2H vectors pDEST-DB and pDEST-AD-CYH255 with respect to the LEU2 and TRP1 as selectable markers. The resulting destination vectors pDEST-DB-QZ212 and pDEST-AD-QZ213 also carry CAN1 or CYH2 genes as counter-selectable markers, respectively. The CYH2 and CAN1 counterselectable markers facilitate plasmid shuffling for the identification of auto-activators71. Gateway LR reactions between yeast ORFs flanked by attL1 and attL2 sites with the attR1 and attR2 sites of pDEST-DB-QZ212 and pDEST-AD-QZ213 result in attB1 and attB2 sites flanking yeast ORFs now fused downstream of either the Gal4 DB or Gal4 AD sequences of the respective destination vector. See Supplementary Tables 5 and 6 for detailed information.

Benchmarking Yeast Two-Hybrid (Y2H) assay versions
Assay versions were benchmarked using scPRS-v2 and scRRS-v2. The Y2H version with destination clones in vectors pDEST-DB-QZ212 and pDEST-AD-QZ213 was named Y2H version 4 (Y2H-v4) as Y2H-v1, -v2, and -v3 have been described previously15. The performance of Y2H-v4 was compared to Y2H-v1, which consists of destination clones in pDEST-AD-CYH2 and pDEST-DB, and was used to generate CCSB-YI111.

The Y2H assay was performed as described previously37,55. Briefly, Y8930:pDEST-DB-QZ212-ORF and Y8800:pDEST-AD-QZ213-ORF haploid strains were inoculated and mated in YEPD. After enrichment for diploids in synthetic complete media lacking leucine and tryptophan (SC-Leu-Trp), diploids were spotted on SC-Leu-Trp-His solid media containing 1mM 3-Amino-1,2,4-triazole (3-AT), testing for GAL1::HIS3 activation and on a set of SC-Leu-His+1mM 3AT plates supplemented with 10 mg/l cycloheximide (CHX) to identify spontaneous DB-ORF auto-activators55. After 3 days incubation at 30°C, yeast strains growing on SC-Leu-Trp-His+3AT solid media and not on SC-Leu-His+3AT+CHX media were scored as positives. The interacting pairs were identified based on plate position.

Generation of an expanded yeast ORFeome collection
Our version of the Yeast FLEXGene clone collection20 of full length ORFs cloned in either pDONR201 or pDONR221, both KanR, contains 5,296 unique ORFs. After comparing with the SGD annotation (version 2014) (https://www.yeastgenome.org/) we removed 26 obsolete ORFs, 18 ORFs with sequences that did not match the reference, 175 dubious ORFs, 126 intron-containing ORFs, and 18 other ORFs no longer annotated as protein-coding. This resulted in 4,933 SGD-annotated ORFs. For the 950 SGD-annotated ORFs not present in the yeast FLEXGene collections, entry clones were generated in-house and are referred to as “supplemental ORFeome collection”. ORF sequences were amplified without their native stop codon from either S. cerevisiae S288C genomic DNA (ORFs without introns) or cDNA (for ORFs in genes containing introns) using KOD high fidelity polymerase (Novagen) and 18-20 nucleotide ORF-specific forward and reverse PCR primers tailed with Gateway attB1 and attB2 sequences
attB1 Forward primer tail 5’ GGGGACAAGTTTGTACAAAAAAGCAGGCTCCACC
attB2 Reverse primer tail 5’ GGGGACCACTTTGTACAAGAAAGCTGGGTCCTA
from Hu et al.20, essentially as described 72. The CTA sequence in the Gateway tail of the reverse primer provided a synthetic stop codon for all ORFs. Amplified ORFs were transferred to pDONR223 (SpecR) by Gateway BP recombination cloning (Invitrogen) and transformed into chemically competent DH5α E. coli cells. Sanger sequencing of PCR products, generated with universal forward and reverse primers, was used to confirm the identity of all cloned ORFs as described 72. 921 ORFs were obtained using this approach.

The cloned ORFs were longer on average than the previously cloned ORFs in the FLEXGene collection (mean of 1,916 nucleotides compared to 1,359 nucleotides in FLEXGene, P = 9 x 10-9, two-sided Mann-Whitney U test).

ORFeome cloning in Y2H destination vectors
To generate an arrayed library of DB-X and AD-Y expressing strains, the yeast ORFs were transferred into both destination vectors, pDEST-DB-QZ212 and pDEST-AD-QZ213, by Gateway LR recombination cloning (Invitrogen). Gateway LR reaction products were transformed into DH5α E. coli cells, plasmid DNA was extracted and used to transform yeast strains. pDEST-DB-QZ212 and pDEST-AD-QZ213 expression clones were transformed into yeast strains MATα Y8930 and MATa Y8800, respectively55. To confirm the quality of the collection, we verified the identity of a sample of clones from each plate. We sequenced one row (row E) from each plate of transformed yeast cells using SWIM-seq (see Yeast colony sequencing below) and obtained 91% and 90% sequence confirmation rates for yeast DB and AD clones respectively.

Auto-activator detection for filtering before Y2H screening
We tested for auto-activation of the GAL1::HIS3 reporter gene by AD-ORF or DB-ORF fusion proteins in both haploid and diploid yeast cells. To identify auto-activator clones in haploid yeast cells, Y8930:DB-ORF and Y8800:AD-ORF strains were grown to saturation in SC medium lacking leucine (SC-Leu) or tryptophan (SC-Trp), respectively. After 24 hours of incubation, Y8930:DB-ORF and Y8800:AD-ORF haploids were spotted on SC-Leu-His+1mM3AT or SC-Trp-His+1mM3AT to test for GAL1::HIS3 activation. Viability of the haploids was confirmed by growth on SC-Leu or SC-Trp media, respectively. To identify auto-activators in diploid yeast cells, MATα Y8930:DB-ORF and MATa Y8800:AD-ORF strains were mated against their respective opposite mating type strains carrying the corresponding destination vectors without any fused ORFs. Mating was conducted in rich medium, YEPD, and resulting diploids were enriched following growth in SC-Leu-Trp. Diploids were spotted on SC-Leu-Trp-His+3AT, to test for GAL1::HIS3 activation, and on SC-Leu-Trp to confirm the viability of the diploids. For both haploids and diploids, after incubation at 30°C for 3-4 days, strains growing in the absence of histidine and presence of 3AT were considered auto-activators. Altogether 560 DB-ORFs and 1 AD-ORF auto-activators were removed from the final screening collection.

The remaining DB-ORF and AD-ORF clones were re-arrayed into four different groups to separate ORFs with similar nucleotide sequences, defined as BLAST scores of 100 and above. Separation of similar ORFs makes the downstream sequence identification of the short NGS reads more accurate, as the reads are aligned to specific groups of ORFs without sequence ambiguity. Filtering for pairs that passed auto-activator screening and successful cloning resulted in a final collection, which was then used for systematic screening, encompassing 4,778 DB-ORF clones and 5,700 AD-ORF clones, covering a total of 5,854 yeast ORFs.

Primary yeast two-hybrid (Y2H) screening
Three replicate Y2H screens were performed. Individual MATα Y8930:DB-ORFs were mated in YEPD against a pool of ~700 (FLEXGene collection) or ~200 (supplemental collection) MATa Y8800:AD ORFs. AD-ORF pool size was decreased for the supplemental collection to facilitate screening. After enrichment in SC-Leu-Trp, 5µl of the culture was spotted on SC-Leu-Trp-His+1mM 3AT solid media and on SC-Leu-His+1mM 3AT+ 10mg/l CHX to identify spontaneous DB-ORF auto-activators55. After incubation at 30°C for 3 days, strains growing on SC-Leu-Trp-His+3AT but not on SC-Leu-His+3AT+CHX were picked and grown in liquid SC-Leu-Trp. As we used libraries of pools of MATa Y8800:AD-ORF, it is possible to obtain more than one interaction per mini-library. To account for that, we picked up to three colonies per growth spot. Cell lysates were prepared from the saturated cultures and used as templates in PCR reactions to amplify and identify the bait and prey sequences55.

Yeast colony sequencing
To efficiently and cost-effectively identify both bait and prey proteins for thousands of positive colonies, we used a method called SWIM-seq (Shared-Well Interaction Mapping by sequencing) as previously described15. Briefly, DB- and AD-ORFs were simultaneously amplified from 3μl yeast lysate, using well-specific primers. PCR reactions were performed using Platinum Taq (Life Technologies). After PCR amplification, barcoded PCR products from an entire 96 well plate were pooled together and purified (Qiagen, PCR Purification Kit). The pooled amplicons from each plate were subjected to Nextera “tagmentation” using Tn5 transposase to generate a library of amplicons with random breaks to which the adapters were ligated73. We then re-amplified those fragments to generate a library of amplicons such that one end of each amplicon bears the well-specific tag and the other “ladder” end bears the Nextera adapter. A final Illumina sequencing library was prepared by adding plate indexes using the i5 and i7 Illumina adapter sequences. Next generation sequencing was performed with Illumina Solexa technology allowing for identification of interacting first pass pairs of proteins (FiPPs) (see Sequence identification of interacting ORFs). Due to the small number of pairs to be identified, interacting pairs from the first screen of the supplemental space were amplified with the universal AD and DB forward and reverse primers and ORF sequences were identified by Sanger sequencing (Genewiz). All SWIM-primers (Supplementary Table 7) were synthesized by Thermo Fisher Scientific, whereas the universal AD, DB and term primers were synthesized by Eurofins Genomics.

Pairwise test
To confirm all FiPPs, pairwise tests were performed in the same DB-X/AD-Y orientation they were found in the primary screens. Briefly, glycerol stocks from Y8930:DB-ORF and Y8800:AD-ORF haploid strains were inoculated in SC-Leu or SC-Trp, respectively. Saturated cultures were mated in YEPD. After enrichment for diploids, yeast cultures were spotted on SC-Leu-Trp-His+1mM 3AT solid media, testing for GAL1::HIS3 activation. Preliminary investigations using four technical replicates demonstrated that in 97% of the cases, the quadruplicates behaved identically (data not shown). Therefore, given the high reproducibility of technical replicates, the culture was spotted only once per selective media. To increase the robustness of our approach we implemented an additional test to identify de novo auto-activators in which Y8930:DB-ORF strains were mated against a Y8800:AD with no ORF fused to the activation domain (Y8800:AD-Empty ORF) and spotted on SC-Leu-Trp-His+1mM 3AT solid media. Diploids that gave rise to growth on SC-Leu-Trp-His+1mM 3AT media, but did not grow when the respective Y8930:DB-ORF was mated to Y8800:AD-Empty ORF, were selected as positive interacting pairs of proteins. Positive protein pairs were sequence confirmed as done for the primary screens as described above. As positive and negative controls, the scPRS-v2 and scRRS-v2 pairs were distributed randomly across the respective mating plates and tested at the same time. For a batch of pairwise testing to be considered successful we required no more than 1% of RRS and between 10-25% of PRS to score positive.

Sequence identification of interacting ORFs
We used an existing computational pipeline15 to process demultiplexed paired-end reads returned from Illumina sequencing and identify the interacting ORF pairs from the Y2H screens. Paired-end reads are in fastq format, with one read, R1, containing a part of the ORF sequence and the other paired read, R2, containing the well index. We used Bowtie 216 (v2.2.3) to align all R1 reads to reference sequences and extracted the well-identifying indices from the R2 reads. AD-ORFs and DB-ORFs that shared the same well indices were paired together and called FiPPS. To identify likely true AD/DB pairs, we developed a “SWIM score”15 that takes into account the AD and DB reads in each well, total reads returned from the sequencing run, and other factors.
S = (x + y) / (a + M) * (d + N)
where x and y are read counts of an AD-ORF and DB-ORF in a given well, respectively, a and d are total read counts of all aligned AD-ORF and DB-ORF in that well, and M and N are pseudo-counts for AD and DB, respectively, which were constant for each sequencing batch but varied for different batches. We then selected FiPPs for pairwise testing using a cutoff that balances the risk of testing too many false positives FiPPs versus not testing too many true positive FiPPs. The cutoff varied for different screens and sequencing runs to adjust for slight variations in the screening and sequencing protocol.

Validation in orthogonal assays
To assess the precision of various datasets30, PPIs were validated in two orthogonal assays: Mammalian protein-protein interaction trap (MAPPIT)17 and Gaussia princeps luciferase protein complementation assay (GPCA)16. As positive and negative controls, we used pairs of scPRS-v2 and scRRS-v2 respectively. For both assays, expression clones were generated by Gateway LR recombination cloning as described above. Expression clones for GPCA were generated by transferring ORFs into pSPICA-N1 and pSPICA-N2 destination vectors16, each expressing a different fragment of humanized Gaussia princeps luciferase (GL1 and GL2)74. MAPPIT expression clones were generated by LR transfer of ORFs into pMBU-I-2994 and pMBU-I-4199 destination vectors17. After transformation of all expression clones into DH5α E. coli cells, plasmid DNA was extracted and purified using Qiagen 96 Turbo kits (Qiagen) on a BioRobot 8000 (Qiagen). Three different GPCA and two different MAPPIT experiments were performed.

Gaussia princeps protein-fragment complementation assay (GPCA)
GPCA experiments were performed as described previously16. Briefly, on the first day of the assay, ~30,000 to 40,000 HEK293T cells were seeded in each well of a 96 well microtiter plate (Greiner Bio-One). DNA concentration was measured for all clones and samples were diluted to a final concentration of 25ng/μl. After a 24-hour incubation at 37°C, confluent cells were transfected with 300ng of pSPICA-N1-ORF and pSPICA-N2-ORF vectors using polyethylenimine (PEI). After a second 24-hour incubation at 37°C, cells were washed with PBS supplemented with calcium and magnesium chloride. To lyse the cells, 40μl of 5x diluted Renilla lysis buffer (Promega) were added to each well. The plate was then covered with aluminum foil and agitated at 900 rpm for 30 minutes at 37°C for cell lysis. Luciferase activity was measured on a TriStar Berthold Microplate reader by adding 50μl per well of Renilla luciferase substrate (Renilla Luciferase Assay System, Promega), with a measurement time of 4 seconds. The measurement score, RLU (relative light unit), was assigned to the tested pair.

Mammalian Protein-Protein Interaction Trap (MAPPIT)
As an additional orthogonal validation assay, MAPPIT experiments were performed as described elsewhere15,37. In short, HEK293T cells were grown in 384-well plates and co-transfected with a luciferase reporter and plasmids for both bait and prey fusion proteins. Twenty-four hours post-transfection, cells were either stimulated with ligand (erythropoietin) or left untreated, then incubated for an additional 24 hours before luciferase activity was measured in duplicate. The MAPPIT validation experiment was deemed valid, if both bait and prey were successfully cloned into expression vectors and bait expression was detected using a luminometer. “Fold-induction” values (signal from stimulated cells divided by signal from unstimulated cells) were calculated for each tested pair, and two negative controls (no bait with prey and bait with no prey). Each tested pair was assigned a quantitative score: the fold-induction value of the pair divided by the maximum of the fold-induction value of the two negative controls.

Experimental benchmarking of public PPI datasets
PPIs extracted from the biophysical maps described in Supplementary Table 3 were tested in assays Y2H-v4, GPCA, and MAPPIT following the same experimental procedures as described above. A summary of the number of tested pairs in each dataset is available in Supplementary Table 3. Samples, if used, were drawn randomly.

An additional Y2H-v4 experiment was performed to test pairs from the AF/RF dataset, along with scPRS-v2 and scRRS-v2. Roughly half of AF/RF PPIs had already been tested in the first Y2H-v4 experiment, as they overlapped with one or more of the tested datasets. So, in the additional Y2H-v4 experiment we only tested pairs that had not been tested in the first experiment. After checking that the scPRS-v2 and scRRS-v2 results were consistent between the two experiments, the overall recovery of AF/RF was calculated combining the data from both experiments, with every pair having been tested in exactly one of the two experiments.

Y2H-v1 test of AI-predicted human interactome
We tested 4,100 randomly sampled pairs from the 2024 bioRxiv version of Zhang et al.69, along with controls of 250 randomly sampled Lit-BM-24, and 250 random pairs. All were restricted to ORFs in our collection with identical amino acid sequences to the reported UniProt entries. We also included hsPRS-v2, and hsRRS-v2 controls. In analysis these pairs were subsequently filtered for those contained in the 2025 Science version of Zhang et al.3, resulting in 3,222 tested pairs.

We pairwise tested these in Y2H-v115. Compared to Y2H-v4, v1 utilizes a different AD vector (pDEST-AD-CYH2) and DB vector (pDEST-DB) which are centromeric, low copy plasmids (Supplementary Table 8). Briefly, each ORF was cherry picked from our existing human ORFeome collection (hORFeome v9.1) and MATα Y8930:DB-ORF yeast strains (bait) were mated against MATa Y8800:AD-ORF yeast strains, respectively. Each DB-ORF was also tested against an “AD-null” plasmid without any ORF in the cloning site to control for de-novo autoactivation.

After growing the yeast for 2 days at 30°C, mating was performed by adding 5ul AD and 5ul DB yeast culture into 200ul YEPD. The next day, 10ul of mated yeast were transferred into 200ul SC-Leu-Trp media to select for diploid yeast cells. After growing the cells overnight at 30°C, 5ul of the yeast cultures were spotted on selective media (SC-Leu-Trp-His+1mM 3AT and SC-Leu-Trp). After growing them for 3 days at 30°C and 1 day at room temperature, the plates were scored for growth. Spots that did not grow on SC-Leu-Trp were annotated as NA (not successfully tested). Pairs that grew on SC-Leu-Trp-His+1mM 3AT were picked into SC-LW to make lysates and perform barcoded 96-well PCR for AD and DB ORFs. To confirm pairs that tested negative (no growth on the 3AT plate), we also sequenced the SC-Leu-Trp plates. Therefore, 10ul of each well from all fifty 96-well Costar plates were stacked using a 96-well liquid handling robot.

Each well was then purified with the ChargeSwitch™ Plasmid Yeast Mini Kit (ThermoFisher Scientific, CS10203) and PCR amplified with 96-well barcoded AD and DB-forward and barcoded reverse primers. All samples were pooled, purified and sequenced with Nanopore sequencing (Plasmidsaurus). 90% of the positive pairs were sequence confirmed, 83% of the SC-Leu-Trp samples.

Direct or indirect contact in a complex structure
We queried Interactome3D (version 2020_01)9 for complexes involving three or more proteins with an experimental structure available. For all combinations of protein pairs within a complex the number of residue-residue contacts was calculated with the code and method used in Interactome3D, by accounting for hydrogen bonds, van der Waals interactions, and salt and disulfide bridges. We defined protein pairs with five or more contacts as direct, and remaining pairs as indirect. Using this annotation for each dataset, the fraction of direct PPIs was calculated as the number of direct PPIs reported in the dataset divided by the number of direct and indirect pairs reported in the dataset.

List of genes of unknown function
A list of 979 S. cerevisiae genes of unknown function was obtained from Table S9 of Wood et al. 201929, of which 950 were in the list of yeast ORFs considered for this study (see section Yeast protein-coding ORFs).

Publications per gene
Number of publications per gene was extracted from the gene2pubmed file from NCBI, downloaded on 2018-08-01.

Domain-Domain and Domain-motif structural interaction datasets
Domain-domain interactions (DDIs) were taken from 3DID75 release 2024-12. Domain-motif interactions (DMIs) were from the ELM76 interaction set dated 2024-01-30. Both DDI and DMI datasets were then filtered for only PPIs present in I3D-exp. The small number of pairs that were in both the DDI and DMI sets were removed from the DDI set.

Treatment of heterodimers and homodimers
Unless otherwise noted, homodimers were excluded from most analyses since comparisons between physical interactions and functional relationships are obviously not applicable to single genes (all PCC values of functional profiles would be 1.0 by definition).

Calculation of recovery rates in Y2H-v4, MAPPIT and GPCA
In MAPPIT and GPCA assays, pairs were scored positive or negative based on thresholds set by the highest scoring scRRS-v2 pair in the corresponding experiment. For all three assays, pairs without valid quantitative scores were dropped, and recovery rates were calculated as the number of positive pairs over the sum of the positive and negative pairs. The error bars on the recovery rates were calculated using a Bayesian model (a binomial likelihood with a uniform prior), taking the central 68.27% interval of Beta (p + 1, n + 1), where p and n are the numbers of pairs testing positive and negative, respectively. P-values for the difference in recovery between two datasets tested in the same experiment are calculated using Fisher’s exact test, two-sided in all cases except when testing a dataset against the scPRS-v2 / scRRS-v2 positive or negative controls, where a one-sided test is used. For the yeast AF/RF Y2H-v4 results, where the data was split across two experiments, the scPRS-v2 recovery is calculated as an average of the two experiments, weighting by the number of positive AF/RF pairs in each experiment. For the human AF/RF Y2H-v1 results, sliding window averages were unweighted and used a window size of 250 successfully tested pairs, and error bands were calculated identically to the error bars.

Calculation of interface areas of PPIs
We retrieved experimental structures using Interactome3D version 2020_059. The interaction interface area was calculated using pymol v3.0.0 as half the difference between the Solvent Accessible Surface Area (SASA) of the sum of the isolated individual subunits compared to the complex.

Residue-residue contacts are defined as in Mosca et al.9. The definition is:
(i) covalent interactions (disulfide bridges), defined as two sulfur atoms of a pair of cysteines at a distance ≤ 2.56 Å (two times the covalent radius of sulfur plus 0.5 Å);
(ii) hydrogen bonds, defined as all atom pairs N-O and O-N at a distance ≤ 3.5 Å;
(iii) salt bridges, defined as all atom pairs N-O and O-N at a distance ≤ 5.5 Å;
(iv) van der Waals interactions, defined as all pairs of carbon atoms at a distance ≤ 5.0 Å

Interaction 2D histogram heat maps
For a particular gene/protein property and a network, we ranked all proteins using that property. Tied values were sorted randomly. The proteins were split into an equal number of bins, creating 2D bins of the protein-by-protein space. The number of edges in the diagonal bins was multiplied by a factor of N2 / (N2 / 2 - N / 2), where N is the number of proteins in the bin, to correct for the smaller number of possible pairwise combinations, since edges were undirected. Homodimeric interactions were excluded.

Calculation of enrichment for connecting proteins in the same subcellular compartment, pathway, and complex
Subcellular compartment data was obtained from CYCLoPS77, using the WT data, annotating a protein to a compartment if it has any non-zero value in any of the three repeats. Pathways were obtained from KEGG78. Complexes were obtained from CYC200879. The number of PPIs that connected two different proteins in the same compartment, pathway or complex was divided by the mean value for 1,000 degree-preserved randomized networks, generated using the Viger and Latapy algorithm implementation through python iGraph v0.11.480, and CI values were taken from the innermost 68.27% of the random networks.

SAFE network visualization
We used the SAFE network visualization tool (v1.5)28. The layouts were generated with Cytoscape (v3.4.0)81 using the edge-weighted spring embedded layout. GO terms were downloaded from SGD database (version on Jan 17th 2019) and GOE28 terms enriched with P < 0.05 were colored and labeled. SAFE analysis was run with the default option except layoutAlgorithm = none (using layout generated by Cytoscape), neighborhoodRadius = 200, and neighborhoodRadiusType = absolute.

Estimates of the complete yeast interactome size
We used four estimates, relying on partially overlapping assumptions and data, made by independent groups, that predicted the yeast protein binary interactome contains between ~18,000 and ~38,000 direct binary interactions, corresponding to ~0.1-0.2% of all ~18,000,000 possible protein pair combinations11.
● From Grigoriev 200313. 19,500 (15,000-24,000). The reported range was 16,000-26,000 but was calculated using a size of 6,300 protein-coding genes, so we adjusted the values using our value of 5,883. No central value was reported and so we used the middle of the range.
● From Yu et al. 200811 18,000 (13,500-22,500 95% CI). Taken from Page 107: “we estimated that the yeast binary interactome consists of ~18,000 +- 4,500 interactions (SOM VI)” From SOM VI the +/- refers to the 95% CI.
● From Stumpf et al. 200812 28,472 (26,650-30,460 95% CI). Taken from the Uetz et al. numbers from Table 1. We use the estimate made using Uetz et al. because three of the other datasets contain indirect protein-protein associations 58,61,64 and the estimate using Ito et al. uses the full dataset, mainly made up of the ‘Ito-noncore’ subset that was shown to be of poor quality when retested using Y2H and PCA 11.
● From Sambourg and Thierry-Mieg 201010 37,600 (32,252-43,472 95% CI). Taken from Page 6: “Taken together, this allows us to estimate the size of the binary yeast interactome at ~ 37,600 interactions (95% confidence inter- val: 32252-43472, constructed with the normal approximation method).”
One relatively minor difference between the estimates is that Stumpf et al. are considering only heterodimeric PPIs whereas Yu et al. and Sambourg et al. are also counting homodimeric PPIs. We account for this when estimating the fraction of predicted interactome mapped by excluding homodimers for the Stumpf et al. estimate and including them for the Yu et al. and Sambourg et al. estimates.

The single combined meta-estimate in the main figure was generated using the HKSJ random effects model.

Prediction of gene functions using guilt-by-association approach
In the guilt-by-association approach the function of a node (protein) is inferred from the function of its neighbors. In particular, for each node we count the number of its neighbors annotated with a given function (n). This score is then compared to a random benchmark, obtained by randomizing the network 2,000 times in a degree-preserved way. Calculating the z-score, z = (n-n̄)÷σ, is the traditional way of such comparison, obtained by standardizing the original score with the expectation value (n̄) and standard deviation (σ) of the score that would be expected by chance. Yet, the z-score is not free from degree biases and prefers low degree nodes with extremely small σ. We therefore apply a related measure, called the effect size. The effect size n-(n̄ +ασ)n̄ is obtained by comparing the original score with the reasonably expected value of the random benchmark, estimated as the mean value (n̄) and α-times the standard deviation (σ). In practice, we used α = 5, selecting the same candidates as a traditional z-score threshold of z ≥ 5, but ordering them based on the amount of signal beyond random expectations to avoid a bias towards low-degree nodes. Functional annotations of genes with GO Biological Process terms were obtained as described above and further restricted to annotations with the experimental evidence codes EXP, IDA, IPI, IMP, IGI, IEP, HTP, HTP, HDA, EHMP, HGI, and HEP.

Network fragmentation
The error bands are generated from randomly sampling a fraction of the PPIs from each network, where the fraction varies from 5% to 95% in 5% increments, with 1,000 random subnetworks generated at each point.

Degree distribution plots
Degree distributions were plotted according to Chapter 4, Advanced Topic 3.B of Network Science82, on a log-log scale with logarithmic binning, with the unbinned data shown in grey.

Thresholds for AF/RF confidence based on Y2H test results
For the yeast AF/RF, which was generated using a single threshold, the predicted pairs were divided into six bins of contact probability of roughly equal size, and the -core cutoff was defined as the start of the first bin whose Y2H-v4 positive rate was not significantly lower than that of scPRS-v2, which was 0.95. For the human AF/RF, which was generated using multiple thresholds, the positive rate was calculated across a sliding window of 250 pairs, separately for each strategy, using the corresponding score for each strategy, and the threshold was defined as the low end of the lowest score window to not be below the lower bound of the confidence interval on the Lit-BM-24 positive rate. The derived thresholds were 0.9696 for strategy 3 (genetic interactions → RF2-PPI), 0.9787 for strategy 5 (de novo screen on pairs with common cellular compartments → AF2), and 0.9800 for strategy 7 (genetic interactions → RF2-PPI → AF2).

Observation in previous datasets
Experimental structures were defined as I3D-exp-24 (yeast) I3D-exp-20 (human). Interolog (homologous PPI) structures were defined by first performing a sequence alignment of all translated ORFs from SGD (yeast) or the UniProt reference proteome UP000005640_9606 (human) against the a.a. sequences in the PDB (https://ftp.ncbi.nlm.nih.gov/blast/db/FASTA/pdbaa.gz downloaded 2024-02-06), using mmseqs with a E-value threshold of 10-5, then any PDB structure where any two different chains were matched to the two different interacting proteins was considered an interolog. High-quality binary PPIs referred to ValBin-24 (yeast) and Lit-BM-24 + HI-union (human). Co-complex associations referred to all other literature data for biophysical interactions and presence within the same complex from EBI Complex Portal (release 2026-01-09). For AF/RF-core (yeast) the previous datasets were restricted to those published before its publication date of 2021-11-11.

AlphaFold structure predictions
Predicted structures for pairs in YeRI, scPRS-v2, and scRRS-v2 were generated with AlphaFold version 2.3.1 5. With options:
--model_preset=multimer
--db_preset=full_dbs
--max_template_date=2023-05-05 --num_multimer_predictions_per_model=1
--enable_cpu_relax
--use_precomputed_msas.
We could only predict structures for the pairs whose combined number of amino acids were 2,500 or less. We extracted the predicted aligned error (PAE) matrix from the AlphaFold pickle output using the alphapickle script v1.4.1 (https://github.com/mattarnoldbio/alphapickle DOI:10.5281/zenodo.5752375). We calculated five different confidence metrics for each structure
1. Model confidence (0.8 ipTM + 0.2 pTM)
2. Contact probability (max probability of a pair of residues between the two proteins < 12 Å)2
3. pDockQ (logistic function of interface plDDT and number of contacts)40
4. iPAE 41 (median interface PAE)
5. CCC (Number of residue-residue contacts with PAE < 4 Å)
Each putative residue-residue contact in an AlphaFold-multimer predicted structure also has a corresponding predicted aligned error (PAE) in Angstrom.
AlphaFold3 predictions were obtained from the AlphaFold Server6.

Confident Contacts Count (CCC)
We designed the CCC metric with the expectation that it should be more robust than the contact probability because it is not driven by only a single pair of high-confidence proximal residues. Additionally, CCC should avoid the issue with most alternative metrics which, by averaging the confidence scores of all interface residues, can miss a high-confidence interface if it is obscured by another interface predicted with low confidence. The 4 Å cutoff on PAE was an arbitrary choice reasoned to be around the length of the non-covalent bonds.

Precision-recall curves
We calculated balanced precision and plotted monotonized precision-recall curves as described by Wu et al.83.

PPIs with no structural model
Yeast PPIs with no structural model were defined as not being present in I3D-exp-24, not having an interolog structure (defined above), and not being in the full AF/RF dataset (i.e. the dataset before applying the higher, experimentally-derived contact probability threshold).

Data availability: Protein interaction data have been submitted to the IMEx consortium (http://www.imexconsortium.org) through IntAct57 and assigned the identifier IM-30553 [https://imexcentral.org/icentral/pubedit.action?id=30553]. Predicted structures of YeRI PPIs are deposited at doi:10.5281/zenodo.18601049 [https://zenodo.org/records/18601049]. YeRI, Y2H-union-25, and ValBin-25 maps are available at our OpenPIP84 website: https://yeast.interactome-atlas.org/. Source data are provided with this paper.

Code availability: Analysis code is available at https://github.com/ccsb-dfci/ai-interactome-experimental-assessment, archived together with the input data at doi:10.5281/zenodo.18499797 [https://zenodo.org/records/18499797].

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_1": "Report on Lambourne et al. « Experimental assessment of AI-based interactome mapping «\nLambourne and team provide a straight forward (in the sense of well written and illustrated) comparison of two AF complex\nprediction data set, one from yeast and one from human (AF/RF, still preprint), with experimental Y2H PPI data. Intriguingly,\nthe (novel and interesting) outcomes from the comparisons for yeast and human are very, very similar for both species: The\npredictions are of high quality, they can be experimentally corroborated to a similar rate than known positive reference PPIs.\nThere are no hubs, the AF predicted networks appear disintegrated, likely because PPIs with IDPs are missing. Interfaces\nare larger, domain-domain PPIs are predominant, there is a higher knowledge bias, and most importantly the AF predictions\ncontribute very few novel PPIs only. Experimental PPI discovery is warranted. Along this conclusion, a new Y2H yeast PPI\ndata set is presented, high quality. It is derived from the all vs all Y2H mapping of 5854 ORFs and adds 1880 PPIs. The\ncombined ValBin-25 PPI yeast dataset has resource character. The study ends by using AF to predict the 3D structure of\nreference pairs and novel PPIs to demonstrate how AF/RF can contribute to interactome research. In summary, this work is a\nsignificant contribution!\nPoints for consideration.\na) The authors use a new Y2H-v4 vector setup which it is claimed to be an orthogonal system with respect to previous\nscreens. However, the screen delivers very sparse data. Considering the high sensitivity (22%, 1d), I do not see how the\ndata go along with interactome size predictions. In Figure 1J the val-bin24/25 is between 25-50% of the interactome size and\ncounts about 10000. The screen covers roughly all pairs (“a foundation for the systematic completion of the S. cerevisiae\ninteractome map”) and detects 1880/0.22= 8545 PPIs. Please, consider that the sensitivity using the PRS is a substantial\noverestimate.\nb) As mentioned above, the study ends by using AF to predict the 3D structure of reference pairs, 37% success and some\nnovel PPIs with 14% success. The authors provide hypothesis for the difference, which is not reflected in the experimental\nvalidation. While I go well along with the offered explanations, the authors could add at least two more limitations to the\nanalysis. This is the PRS is more like the domain-domain AF data, or at least theoretically the new yeast data are not as\nreliable. What is the average surface area of RPS pairs etc ...\nc) After detailed analysis of yeast data (figs 1-3 ) the authors perform the analyses of human data, with analogous outcome.\nWhile these results are very well supported and thus convincing, the human time line is not. In analogy, the human time line\n(Figure 4d) should include (similar to yeast Figure 3i), also the non-Vidal-Lab large scale data sets that contributed to the\nbinary interactome mapping. The Vidal lab contributed the most significant data sets, but not exclusively. Including the most\nsignificant data sets also from other labs will strengthen the point of the Figure.\nd) It should be note that while I strongly agree that the novel PPIs in the human AF paper (which I have read carefully) are\nvery few in number, they mostly relate to membrane-membrane PPIs, where Y2H is negatively biased. Therefore, the novel\nAF PPIs could be largely complementary to the experimental binary PPIs. This needs to be checked e.g. in terms of which\nproteins are involved etc. How many of the strictly novel PPIs of the human AF/RF core have been retested with Y2H-v4 in\nthe presented experiments? I suggest a similar panel like the ones in Figure 4a that show the success of de novo screens\nwould be revealing if shown for strictly novel PPIs (provided that they were tested).\ne) The predictions of novel Y2H were done with success even though the pairs were not in the AF predicted data set (maybe\ndifferent scoring, ccc). However, where these novel pairs considered in the AF predictions in the first place? Did they end up\nbelow cut off (cf . Figure 4c). The human predictions were guided by known pairs though ...", "reviewer_2": "The manuscript by Lambourne et al consists of three parts. In the first, the authors assess the current state of interactome\nmapping for budding yeast. Using three binary interaction assays, they test subsets of pairs from various interaction datasets\nto evaluate their quality. Most of the interactions are retested using an updated version of the Vidal lab yeast two-hybrid\n(Y2H) pipeline that is able to positively identify more interactions (higher sensitivity) while maintaining a low false-positive\nrate. The main conclusion is that most datasets are of low quality in terms of accurately reporting direct protein-protein\ninteractions, possibly in part because many datasets report indirect associations (i.e. proteins in a larger complex) in the form\nof interacting pairs. The authors combine the high-quality datasets into a single dataset they term ValBin-24 for validated\nbinary interactions in 2024. This section provides a thorough and thoughtful overview of the current state of interactome\nmapping.\nIn the second part of the manuscript, the authors experimentally expand the yeast binary interaction map using i) their\nupdated Y2H pipeline, and ii) a near complete yeast ORFeome collection generated here by the authors (adding 921 ORFs\nto an existing collection of 4933 ORFs). This adds 1723 interactions to previous Y2H high confidence interactions, for a total\nof 4556 interactions. Both the expanded ORFeome collection and new interactions are of high value to the scientific\ncommunity.\nIn the third part, the authors assess the accuracy of yeast and human protein interactions predicted by new structure\nprediction algorithms. They do this by testing all predicted yeast interactions and a random sampling of human protein\ninteractions in their Y2H pipeline. I found this an interesting analysis to read. The main conclusion is that while predictions\nare of high quality, very few new interactions are predicted. The likely reasons for this are that structural prediction\nalgorithms are better at prediction structures for proteins similar to proteins with know structures, and are not good at\nidentifying interactions with disordered regions, which mediate many protein interactions. While these observations are not\nnecessarily unexpected, it is very useful to have a thorough examination of the predictive power of new AI methods.\nOverall, the work provides an advancement in our knowledge of protein-interactions and in tools (Y2H system and new\nORFeome dataset). The evaluation of AI methods should be of broad interest and is an important step in assessing the use\ncases for these new tools.\nI have little if any qualms with the experimental approaches used. The Vidal lab are absolute experts in the field of binary\nprotein interaction mapping, having honed their Y2H pipeline over 2 decades of work and having increasingly emphasized\nthe value of rigorous benchmarking of interaction assays with positive and negative reference sets. The comparisons with\nAlphafold predictions also seem to have been thoroughly performed, with attention to the computational methods and\nthresholds used. My comments for improvement are below, most address readability.\n- A table with all the various datasets mentioned would be very helpful, even being familiar with past work from the Vidal lab\nthe many dataset abbreviations confused me. Most are explained in the methods, but a overview table to refer to while\nreading would be a welcome addition.\n- The multiple Lit-BM sets are not well explained. On page 4 line 28, the authors state that interactions from Lit-BM-17 were\ntested, but on line 32 it seems a Lit-BM-13 was also tested. What is the difference?\n- On page 5 line 26, why are the new interactions compared to Lit-BM-13? In figure 1, Lit-BM-17 was used.\n- Page 5 line 36: the sentence ‘thus we generated a new high quality map’ would be better placed on line 23, before going\non to evaluating the new map. The way it reads now I was wondering if the new map was generated using criteria based on\nthe comparison with the PSNs, e.g. only incorporating those that were enriched in one or more PSNs. (or if that is actually\nthe case, clarify the criteria).\n- The naming of the Yeast Reference Interactome seems and odd choice. What makes this a reference interactome, why\nwould CCSB-YI1 not be a reference interactome, or the next interactome the Vidal lab generates. Wouldn’t a year name be\nmore consistent with their nomenclature, e.g. CCSB-YI25? Or YI2? The analyses below are also not done on YeRI but on\nunion-25 and on ValBin-25, so it isn’t used as a reference set here either.\n- The combining of YeRI with ‘the datasets previously identified as high-quality binary interactions’ wasn’t clear to me. Do\nthe authors mean ValBin-24? I also don’t think that a table of ValBin-25 with the source(s) of each interaction is included.\n- On page 5, the authors note that the Y2H systems stands out for having contributed large numbers of reliable and primarily\ndirect PPIs. If the authors want to make a point that Y2H is a better system than others, this statement needs more\nsubstantiation. Specifically, I think there may be a circular argument here for 2 reasons: 1) most available binary interactions\nwere identified by Y2H, and 2) in the retesting, most interactions were retested by Y2H thus having a higher chance to retest\npositively than interactions identified by other methods. While most high quality direct interactions may stem from the\nY2H system, this may therefore not be of particular note and not reflect an intrinsic higher quality of the Y2H system.\n- The guilt-by-association experiments are difficult to interpret in their current form. The example in note 2 is clear, but the\nexperiment is not. I understand that the authors first assign a potential function based on potential functions of its neighbour\n(GO term based). Then, during the time between the analysis and paper writing, some proteins went from unknown to\nstudied, allowing a determination of whether that assessment was correct. So I was expecting table 18 to have a list of\nproteins, a predicted function, and an assessment whether new information shows if that prediction is right. But that is now\nwhat table 18 shows. How should the reader interpret that table? Their example YGR168C is listed 86 times, so apparently\n86 possible functions were assigned? And how do I see from the table that that assignment was correct?\n- Page 6 line 19 ‘testing all 1505 pairs’. This sounds like a new experiment, but this is just data taken from YeRI correct?\n(Since that experiment tested all pairwise combinations).\n- Page 7 line 11: how do the strictly novel predicted PPIs fare in the Y2H restest?\n- Page 7 line 39: why was Y2H-v1 used to retest the predicted human interactions?\n- Page 9 line 38: shouldn’t ValBin-25 be the new gold-standard?\n- Page 10 line 14/15. If AlphaFold can predict models for interactions not detected in the interaction screen, does that mean\nthat the computational prediction pipeline can be improved and identify more of the missing interacitions?\n- Page 10 line 20: typo in ‘could be may have been’\n- Supplementary info: in generating the new negative reference set the authors filter out any pair that appeared in any\nexperimental dataset. While a logical approach, I wonder if at some point you start to bias the negative reference set for\nprotein pairs that are particularly unlikely to be detected as false positives, for example because the have overall opposing\ncharges or structures that are not likely to be ‘sticky’. In theory you want to test true random protein pairs but this approach is\nno longer truly random. Perhaps only validated direct protein interactions should be subtracted.", "reviewer_3": "On the manuscript entitled “Experimental assessment of AI-based interactome mapping”, Vidal and co-authors present a\ncomprehensive effort to evaluate the performance of AlphaFold in global deciphering of quaternary structures (protein-\nprotein interactions, PPIs) in yeast and human, and at the same time provide the community with a new experimental\nmethod for detection of such interactions that provide a large number of novel high-quality PPIs. The work is thorough and\nsound. The presented study is endowed with multiple virtues and contributions, among which the critical assessment of the\nusability of PPI data is not the least. Overall, the raised claims, the presented results and the contextual discussion\ncorrespond well with each other. The manuscript is well-written (although sometimes a bit difficult to read, due to the lack of\nspace and the need to consult with the profuse additional materials). In my opinion, the manuscript would be of the highest\ninterest in PPI research and may also appeal to the wider community, and has my enthusiastic support for its publication in\nNature Communications.\nThere are a few aspects that, in my opinion, could make the manuscript stronger, however. I will list them below for the\nAuthor’s consideration:\n1. One of the main contributions of the manuscript (highlighted at the very end of the discussion) is the new YeRI dataset,\nproviding over 1400 novel protein interactions in yeast. This is achieved by implementing a variation of the Yeast-two-Hybrid\n(Y2H) complementation assay, aptly named Y2H-v4 in the manuscript. For the benefit of the wider audience, I think the\nmanuscript would benefit from a more thorough explanation of the technique and an explanation of its similarities and\ndifferences with the previous versions of Y2H. Ideally, Fig.1 (and failing to that, the corresponding supplementary\ninformation) could allocate these explanations.\n2. The Authors mention a large number of existing datasets, especially in the first part of the manuscript (yeast). A graphic\nnavigating the reader through those datasets would be extremely useful, either in the main manuscript or in its\naccompanying supplementary information. Such a guide could include the nature of each of the sets, their size, and the\nreason for being used in the current study.\n3. Following the previous point, the Authors define Y2H-union-25 as the merging of their YeRI with three previously existing\ndatasets, comprising a total of 4,307 PPIs. I had issues identifying which previously existing datasets the Authors refer to. I\nguess these are refs. 11, 18, and 19, but clarification on the manuscript would be nice. On the same tone, on pg. 6, the\nAuthors state that adding YeRI information to “previously identified as high-quality binary interactions results in a ValBin-25\ndataset of 10,759 PPIs.” It would be valuable if the Authors could identify (refer to) those previously described high-quality\nbinary interactions.\n4. A key question that is only partially addressed during the text is the true complementarity or orthogonality nature of\ndifferent methods. I will illustrate this with the following case. YeRI detects around 1900 PPIs (of which more than 1400\nnovel) while the estimates for the total yeast interactome range between 20 and 40 thousand (pg. 3). The Authors, later by\nintegrating YeRI to Y2H-union24 and further to ValBin25, increase the coverage to around 10 thousand. The open questions\nhere are twofold: what are the remaining (10-30K) yeast PPIs? How can a naive user remove older datasets containing\nincorrectly recorded PPIs? The Authors address the latter question in the second part of the manuscript (human), stating that\nvarious thresholds on different estimation techniques can be applied (Fig. 4a), but a clearer guideline on what can be\nconsidered a wrong PPI would be useful to the readers.\n5. On page number 8, the Authors state that the human AF/RF-core dataset consists of 87% of the predicted PPIs. This\nnumber is (strikingly?) different to that obtained for yeast (64%, pg. 6). Could the Authors comment if these differences are\nsignificant and, if so, what are their plausible origins?\n6. On the side of minor notes:\n6.1. At the beginning of the manuscript, the Authors refer systematically to the positive reference set as scPRS-v2. As the\nmanuscript develops, this is sometimes substituted by PRS. Unity on naming would help the reader to understand that the\nAuthors are referring to the same dataset.\n6.2. Despite the amazing effort in keeping the text short and clear, I think the Authors have some editing errors on page 10,\nline 20. The text reads “[…] some of the interactions for which structures could be may have been missed […]”. I think the\ncurrent grammatical formulation is incorrect. Instead, “could have been missed” or “may have been missed” sounds better."}, "author_response": "We thank the reviewers for their thorough and constructive evaluation, which included\nboth suggestions for additional analyses and conceptual questions as well as requests\nfor clarification. In response, we have performed such analyses and addressed all their\nquestions and requests.\nIn addition, we took the initiative to address an issue that arose since we received our\nreviews. Indeed, the bioRxiv preprint (bioRxiv 2024.10.01.615885) describing AI-\npredicted human protein-protein interactions (PPIs), originally experimentally assessed\nin our manuscript, has now been published (Zhang et al. Science 2025). And we noticed\nthat the final dataset published in Science is different from the one described in the\nbioRxiv version. By rerunning our analyses on the Science version of the predicted\ndataset, we verified that this update did not affect our conclusions.\nThe manuscript has been revised to incorporate these new analyses and corresponding\ndiscussion. Importantly, all additional analyses support our original conclusions.\nWe hope that the editor and reviewers will agree that these revisions have clarified and\nsubstantially strengthened the manuscript. Below, we address each comment in turn and\ndescribe how the revised manuscript has been modified since the previous submission.\nAUTHOR NOTE TO THE REVIEWERS AND THE EDITOR\nThe set of human predicted PPIs we tested in our original submission was taken from a\nbioRxiv preprint (bioRxiv 2024.10.01.615885). After we received the reviewers’\ncomments, a revised version of this preprint was published in Science on September 25,\n2025, accompanied by an updated version of the dataset. To determine whether the\nupdates introduced in the published dataset affected the conclusions drawn from our\nexperiments and analyses based on the preprint data (see Figure below), we\nsystematically examined the differences between the preprint and published versions.\nImportantly, all algorithms and input data (i.e., the version of the PDB, the multiple\nsequence alignments, and the versions of the PPI and genetic interaction databases)\nremained unchanged between the preprint and published versions. The differences arise\nfrom two main changes to the methodology. First, the published version introduces an\nadditional filter for homologous protein pairs, which the authors determined reduced false\npositive predictions. This change accounts for predicted PPIs present in the preprint\ndataset but absent from the published dataset. Second, input pairs with prior support from\nPPI databases were subdivided according to their source database and the amount of\nsupporting evidence, with separate thresholds derived for each subset. This change\naccounts for predicted PPIs included in the published version that were not present in the\npreprint.\nIn our updated analysis, we were able to derive thresholds for the de novo screening and\ngenetic interaction strategies using our experimental data by filtering out these\nhomologous pairs. However, we were not able to accurately derive thresholds for the new\nPPI DBs strategies (i.e., the subset of predictions where pre-existing PPI evidence is\nused) based on the random sample of the preprint dataset that we tested. However, given\nthat these pairs have been observed previously, we think their evaluation is significantly\nless critical to our findings than the experimental evaluation of pairs that emerged from\nthe de novo screen.\nWe have updated our manuscript by restricting the analyses to the 17,849 pairs in the\ndataset published in Science. After filtering out the predicted PPIs from the original\nbioRxiv version that do not now pass the adjusted thresholds of the published version,\nthe number of pairs we tested in Y2H decreased from 4,046 to 3,222. Importantly, the\nvalues of the thresholds that we derived using our approach were highly similar between\n2\nthe preprint and published versions of the dataset, and our conclusions remain\nunchanged. This can be seen in the before and after versions of Figure 4 below.\nPreviously submitted version of Figure 4, based on the bioRxiv dataset.\nReport on Lambourne et al. Experimental assessment of AI-based interactome\nmapping\nLambourne and team provide a straight forward (in the sense of well written and\nillustrated) comparison of two AF complex prediction data set, one from yeast and one\nfrom human (AF/RF, still preprint), with experimental Y2H PPI data. Intriguingly, the\n(novel and interesting) outcomes from the comparisons for yeast and human are very,\nvery similar for both species: The predictions are of high quality, they can be\nexperimentally corroborated to a similar rate than known positive reference PPIs. There\nare no hubs, the AF predicted networks appear disintegrated, likely because PPIs with\nIDPs are missing. Interfaces are larger, domain-domain PPIs are predominant, there is a\nhigher knowledge bias, and most importantly the AF predictions contribute very few novel\nPPIs only. Experimental PPI discovery is warranted. Along this conclusion, a new Y2H\nyeast PPI data set is presented, high quality. It is derived from the all vs all Y2H mapping\nof 5854 ORFs and adds 1880 PPIs. The combined ValBin-25 PPI yeast dataset has\nresource character. The study ends by using AF to predict the 3D structure of reference\npairs and novel PPIs to demonstrate how AF/RF can contribute to interactome research.\nIn summary, this work is a significant contribution!\nAUTHOR RESPONSE\nWe thank the reviewer for this positive summary of our work.\nPoints for consideration.\na) The authors use a new Y2H-v4 vector setup which it is claimed to be an orthogonal\nsystem with respect to previous screens. However, the screen delivers very sparse data.\nConsidering the high sensitivity (22%, 1d), I do not see how the data go along with\ninteractome size predictions. In Figure 1J the val-bin24/25 is between 25-50% of the\ninteractome size and counts about 10000. The screen covers roughly all pairs (“a\nfoundation for the systematic completion of the S. cerevisiae interactome map”) and\ndetects 1880/0.22= 8545 PPIs. Please, consider that the sensitivity using the PRS is a\nsubstantial overestimate.\nAUTHOR RESPONSE\nWe agree with the reviewer that the assay sensitivity estimated using the PRS does not\nentirely reflect the overall sensitivity of the YeRI dataset. As demonstrated by our group\n(Yu et al. Science 2008 and Venkatesan et al. Nature Methods 2009), another important\nknown factor affecting the ability of YeRI to approach the theoretical assay sensitivity, as\nmeasured by the PRS, is the sampling sensitivity, which quantifies the saturation of\n5\nscreens relative to the pairwise tests used in the benchmarking. The sampling sensitivity\nfrom Yu et al. Science 2008 and Venkatesan et al. Nature Methods 2009 was around\n80% after three screens. However, those experiments were performed using pool sizes\nof 188 ADs. In contrast, YeRI was mostly generated with pool size of 700 ADs, and we\nknow, based on unpublished internal data, that an increasing pool size negatively affects\nthe sampling sensitivity, i.e., decreases the saturation of a single screen. Therefore, we\nexpect a correspondingly lower sampling sensitivity, which would explain a large part of\nthe discrepancy between your value of 8,545 PPIs and the interactome size estimates.\nIf we conservatively estimate the sampling sensitivity to be 50%, instead of the 80%\npreviously determined for the smaller pools, applying the empirical framework to Y2H-v4\nwould yield an estimate of ~17,000 interactions. This is in line with two of the four previous\nestimates we use in the manuscript, one of which used the same methodology on a\ndifferent dataset (Yu et al. Science 2008), the other used a completely different\nmethodology (Grigoriev NAR 2003).\nAt the same time, the assay sensitivity estimated from ~100 pairs is associated with an\nerror margin. In addition, we agree that it is possible that the PRS is biased, as we rely\non the existing literature for assembling this high-quality benchmark set. As a reminder,\nto serve as a widely acceptable positive control set, the PRS is composed of interaction\npairs that are well-established and validated by multiple studies, both biophysically and\nfunctionally. Thus, biases will be introduced by the focus of the existing literature and by\nthe need to only include interactions that have been reproduced by the community. It is\nimportant to point out that no additional filters are applied, meaning we do not remove\ninteraction pairs that are dependent on post-translational modifications or involve\ntransmembrane proteins. We consider the difficulties that any assay (including the Y2H)\nmay have with such ‘special’ pairs as part of the assay’s inherent limitations that we aim\nto capture with the assay sensitivity as determined by the PRS. We’ve added this\nclarifying statement to the main text at the end of the second section describing the\ngeneration of YeRI:\nEXCERPT FROM REVISED MANUSCRIPT\n“Notably, the overall sensitivity of YeRI is lower than the 22% scPRS-v2 recovery (Fig.\n1d) due to the ‘sampling sensitivity’30 of the screen.”\nb) As mentioned above, the study ends by using AF to predict the 3D structure of\nreference pairs, 37% success and some novel PPIs with 14% success. The authors\nprovide hypothesis for the difference, which is not reflected in the experimental validation.\nWhile I go well along with the offered explanations, the authors could add at least two\nmore limitations to the analysis. This is the PRS is more like the domain-domain AF data,\nor at least theoretically the new yeast data are not as reliable. What is the average surface\narea of RPS pairs etc …\nAUTHOR RESPONSE\nWe thank the reviewer for making this important point. The average interface area of the\nsubset of scPRS-v2 pairs for which there are experimental structures available is indeed\nhigher than the full set of experimentally resolved PPIs (see plot below). There are 31\npairs of the scPRS-v2 in the 3DID domain-domain dataset (1,250 total DDIs) and only 1\nscPRS-v2 pair in the ELM domain-motif dataset (95 total DMIs).\nInterface area of PPIs in the yeast positive reference set, relative to other structurally resolved PPIs.\nWe do not believe the available evidence supports the hypothesis that the difference in\nconfident structure prediction rates between YeRI and scPRS-v2 is primarily due to the\nreliability of the YeRI data. Explaining the observed discrepancy would require a false\npositive rate of ~60%, which is inconsistent with the validation results from the MAPPIT\nand GPCA assays. We have added the following to the main text:\nEXCERPT FROM REVISED MANUSCRIPT\n“The lower proportion of successful AlphaFold2 predictions in the novel YeRI pairs\nrelative to scPRS-v2 could be due to a greater proportion of transient interactions,\ninteractions involving disordered regions, and interactions between less evolutionarily\nconserved proteins detected in YeRI compared to the reference set42, in addition we\nnoticed a bias in the scPRS-v2, which contains PPIs with significantly larger interface\nsizes on average (P = 1.7 x 10-6 , two-sided Mann-Whitney U test).”\n7\nc) After detailed analysis of yeast data (figs 1-3 ) the authors perform the analyses of\nhuman data, with analogous outcome. While these results are very well supported and\nthus convincing, the human time line is not. In analogy, the human time line (Figure 4d)\nshould include (similar to yeast Figure 3i), also the non-Vidal-Lab large scale data sets\nthat contributed to the binary interactome mapping. The Vidal lab contributed the most\nsignificant data sets, but not exclusively. Including the most significant data sets also from\nother labs will strengthen the point of the Figure.\nAUTHOR RESPONSE\nWe thank the reviewer for pointing this out and offer our apologies for this oversight. We\nhave edited the timeline of reported human PPIs to include two other large-scale\nsystematic binary datasets that we had previously missed (as shown below) and added\nthe corresponding references to the text. Those datasets, MDC-HI1 (Stelzl et al. Cell\n2005, with the full-length subset validated in Venkatesan et al. Nature Methods 2009) and\nNN1.0 (Haenig et al. Cell Reports 2020), were the only two we found, after searching the\nliterature, that met our criteria for being validated using random samples in orthogonal\nassays and benchmarked using positive and random reference sets. If the reviewer is\naware of any other datasets that meet these criteria, we will gladly add them to the\ntimeline.\n“36. Stelzl, U. et al. A Human Protein-Protein Interaction Network: A Resource\nfor Annotating the Proteome. Cell 122, 957–968 (2005).\n38. Haenig, C. et al. Interactome mapping provides a network of\nneurodegenerative disease proteins and uncovers widespread protein aggregation in\naffected brains. Cell Rep. 32, 108050 (2020).”\n8\nd) It should be note that while I strongly agree that the novel PPIs in the human AF paper\n(which I have read carefully) are very few in number, they mostly relate to membrane-\nmembrane PPIs, where Y2H is negatively biased. Therefore, the novel AF PPIs could be\nlargely complementary to the experimental binary PPIs. This needs to be checked e.g. in\nterms of which proteins are involved etc. How many of the strictly novel PPIs of the human\nAF/RF core have been retested with Y2H-v4 in the presented experiments? I suggest a\nsimilar panel like the ones in Figure 4a that show the success of de novo screens\nwould be revealing if shown for strictly novel PPIs (provided that they were tested).\nAUTHOR RESPONSE\nThis is an interesting point. According to UniProt annotations, 14% (2,575/17,849) of the\nhuman AF/RF dataset is between two transmembrane proteins, and 12% (2,119/17,849)\nis between a transmembrane protein and a non-transmembrane protein. The\ncorresponding fractions for the strictly novel pairs increase to 21% (1,028/4,789) and 18%\n(847/4,789), respectively. To test the impact of this on our results, we therefore generated\nFig. 3a after removing PPIs involving transmembrane proteins. As can be seen, the trend,\nand hence our conclusions, remain the same:\nThus, the increased number of interactions involving membrane proteins is not\nresponsible for the observed drop in Y2H recovery rate at lower contact probability\nscores.\nRegarding the reviewer’s questions about the number of tested PPIs: the number of\nstrictly novel human predicted PPIs successfully tested in Y2H was 604, of which 102\nwere between two transmembrane proteins, 139 were interactions between\ntransmembrane and non-transmembrane proteins, and 363 didn’t involve a\ntransmembrane protein. A version of the de novo screen results of Fig. 4a, restricted to\n9\nstrictly novel predictions, with and without excluding transmembrane proteins is shown\nbelow:\nIt is apparent that the novel pairs test positive at very low rates and that this is not\nexplained by the enrichment in PPIs involving transmembrane proteins. One possible\nexplanation would be that the novel pairs are indeed of low quality. However, it also\ncannot be excluded that these pairs are enriched in other properties that are indeed\ncomplementary to the detection profile of Y2H, akin to interactions involving\ntransmembrane domains. This question needs to be addressed in subsequent\nexperimental and computational analyses that are beyond the scope of this manuscript.\ne) The predictions of novel Y2H were done with success even though the pairs were not\nin the AF predicted data set (maybe different scoring, ccc). However, where these novel\npairs considered in the AF predictions in the first place? Did they end up below cut off (cf\n. Figure 4c). The human predictions were guided by known pairs though ...\nAUTHOR RESPONSE\nOf the yeast AF/RF pairs below the ‘-core’ cutoff, there are 17 in YeRI, of which 12 pass\nthe confident contacts count (CCC) threshold, of which 5 already have an existing\nexperimental structure and 1 has a homologous structure, leaving 6 pairs that were\nexcluded from the 237 pairs based on their appearance in the AF/RF dataset below the\n0.95 contact probability threshold. So, only a small contribution.\nUnder our definition of the 237 confidently modeled YeRI pairs that did not have previous\nstructural models, we excluded all pairs in the full AF/RF dataset before the cutoff. We\nhave clarified this by adding a section to the methods:\n“PPIs with no structural model\nYeast PPIs with no structural model were defined as not being present in I3D-exp-24, not\nhaving an interolog structure (defined above), and not being in the full AF/RF dataset (i.e.,\nthe dataset before applying the higher, experimentally derived contact probability\nthreshold).”\nTo answer the question of why the novel confidently modeled novel YeRI pairs were not\nin the AF/RF dataset in more detail, we also looked at the search space that was screened\nin the AF/RF dataset and added our findings to the discussion:\nEXCERPT FROM REVISED MANUSCRIPT\n“For YeRI, AlphaFold could infer new structural models for 237 PPIs, of which 72% (171)\nwere not in the AF/RF search space because the pair had few homologues (shallow MSA\ndepth), whereas 28% (66) were tested and were below the thresholds. These could either\nhave been missed by the screening algorithm whilst being detected by the experimental\nscreening or could have been recovered due to the improved CCC interaction score for\nAlphaFold structures, which we developed based on the scPRS-v2/scRRS-v2\nbenchmarking data, and which outperformed the previously described metrics.”\nReviewer #2 (Remarks to the Author)\nThe manuscript by Lambourne et al consists of three parts. In the first, the authors assess\nthe current state of interactome mapping for budding yeast. Using three binary interaction\nassays, they test subsets of pairs from various interaction datasets to evaluate their\nquality. Most of the interactions are retested using an updated version of the Vidal lab\nyeast two-hybrid (Y2H) pipeline that is able to positively identify more interactions (higher\nsensitivity) while maintaining a low false-positive rate. The main conclusion is that most\ndatasets are of low quality in terms of accurately reporting direct protein-protein\ninteractions, possibly in part because many datasets report indirect associations (i.e.\nproteins in a larger complex) in the form of interacting pairs. The authors combine the\nhigh-quality datasets into a single dataset they term ValBin-24 for validated binary\ninteractions in 2024. This section provides a thorough and thoughtful overview of the\ncurrent state of interactome mapping.\nIn the second part of the manuscript, the authors experimentally expand the yeast binary\ninteraction map using i) their updated Y2H pipeline, and ii) a near complete yeast\ncollection of 4933 ORFs). This adds 1723 interactions to previous Y2H high confidence\ninteractions, for a total of 4556 interactions. Both the expanded ORFeome collection and\nnew interactions are of high value to the scientific community.\nIn the third part, the authors assess the accuracy of yeast and human protein interactions\npredicted by new structure prediction algorithms. They do this by testing all predicted\nyeast interactions and a random sampling of human protein interactions in their Y2H\npipeline. I found this an interesting analysis to read. The main conclusion is that while\npredictions are of high quality, very few new interactions are predicted. The likely reasons\nfor this are that structural prediction algorithms are better at prediction structures for\nproteins similar to proteins with know structures, and are not good at identifying\ninteractions with disordered regions, which mediate many protein interactions. While\nthese observations are not necessarily unexpected, it is very useful to have a thorough\nexamination of the predictive power of new AI methods.\nOverall, the work provides an advancement in our knowledge of protein-interactions and\nin tools (Y2H system and new ORFeome dataset). The evaluation of AI methods should\nbe of broad interest and is an important step in assessing the use cases for these new\ntools.\nI have little if any qualms with the experimental approaches used. The Vidal lab are\nabsolute experts in the field of binary protein interaction mapping, having honed their Y2H\npipeline over 2 decades of work and having increasingly emphasized the value of rigorous\nbenchmarking of interaction assays with positive and negative reference sets. The\ncomparisons with Alphafold predictions also seem to have been thoroughly performed,\nwith attention to the computational methods and thresholds used. My comments for\nimprovement are below, most address readability.\nAUTHOR RESPONSE\nWe thank the reviewer for this positive summary of our work.\n- A table with all the various datasets mentioned would be very helpful, even being familiar\nwith past work from the Vidal lab the many dataset abbreviations confused me. Most are\nexplained in the methods, but a overview table to refer to while reading would be a\nwelcome addition.\nAUTHOR RESPONSE\nWe thank the reviewer for this useful suggestion. We have added both a new Extended\nData Figure 3 and Tables 1 and 2, copied below:\nTable 1: Descriptions of the biophysical datasets used in this study. Dataset names ending in numbers refer\nto the year of generation.\n- The multiple Lit-BM sets are not well explained. On page 4 line 28, the authors state\nthat interactions from Lit-BM-17 were tested, but on line 32 it seems a Lit-BM-13 was also\ntested. What is the difference?\nAUTHOR RESPONSE\nThe Lit-BM dataset is continuously updated every few years, and we performed each\nexperiment with the most recent version at that time. The MAPPIT and GPCA\nexperiments were performed with Lit-BM-13, whereas the Y2H was performed with Lit-\nBM-17. The datasets are largely overlapping, as you would expect. We have clarified this\nin the text as follows:\nEXCERPT FROM REVISED MANUSCRIPT\n“Specifically, we tested 8,999 pairs in Y2H-v4, which included all protein pairs with\nexperimental 3D structures curated in 2017 (I3D-exp-17), literature pairs with multiple\nevidence of which at least one can detect binary PPIs (Lit-BM-17), and three\nsystematically generated binary maps (Y2H-union-08) (Fig. 1f and Extended Data Table\n10). Since most of the systematic binary approaches and many of the studies that\ncomprise Lit-BM used the Y2H assay, we also tested Lit-BM-13, Y2H-union-08, and\nDHFR-PCA in GPCA and MAPPIT (Fig. 1g, h, and Extended Data Table 11). Of all\ndatasets tested, we found that I3D-exp-17, Lit-BM-13/-17, and Y2H-union-08 tested\ncomparably to the positive control set, whereas the other datasets, which make up the\nmajority of the published pairs, tested at rates closer to the random pairs.”\n- On page 5 line 26, why are the new interactions compared to Lit-BM-13? In figure 1, Lit-\nBM-17 was used.\nAUTHOR RESPONSE\nAs stated for the previous point, the MAPPIT and GPCA experiments were performed\nusing pairs originally identified in Lit-BM-13.\n14\n- Page 5 line 36: the sentence ‘thus we generated a new high quality map’ would be better\nplaced on line 23, before going on to evaluating the new map. The way it reads now I was\nwondering if the new map was generated using criteria based on the comparison with the\nPSNs, e.g. only incorporating those that were enriched in one or more PSNs. (or if that is\nactually the case, clarify the criteria).\nAUTHOR RESPONSE\nWe have moved the sentence.\n- The naming of the Yeast Reference Interactome seems and odd choice. What makes\nthis a reference interactome, why would CCSB-YI1 not be a reference interactome, or the\nnext interactome the Vidal lab generates. Wouldn’t a year name be more consistent with\ntheir nomenclature, e.g. CCSB-YI25? Or YI2? The analyses below are also not done on\nYeRI but on union-25 and on ValBin-25, so it isn’t used as a reference set here either.\nAUTHOR RESPONSE\nWe use the term reference interactome because of its all-by-all testing of the search\nspace, i.e., close to a complete ORFeome collection. This is consistent with the naming\nof our human reference interactome, HuRI (Luck et al. Nature 2020). Previous systematic\nmaps in yeast left substantial portions of the search space unscreened (see Fig. 2a\nbelow). By increasing the search space coverage to the point where it is effectively\ncomplete, YeRI takes this important variable out of consideration. The resulting gateway-\ncompatible clone collection can then be employed by others, using different PPI assays,\nto build towards a complete map of the interactome (see Choi et al. Nature\nCommunications 2019), with YeRI acting as a cornerstone of those future efforts.\nImportantly, the use of the term reference is analogous to that of the human reference\ngenome, which, in its original form, was substantially incomplete but nevertheless did\nserve its valuable purpose as a reference for the community.\n- The combining of YeRI with ‘the datasets previously identified as high-quality binary\ninteractions’ wasn’t clear to me. Do the authors mean ValBin-24? I also don’t think that a\ntable of ValBin-25 with the source(s) of each interaction is included.\nYes, we were referring to ValBin-24. We thank the reviewer for highlighting that this was\nunclear. We have also added AF/RF-core to ValBin-25, since it is also a validated dataset\nof binary PPIs. We had originally included a table of ValBin-25, but by mistake, omitted\nthe name in the description. We have added that in.\nEXCERPTS FROM REVISED MANUSCRIPT\n“Adding YeRI to the datasets we previously identified as high-quality binary interactions\n(ValBin-24) results in a total of 10,759 PPIs.”\n“Extended Data Table 22. ValBin-25: the union of high-quality binary experimental and\ncomputational interactome maps. The union of I3D-exp-24, Lit-BM-23, AF/RF-core, and\nY2H-union-25. Note: homodimers are included for completeness but were generally not\nused in the analysis in the manuscript.”\n- On page 5, the authors note that the Y2H systems stands out for having contributed\nlarge numbers of reliable and primarily direct PPIs. If the authors want to make a point\nthat Y2H is a better system than others, this statement needs more substantiation.\nSpecifically, I think there may be a circular argument here for 2 reasons: 1) most available\nbinary interactions were identified by Y2H, and 2) in the retesting, most interactions were\nretested by Y2H thus having a higher chance to retest positively than interactions\nidentified by other methods. While most high quality direct interactions may stem from the\nY2H system, this may therefore not be of particular note and not reflect an intrinsic higher\nquality of the Y2H system.\nAUTHOR RESPONSE\nThis is a fair comment. We note that we also performed MAPPIT and GPCA assays to\nevaluate the quality of existing datasets. However, that statement, as it was written, did\nnot accurately express what we wanted to say. We were not trying to focus so much on\nrelative quality compared to other methods, but rather on its relative practicality, having\nbeen used to generate large high-quality datasets. So we have rephrased as follows:\nEXCERPT FROM REVISED MANUSCRIPT\n“Notably, of all the experimental methods, systematic high-throughput Y2H-based\nmapping has proven to be a particularly practical and scalable approach, having produced\nmultiple large interactome maps of reliable and primarily direct PPIs.”\n- The guilt-by-association experiments are difficult to interpret in their current form. The\nexample in note 2 is clear, but the experiment is not. I understand that the authors first\nassign a potential function based on potential functions of its neighbour (GO term based).\nunknown to studied, allowing a determination of whether that assessment was correct.\nSo I was expecting table 18 to have a list of proteins, a predicted function, and an\nassessment whether new information shows if that prediction is right. But that is now what\ntable 18 shows. How should the reader interpret that table? Their example YGR168C is\nlisted 86 times, so apparently 86 possible functions were assigned? And how do I see\nfrom the table that that assignment was correct?\nAUTHOR RESPONSE\nThank you for pointing out this lack of clarity. We have updated the column headings and\nthe table description.\nIn the table, the 86 entries for YGR168C are each different GO terms that exceeded the\nZ-score > 2 threshold, for both networks the predictions were performed on (YeRI and\nY2H-union-25). The main reason there are so many is because of the large redundancy\nin GO terms. For example, the top three terms (ranked by p-value) for that gene are:\nGO:0005778 peroxisomal membrane, GO:0007031 peroxisome organization, and\nGO:0016559 peroxisome fission. Another reason for the large number of entries is\nbecause we used a very relaxed threshold of Z-score > 2 when generating the table, with\nthe idea that the reader would rank and filter themselves. On reflection, this was a poor\nchoice, and so we have increased the stringency of the predictions in the table – to Z-\nscore > 5 and at least 2 partners with the predicted GO term – which has reduced the\nnumber of rows in the updated table.\nRegarding the genes whose names had changed, previously we had simply inspected\nthe top handful of predictions and found that the top prediction, YGR168C, had been\nrenamed PEX35. We have now added comments on the other 3 predictions which have\nbeen renamed to the text.\nEXCERPT FROM REVISED MANUSCRIPT\n“There are four such examples, which can be used to test the accuracy of our predictions.\nFirstly, owing to its recently demonstrated role as a regulator of peroxisomal abundance,\nYGR168C is now known as PEX3568. Ygr168c/Pex35 has 23 PPIs in Y2H-union-25, all\nfrom YeRI, out of which eight are proteins involved in peroxisomal biology, and so was\npredicted to function in peroxisomal protein import, showcasing the ability of Y2H-union-\n25 to predict gene function accurately (Extended Data Fig. 3i). Moreover, Pex35 interacts\nwith another protein of unknown function, Ykl018c-a/Mco12, which can be now\nhypothesized to be involved in peroxisomal abundance as well. Secondly, the highest-\nranked prediction for Yjr011c was to be involved in the CCR4-NOT complex, and it is now\nknown as Cal4, an accessory component of the CCR4-NOT complex69. Thirdly, the\n17\nhighest-ranked prediction for Ykl075c is for mitochondrial respiratory chain complex II\nassembly. This gene is now named AAN1, based on “impacts on actin cable stability,\nmitochondrial function, BCAA metabolism, and cellular lifespan”, with its deletion being\nfound to significantly reduce mitochondrial oxidation levels70. Ykl075c also localizes to\nthe mitochondria upon rapamycin treatment71. Finally, Ypr174c is predicted to be\nlocalized to a membrane. It is now named Csa1 and found to anchor Cdc5 at spindle pole\nbodies 72. It binds phosphatidylinositols and phosphatidylethanolamines73 and is localized\nto the nuclear periphery45. In summary, where there is evidence to evaluate our\npredictions, it broadly supports their accuracy, especially considering the heuristic nature\nof their generation, and thus supports the useful information contained within our\nsystematically generated PPI network.”\n“69. Pillet, B. et al. Dedicated chaperones coordinate co-translational regulation\nof ribosomal protein production with ribosome assembly to preserve proteostasis.\nElife 11, (2022).\n70. Sing, C. N. et al. Identification of a modulator of the actin cytoskeleton,\nmitochondria, nutrient metabolism and lifespan in yeast. Nat. Commun. 13, 2706\n(2022).\n71. Koh, J. L. Y. et al. CYCLoPs: a comprehensive database constructed from\nautomated analysis of protein abundance and subcellular localization patterns in\nSaccharomyces cerevisiae. G3: Genes|Genomes|Genetics 5, 1223–1232 (2015).\n72. Örd, M. et al. Proline-rich motifs control G2-CDK target phosphorylation and\npriming an anchoring protein for Polo kinase localization. Cell Rep. 31, 107757\n(2020).\n73. Gallego, O. et al. A systematic screen for protein-lipid interactions in\nSaccharomyces cerevisiae. Mol. Syst. Biol. 6, 430 (2010).”\n- Page 6 line 19 ‘testing all 1505 pairs’. This sounds like a new experiment, but this is just\ndata taken from YeRI correct? (Since that experiment tested all pairwise combinations).\nAUTHOR RESPONSE\nNo, this is a new experiment. YeRI was not generated by testing each pair individually,\nsince that number is not experimentally feasible. Rather, we first tested individual DB-\nORF fusions with pools of hundreds of AD-ORF fusions, referred to as the screens, and\nthen subsequently pairwise tested the hits from the screens. The experiments with all\n1,505 AF/RF yeast pairs were pairwise tests, which we performed since they have\nsignificantly higher sensitivity than the pooled screens. We have clarified this in the text\nas follows:\nEXCERPT FROM REVISED MANUSCRIPT\n18\n“To assess the accuracy of these predictions, we experimentally evaluated the yeast\nAF/RF dataset, performing Y2H-v4 pairwise tests of all 1,505 pairs.”\n- Page 7 line 11: how do the strictly novel predicted PPIs fare in the Y2H restest?\nAUTHOR RESPONSE\nThere are 42 strictly novel predicted pairs, of which 30 were\nsuccessfully tested (after excluding autoactivators and\nexperimental failures). None tested positive when\nbenchmarked against a PRS/RRS set. As in response to a\nsimilar question by Reviewer 1, it is possible that the newly\npredicted pairs represent high-scoring false positives of the\nAF/RF prediction. At the same time, it must be kept in mind\nthat these are a relatively small number of pairs, thus limiting\nthe informativeness of the data. It is possible that the\npredicted pairs have properties that bias them against\ndetection by our validation assay, such as the increased\nproportion of pairs involving one or more transmembrane\nproteins, which are detected at lower rates in Y2H. This question must be resolved in\nfuture studies.\n- Page 7 line 39: why was Y2H-v1 used to retest the predicted human interactions?\nAUTHOR RESPONSE\nFor practical reasons and to perform the experiment as quickly as possible. We already\nhad the entire human ORFeome in the v1 vectors, but not in v4.\n- Page 9 line 38: shouldn’t ValBin-25 be the new gold-standard?\nAUTHOR RESPONSE\nYes. We thank the reviewer for spotting this mistake and have corrected it in the text.\n- Page 10 line 14/15. If AlphaFold can predict models for interactions not detected in the\ninteraction screen, does that mean that the computational prediction pipeline can be\nimproved and identify more of the missing interacitions?\nAUTHOR RESPONSE\nWe believe this is the case, yes. In our view, key points for improvement would be to (a)\nincrease the sensitivity of the fast screening algorithm to approach that of AlphaFold; (b)\nincrease the search space coverage by removing the filters; (c) use an improved scoring\nmetric, i.e., the confident contacts count (CCC). Importantly, however, the potential gains\nof these improvements to the fast screening algorithm would still be constrained to an\nupper limit of the sensitivity of the slower AlphaFold, which we estimate to be 14%, based\non running AlphaFold on the YeRI PPI dataset.\n- Page 10 line 20: typo in ‘could be may have been’\nAUTHOR RESPONSE\nWe have corrected it in the text.\n- Supplementary info: in generating the new negative reference set the authors filter out\nany pair that appeared in any experimental dataset. While a logical approach, I wonder if\nat some point you start to bias the negative reference set for protein pairs that are\nparticularly unlikely to be detected as false positives, for example because the have\nopposing charges or structures that are not likely to be ‘sticky’. In theory you want to test\ntrue random protein pairs but this approach is no longer truly random. Perhaps only\nvalidated direct protein interactions should be subtracted.\nAUTHOR RESPONSE\nThis is a good point, which we will consider in future work. Whilst we checked for\nexperimentally determined interactions when generating scRRS-v2 (random reference\nset), we did not find any that were picked as part of the random selection, and hence,\nno randomly picked pair was actually removed. In principle, though, the reviewer raises a\nvalid point, and it’s something we will consider going forward. In general, the generation\nof negative interaction datasets that are large, clean, and unbiased is fraught with\nchallenges.\nReviewer #3 (Remarks to the Author):\nOn the manuscript entitled “Experimental assessment of AI-based interactome mapping”,\nVidal and co-authors present a comprehensive effort to evaluate the performance of\nAlphaFold in global deciphering of quaternary structures (protein-protein interactions,\nPPIs) in yeast and human, and at the same time provide the community with a new\nexperimental method for detection of such interactions that provide a large number of\nnovel high-quality PPIs. The work is thorough and sound. The presented study is\nendowed with multiple virtues and contributions, among which the critical assessment of\nthe usability of PPI data is not the least. Overall, the raised claims, the presented results\nand the contextual discussion correspond well with each other. The manuscript is well-\nwritten (although sometimes a bit difficult to read, due to the lack of space and the need\n20\nto consult with the profuse additional materials). In my opinion, the manuscript would be\nof the highest interest in PPI research and may also appeal to the wider community, and\nhas my enthusiastic support for its publication in Nature Communications.\nAUTHOR RESPONSE\nWe thank the reviewer for this positive summary.\nThere are a few aspects that, in my opinion, could make the manuscript stronger,\nhowever. I will list them below for the Author’s consideration:\n1. One of the main contributions of the manuscript (highlighted at the very end of the\ndiscussion) is the new YeRI dataset, providing over 1400 novel protein interactions in\nyeast. This is achieved by implementing a variation of the Yeast-two-Hybrid (Y2H)\ncomplementation assay, aptly named Y2H-v4 in the manuscript. For the benefit of the\nwider audience, I think the manuscript would benefit from a more thorough explanation of\nthe technique and an explanation of its similarities and differences with the previous\nversions of Y2H. Ideally, Fig.1 (and failing to that, the corresponding supplementary\ninformation) could allocate these explanations.\nAUTHOR RESPONSE\nWe thank the reviewer for highlighting this contribution, and we’ve made changes to make\nthe differences between Y2H-v4 and Y2H-v1 clearer in the paper.\nWe’ve added a new Extended Data Figure 1e as a schematic overview of the Y2H v1 vs\nv4 differences:\nWe’ve added a new Methods Table 3 to make the assay differences clearer:\n21\n2. The Authors mention a large number of existing datasets, especially in the first part of\nthe manuscript (yeast). A graphic navigating the reader through those datasets would be\nextremely useful, either in the main manuscript or in its accompanying supplementary\ninformation. Such a guide could include the nature of each of the sets, their size, and the\nreason for being used in the current study.\nAUTHOR RESPONSE\nWe thank the reviewer for this suggestion and have added new Extended Data Figure 3\nand Tables 1 and 2.\nTable 1: Descriptions of the biophysical datasets used in this study. Dataset names ending in numbers refer\nto the year of generation.\n3. Following the previous point, the Authors define Y2H-union-25 as the merging of their\nYeRI with three previously existing datasets, comprising a total of 4,307 PPIs. I had issues\nidentifying which previously existing datasets the Authors refer to. I guess these are refs.\n11, 18, and 19, but clarification on the manuscript would be nice. On the same tone, on\npg. 6, the Authors state that adding YeRI information to “previously identified as high-\nquality binary interactions results in a ValBin-25 dataset of 10,759 PPIs.” It would be\nvaluable if the Authors could identify (refer to) those previously described high-quality\nbinary interactions.\nAUTHOR RESPONSE\nWe have edited the sentences to now read as follows:\nEXCERPTS FROM REVISED MANUSCRIPT\n“Therefore, we combined YeRI with the three previous systematic Y2H maps, Uetz-\nscreen18, Ito-core19, and CCSB-YI111, resulting in Y2H-union-25 comprising 4,307 PPIs\n(Extended Data Table 17).”\n“Adding YeRI to the datasets we previously identified as high-quality binary interactions\n(ValBin-24) results in a total of 10,759 PPIs.”\n4. A key question that is only partially addressed during the text is the true\ncomplementarity or orthogonality nature of different methods. I will illustrate this with the\nfollowing case. YeRI detects around 1900 PPIs (of which more than 1400 novel) while\nthe estimates for the total yeast interactome range between 20 and 40 thousand (pg. 3).\nThe Authors, later by integrating YeRI to Y2H-union24 and further to ValBin25, increase\nthe coverage to around 10 thousand. The open questions here are twofold: what are the\nremaining (10-30K) yeast PPIs? How can a naive user remove older datasets containing\nincorrectly recorded PPIs? The Authors address the latter question in the second part of\nthe manuscript (human), stating that various thresholds on different estimation techniques\ncan be applied (Fig. 4a), but a clearer guideline on what can be considered a wrong PPI\nwould be useful to the readers.\nAUTHOR RESPONSE\nIt is not clear what exactly is specific about the missing PPIs, but we can speculate on\nsome possible characteristics of the large number of PPIs that remain to be mapped. It’s\nlikely that undetected PPIs are, on average, weaker than detected PPIs. They may also\nbe more likely to require post-translational modifications to one or other of the two proteins\nor require a specific context in other ways. They may involve proteins that are more\nchallenging experimentally, such as membrane proteins or very large proteins.\nIn general, the question of what the remaining PPIs to be mapped is very interesting but\ntoo broad to be included in the current manuscript. But we’re currently preparing a\nseparate manuscript, looking at the difference between stable co-complex and transient\noutside complex interactions, that at least partially addresses this question.\nFor the second question, the naive reader can use the ValBin-25 dataset as a high-\nconfidence source of interactions in yeast. In generating that dataset, we have removed\ndatasets that didn’t meet our quality cutoff for direct interactions.\n5. On page number 8, the Authors state that the human AF/RF-core dataset consists of\n87% of the predicted PPIs. This number is (strikingly?) different to that obtained for yeast\n(64%, pg. 6). Could the Authors comment if these differences are significant and, if so,\nwhat are their plausible origins?\nAUTHOR RESPONSE\nThis difference is significant, and we assume that this reflects improvements from the\nauthors in their methodology for the generation of the human dataset from the yeast one.\nThe key difference is splitting the single contact probability threshold into multiple different\nthresholds and setting lower thresholds for pairs with previous evidence (in PPI databases\nor genetic interactions) and using more stringent thresholds for hits from their de novo\ncomputational screen. This seems to have increased the dataset’s precision, but without\nincreasing the novelty. They have also changed their screening algorithm, which seems\nto have been an improvement, but it is impossible to say with certainty, as the differences\ncould also be largely as a result of the switch from yeast to human.\n6. On the side of minor notes:\n6.1. At the beginning of the manuscript, the Authors refer systematically to the positive\nreference set as scPRS-v2. As the manuscript develops, this is sometimes substituted by\nPRS. Unity on naming would help the reader to understand that the Authors are referring\nto the same dataset.\nAUTHOR RESPONSE\nWe have fixed this inconsistency in the revised version of the text.\n6.2. Despite the amazing effort in keeping the text short and clear, I think the Authors\nhave some editing errors on page 10, line 20. The text reads “[...] some of the interactions\nfor which structures could be may have been missed [...]”. I think the current grammatical\nformulation is incorrect. Instead, “could have been missed” or “may have been missed”\nsounds better.\nAUTHOR RESPONSE\nThank you for the kind words and for pointing out this mistake. We have fixed it in the\nrevised version.\n26"}, {"round": 2, "reviewer_comments": {"reviewer_1": "Lambourne et al. addressed the points raised and amended text and adapted figures and refs. They also updated their\nhuman interactome analysis in the light of the recently published interactome prediction paper by Zhang and colleagues.\nI want to congratulate the authors to their work and suggest to go ahead.", "reviewer_2": "The authors have extensively revised the manuscript and I appreciate the well written rebuttal, as well as their update of the\nanalysis of the AIpredicted human protein-protein interactions to the published version of that dataset. All of my points have\nbeen thoroughly addressed.", "reviewer_3": "In my opinion, the Authors satisfactorily answered all reviewers' questions. I think the manuscript is ready for publication."}, "author_response": ""}]

Decision letter