ProteoAutoNet: high-throughput co-eluted protein analysis with robotics and machine learning

Domain: biology
Published: 2026-01-22
Editorial outcome: minor_revision

Abstract
Co-fractionation mass spectrometry (CF-MS) enables large-scale profiling of endogenous protein-protein interactions, yet CF-MS data generation is of low throughput and therefore predictive models are often limited by the scarcity and limited diversity of high-quality training data. To address this, we present ProteoAutoNet, a robotic experimental platform integrated with a computational workflow for high-throughput CF-MS analysis. This workflow increases the throughput of sample processing from protein complex to peptide by about two times. The integrated machine learning model incorporates targeted data augmentation to expand and diversify reliable protein interaction data, thereby improving model robustness. When applied to three thyroid cell lines, the model predicted 25,173 co-eluted proteins with an AUROC of 0.78. This analysis revealed significantly upregulated proteasome and prefoldin complexes in the lung metastatic follicular thyroid carcinoma cell line FTC238 compared with the normal thyroid cell line Nthy-ori3-1. Notably, we identified a protein interaction between TGM2 and HK1 that was significantly upregulated in the papillary thyroid carcinoma cell line TPC-1. ProteoAutoNet provides an improved framework for investigating protein-protein interactions and uncovering interactions.

Introduction
Proteins participate in most biological processes, executing functions through a complex network of protein-protein interactions (PPIs). These interactions drive the assembly of protein complexes composed of multiple physically associated proteins and enable functional inference within protein interaction networks1,2. PPIs are critical for advancing our understanding of biological mechanisms and informing drug discovery3. Co-fractionation mass spectrometry (CF-MS) is particularly advantageous for capturing such interactions in their native cellular environments, making it a powerful method for covering PPIs in a near-physiological context4–7. Machine learning has emerged as a powerful method for CF-MS analysis, with software like PrInCE utilizing database-derived annotation to predict protein interactions8. In addition, meta-analysis has been developed to explore CF-MS data using random forest based on multi-species datasets9. While recent studies have reported multiple CF-MS protocols for co-eluted protein analysis10–12, the sample preparation remains time-consuming, and analytical methods suffer from suboptimal performance due to the scarcity and limited diversity of high-quality data annotations9.

Here we built a robotics-assisted platform for CF-MS sample processing. It takes only three days to complete the sample preparation of 540 fractions. We also refined three commonly used complex databases and built an eXtreme Gradient Boosting (XGBoost) based machine learning model for predicting co-eluted proteins with positive sample augmentation. The Complex Portal aggregates protein complexes from 28 species, including 2152 human complexes13. The comprehensive resource of mammalian protein complexes (CORUM) is a manually curated and widely used database, including 3637 human complexes14, while the hu.MAP integrates over 15,000 proteomic experiments data containing 6965 human protein complexes15,16. The three integrated databases yielded 96,635 PPIs. To address the scarcity and limited diversity of known true PPIs and enhance the model’s ability to generalize, we applied data augmentation to positive samples by introducing random perturbations. The positive-sample augmentation expanded the data to ~2.7 million co-eluted proteins from TPC-1, FTC238 and Nthy-ori3-1 cell lines for modeling. We next implemented an XGBoost algorithm for the machine learning model that achieved an AUROC of 0.78 on the internal test set and 0.68 on the independent external validation set derived from FTC133 cell line. The model identified 18,811 high-confidence co-eluted proteins in FTC133.

The ProteoAutoNet workflow integrates robotics-assisted protein complex processing and a machine learning model to identify co-eluted proteins in three thyroid cancer cell lines, and compares differentially expressed PPIs. We performed mass spectrometry analysis using data-independent acquisition (DIA) mode17. Employing ProteoAutoNet with a 30-minute LC gradient on a QE-HF mass spectrometer, we identified 25,173 protein interactions across three thyroid cancer cell lines. Notably, upregulated protein interaction networks (PINs) were associated with the proteasome and prefoldin complexes in FTC238. The subunits of them are potential markers for thyroid cancer treatment and prognosis, such as PSMB6. The predicted PPI comprising TGM2 and HK1 was upregulated in TPC-1. We further investigated its structure using AlphaFold3.

Overall, this study shows the effectiveness of robotics and XGBoost-based machine learning for identifying and characterizing PPIs in thyroid cancer, providing a scalable and comprehensive approach for high-throughput PPIs research.

Results
A robotics-assisted platform for CF-MS sample preprocessing
We established a robotics-assisted sample preparation platform combining Opentrons and JAKA robots18,19, which processes protein complex samples into peptide samples for LC-MS/MS (Fig. 1A). Opentrons can process up to eight 96-well plates, automating pipetting for protein denaturation, reduction, and alkylation during the SDC-lysed peptide preparation, with up to four plates handled per run in 50-60 minutes. For digestion, it can process up to six plates in 40 minutes per run. The full process, from fractionated complexes to peptides, takes 16 hours for four plates, about 18 hours for eight plates and about 24 hours for 12 plates. The JAKA robot supports open-source, customized workflows for desalting, with a single arm capable of processing up to four 96-well plates simultaneously. Desalting four to six plates five times takes about four hours with one arm. Using two arms, processing eight to 12 plates takes approximately eight hours. When integrated with Opentrons and manual interventions, the platform processes four to 12 plates from complexes to peptides in two to three days.

The integrated platform processed samples from three thyroid cell lines, each with three biological replicates. Each sample was co-fractionated into 60 fractions. A total of 540 fractions from three cell lines were processed in six 96-well plates over three days (Fig. 1B). To evaluate the sample processing performance of the platform, we analyzed the coverage and reproducibility of protein identification using nine unfractionated mixtures from three cell lines. These unfractionated samples were processed alongside the 540 fractionated samples and served as quality controls (Fig. 1C). The biological replicates of the three cell lines identified 2897 (Nthy-ori3-1), 2938 (TPC-1) and 2986 (FTC238) shared proteins within each cell line. The Jaccard similarity coefficients for protein identification across replicates within each cell line were all greater than 0.84, exhibiting a high degree of reproducibility of the platform (Supplementary Fig. 1A). The high consistency of protein abundance is shown through a Spearman correlation coefficient of over 0.98. (Fig. 1D).

We further compared the robotics-assisted platform to manual processing using unfractionated TPC-1 samples to validate the performance of the platform further. Compared with manual processing group, the coefficient of variation (CV) of protein counts decreased from 1.08% to 1.01% (Fig. 1E). The number of proteins confirmed the reliability of the robotics-assisted platform (Supplementary Fig. 1B). The mean protein counts were comparable between robotic (3408±34) and manual processing (3404±37). Protein counts in the robotical group showed 25th and 75th percentiles at 3398 and 3428 respectively, compared to 3381 and 3437 for manual processing. We also evaluated reproducibility by calculating the CV of protein abundance. The results showed superior quantitative reproducibility in the robotic processing group, with a lower median CV (20.1% in robotic processing versus 21.6% in manual processing) and a narrower interquartile range (12.6% versus 14.4%) (Fig. 1F).

The robotics-assisted platform of ProteoAutoNet provides a scalable solution for high-throughput CF-MS sample processing, particularly when incorporating offline desalting protocols.

Database curation and machine learning modeling
We refined three protein complex databases, including CORUM, huMAP and Complex Portal by aligning them with the current human proteome, removing absent proteins and unannotated isoforms based on UniProt FASTA (downloaded in May 2024) (Fig. 2A). We developed an extreme gradient boosting (XGBoost) model to predict protein interactions, using the refined databases as the gold standard.

The CORUM includes 4391 human protein entries, with 3428 complexes retained after UniProt alignment14. The hu.MAP focused on high-throughput experimental data contains 9929 reviewed proteins from 6901 complexes post-refinement15. We selected the human-specific protein complexes from the Complex Portal13, leading to 3084 reviewed protein entries from 1830 complexes. Across all databases, 1868 proteins were identified as common complex proteins (Fig. 2B). The hu.MAP contains the most unique proteins at 5910, reflecting the high-throughput affinity purification mass spectrometry (AP-MS) experimental origins of hu.MAP. Complex Portal has the fewest unique entries, with 362 proteins. The hu.MAP database significantly overlaps with CORUM, sharing 3413 proteins, indicating its broad coverage. CORUM and Complex Portal share 2116 proteins, representing 48.19% of CORUM and 68.61% of Complex Portal. Transforming complexes into pairs of PINs showed 7128 interactions common to all databases (Fig. 2B). CORUM and Complex Portal shared 10,533 pairs, representing 18.14% of CORUM and 53.98% of Complex Portal, respectively. hu.MAP has the highest unique pairs (52,488), followed by CORUM (44,403), while Complex Portal only contained 4802 unique pairs (Fig. 2B). This discrepancy likely arises from the loss of complex architecture and protein interdependency when converting protein complexes into protein pairs20. It highlights the diverse sources of protein-protein interaction (PPI) data across databases, underscoring the need to integrate multiple PPI databases for comprehensive research14,21. Converting the complexes into pairs in PINs will increase the number of interactions and may exacerbate the lack of overlap, despite being the prevailing format in co-elution proteomics studies8,9.

From three integrated databases, we identified 96,635 PPIs with 96,616 being non-monomeric interactions. The combined databases offer a reference for exploring the complexity and dynamics of human PPIs.

Nine protein matrices were annotated using the integrated databases described above, yielding interactions labeled as positive, negative, or unknown (Fig. 2C). A two-stage strategy is implemented to address the scarcity and limited diversity of known true PPIs. The first-stage filtering employed a 1:5 negative-to-positive ratio to ensure comprehensive coverage of potentially true interactions before downstream augmentation. In the second stage, high-confidence positive interactions (PrInCE score > 0.5) underwent two-tier augmentation. This involved (1) random value perturbation, where protein elution profile values were scaled by a random factor between 90% and 110% and (2) missing-value perturbation, where the positions of the missing values were randomly either retained in place or shifted to an adjacent time point. Fifty representative traces per original profile were selected from augmented positives based on a composite score.

The combined dataset (1:1 negative-to-positive ratios) contained 14.7 to 15 million traces per cell line, split 8:1:1 into training, validation and test sets. The XGBoost model was optimized to maximize the area under the precision-recall curve (AUPRC) via three-fold cross-validation and evaluated performance on internal and external datasets. Our machine learning model enables the prediction of co-eluting proteins in thyroid cancer cell lines, validated in an external dataset.

A machine learning model for protein interaction prediction
We built an XGBoost model for co-eluted proteins prediction and evaluated its performance using the area under the receiver operating characteristic curve (AUROC) and AUPRC.

In the initial phase, the TPC-1 cell line contained 27,794 positive interactions. The remaining interactions were annotated as either negative (6,139,613) or unlabeled (8,548,311). FTC238 showed comparable positive interactions at 27,615 but exhibited 6,255,162 negative and 8,723,199 unlabeled interactions. Notably, Nthy-ori 3-1 displayed 31,076 positive, 8,397,238 negative and 12,926,198 unlabeled interactions. The greater interaction counts in Nthy-ori 3-1 directly reflect its larger proteome coverage (N), since the potential number of pairs scales with N²/2 (Supplementary Fig. 2A). We evaluated different downsampling ratios and determined that a 1:5 negative-to-positive ratio (precision=0.28, recall=0.73) offered the optimal balance, minimizing the excessive false negatives observed at higher ratios. The 1:5 ratio achieved stable precision in random forest models, as confirmed by five-fold cross-validation (Supplementary Fig. 2B). Density plots verified effective enrichment of high-confidence interactions (PrInCE score > 0.5) under this ratio (Supplementary Fig. 2C). We selected 20,635 co-eluted proteins with positive labels from TPC-1, 20,373 from FTC238 and 22,947 from Nthy-ori 3-1.

To address the scarcity and limited diversity of known PPIs, we employed data augmentation by introducing random perturbations to positive samples. The second phase involved data augmentation through random value perturbation and missing-value perturbation as described above (Supplementary Fig. 2D). The XGBoost model was trained using five key features on group-stratified splits of 44.7 million co-elution traces from three cell lines: TPC-1 with 14.7 million traces (14,715,718), FTC238 with 15.0 million (15,005,976), and Nthy-ori 3-1 with 14.9 million (14,969,894). The final 1:1 balanced dataset was divided into training, validation and test sets, with additional FTC133 data for external validation.

Model evaluation showed robust internal performance in cross-validation, with an AUROC of 0.81 (Fig. 3A) and an AUPRC of 0.82 (Fig. 3B). Hyperparameter optimization in the internal validation set yielded an AUROC of 0.77 (Fig. 3C) and an AUPRC of 0.78 (Fig. 3D). Hold-out test results achieved an AUROC of 0.78 (Fig. 3E) and an AUPRC of 0.79 (Fig. 3F). External validation on the FTC133 dataset at the native sample ratio yielded an AUROC of 0.68 (Supplementary Fig. 2G). The reduced AUROC during external validation reflects both technical differences across cell lines and the inherent challenge of class imbalance in native proteomic data. For high-confidence predictions using a strict probability threshold of 0.95, weighted precision achieved 0.5 in the FTC133 dataset (Fig. 3G), which contained 6558 labeled interactions out of 18,811 interactions (precision > 0.5) from integrated databases (Fig. 3H). The three-cell consensus set of 25,173 interactions exhibited comparable performance with a probability higher than 0.95 and a weighted precision over 0.6 (Supplementary Fig. 2E), including 1197 interactions that were previously documented in curated databases (Supplementary Fig. 2F).

Notably, our approach overcomes a critical limitation of existing co-eluted proteins prediction that require multi-dataset integration for reliable performance9. Previous single-dataset implementations showed limited predictive accuracy with AUROC of 0.65 as reported9. Through systematic integration of data augmentation with the XGBoost algorithm, we achieved a 21.5% improvement in single-dataset predictive accuracy, while maintaining robust external validation performance under native class imbalance.

Protein interaction landscape of thyroid cell lines
To systematically characterize the biological functions of protein interactions identified across three thyroid cell lines (Nthy-ori 3-1, FTC238 and TPC-1), we implemented a two-tiered analytical framework on high-confidence interactions (weighted precision > 0.6). We firstly conducted Kyoto Encyclopedia of Genes and Genomes (KEGG) pathway enrichment analysis (Supplementary Data 1). Shared interactions among all three cell lines were further analyzed through network visualization and Gene Ontology (GO) term enrichment, revealing conserved functional modules in thyroid biology.

The top ten significant KEGG pathways included ribosome, proteasome, biosynthesis of amino acids, DNA replication and cysteine and methionine metabolism (Fig. 4A). Ribosome and proteasome complexes as the top two pathways align with mass spectrometry-based PPI studies21,22, further validating the reliability of ProteoAutoNet workflow. We prioritized pathways by combining the p-value and enrichment score. This approach captures stable protein complexes, leading to focus on five major PINs. The proteasome pathway ranked highest and was followed by sulfur metabolism, DNA replication, ribosome, and 2-oxocarboxylic acid metabolism.

ProteoAutoNet detected 37 out of 43 known proteasome components from the KEGG database (Supplementary Fig. 3A), exhibiting the coverage of 20S core particle with its PSMA and PSMB families23, and the 19S regulatory particle with its PSMC and PSMD families24. We also detected partial components of regulatory factors including the PSME and PSMF families. The detected 20S-19S-regulatory factor architecture matches the known proteasome organization24–26. This methodological convergence reinforces the validity of ProteoAutoNet for studying the proteasome complex. The sulfur metabolism (7 out of 10 known components) (Supplementary Fig. 3B), DNA replication (22/36) (Supplementary Fig. 3C), ribosome pathway (72/131) (Supplementary Figs. 3D, E) and 2-oxocarboxylic acid metabolism (10/18) (Supplementary Fig. 3F) were detected based on KEGG database annotations. We constructed PINs for these pathways and performed Markov clustering (MCL) to identify functional modules.

We displayed common PINs from three cell lines and highlighted edges with experimental evidence in STRING (Supplementary Fig. 3G), and identified three functional modules with extensive overlap with the STRING: translation (Fig. 4B), proteasome-ubiquitin (Fig. 4C) and protein folding (Fig. 4D). As the module with the highest overlap with prior knowledge, the detection of 40S/55S/60S ribosomal subunits and eukaryotic initiation factor 3 (eIF3) complex across all three cell lines validates the reliability of ProteoAutoNet results (Fig. 4B). Besides, the detection of proteasome corroborates preclinical evidence for proteasome inhibitors in medullary thyroid cancer (MTC)27. The Prefoldin and chaperonin-containing TCP-1 (CCT) complexes participate in polypeptide folding, consistent with their canonical roles in eukaryotic protein folding pathways28. Some subunits of these complexes have been reported as biomarkers in gastric and lung cancer29,30.

Although clinical translation remains limited to date, the co-eluted profiles of these complexes and PINs provide a framework to reevaluate their clinical potential. They may guide future studies with improved patient selection.

Identification of protein interactions and potential biomarkers
Using the normal thyroid cell line Nthy-ori 3-1 as the baseline control, we analyzed PINs in FTC238 and TPC-1 cell lines. PPIs were classified as cancer-associated only when they met the following criteria. The analysis retained PPIs exhibiting retention time shifts (ΔRT ≤ 38 sec) across three replicates, while excluding those proteins with molecular weights smaller than those annotated in UniProt. This dual-filter criterion identified 8984 cancer-related PPIs with reproducible profiles.

Methods

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_1": "Review of NCOMMS-25-11440, “ProteoAutoNet: high-throughput co-eluted protein analysis with robotics and a multi-database search strategy\" by Guo and colleagues.\nThis is an interesting automation workflow with potential broad applications for CFMS style interactomic studies. Overall, the ProteoAutoNet offers the potential as a method for reducing intrasample variability and overall analysis time. However, there are some performance gaps warranting a more thorough comparison between robotically and manually-prepared samples would help to elucidate the relative advantages, and whether there are drawbacks in terms of coverage and replicability. Additionally, integration of the software component isn’t fully convincing, as iRefIndex, while well benchmarked, has low overlap with other public databases. Notably, there is approximately a 20% drop in IDs when only considering non-iRefIndex entries, which is worth exploring to improve confidence in the computational aspects of the pipeline.\nSpecific points warranting attention:\n-Use of 9 unfractionated samples: protein abundance is used as QC measure, but overlap of protein IDs and coverage between replicates as well as comparison to non-robotically prepped samples would provide more objective QC measures\n-In general, comparison of robotically-prepped samples to standard workflow would give more robust evidence of benefits of robotic workflow outside of time/effort savings -Page 4, line 151, authors state that they use 81 raw LC-DIA/MS files to generate their database. A further explanation of these files, species, fractionation, quantity etc. would help elucidate this database. The methods page lists 576 DIA files.\n-iRefIndex identified many more PINs in Figure 3B than Figure 2C. Given that Figure 2 is generated from the full human proteome, this suggests that iRefIndex IDed many predictive complexes, and should not be weighted equally with the other three. Similarly, the overlap between databases other than iRefIndex in Figure 2 is much higher in 2C than with Figure 3B. It would be more instructive and easier to visualize if Figure 3B and 2C were both shown as Venn diagrams.\n-Based on figure 2C, there are approx. 17,828 PINs that overlap in more than 2 non-iRefIndex databases. This number is 5,610 for the thyroid cell dataset. This number is lower than the 6665 quoted in the text, but is worth considering as it could rule out interactions with less supporting experimental data", "reviewer_2": "The authors present a robotics-assisted method for sample preparation in Co-Fractionation Mass Spectrometry (CF-MS) experiments. While CF-MS is a powerful technique for studying stable protein interactions, sample preparation for a single experiment typically requires 3-5 days of manual work. The authors use automated laboratory equipment to replace manual performance of two key steps: 1) enzymatic digestion of fractions and 2) peptide cleanup. This roboticization allows more plates to be processed than is feasible manually (performing complex digestion for six plates for 18 hours continuously, and peptide cleanup for 12 hours continuously). The robotics implementation for sample preparation addresses a genuine bottleneck in CF-MS workflows.\nHowever, my concerns are will the computational analysis. While the addition of robotics to the sample preparation workflow is useful, the data analysis and prediction of protein interactions uses existing tools (CCprofiler and PrInCE). Additionally, I have serious concerns with the construction of the gold standard training set.\nGold standard dataset construction: I'm concerned about the \"multi-database search strategy\" for constructing training sets. Firstly, it is not so novel to pull examples of known interactions from multiple databases in order to construct a larger training set. It is fairly standard practice gather examples of known interactions from multiple databases in order to construct a larger training set.\nWhile the authors pull protein interactions from four databases (CORUM, Complex Portal, hu.MAP, and iRefIndex), iRefIndex is a large outlier in terms of size and overlap with other sources. Of the ~170,000 interactions in iRefIndex, only 12, 28, and 200 overlap with the other databases. This strongly suggests that the interactions in iRefIndex are not reflective of stable protein interactions that CF-MS is designed to capture and should be removed from the training set. While the iRefIndex resource website is currently down, it does not make sense to me to me that an interaction database that says it includes CORUM would only contain 28 PPIs in common with CORUM. Additionally, the size of complexes in iRefIndex appears biologically unrealistic (too large) and distinct from curated protein complex databases (Figure 3A). I\nThis strongly suggests to me that the interactions in iRefIndex are not reflective of stable protein interactions, and should be removed from the training set. Additionally, while the iRefIndex resource website is currently down, it does not make sense to me to me that an interaction database that says it includes CORUM would only contain 28 PPIs in common with CORUM. Additionally, the size of complexes in the iRefIndex are biologically unrealistic (too large), and a distinct outlier from the curated protein complex databases (Figure 3A). In the refinement stage, 1754 proteins are removed from iRefIndex, which only removes 3 protein complexes.\nInteraction labeling approach: If protein pair was labeled as a PPI in any of the four databases, it is labeled as true. This extremely permissive criterion likely incorporates hundreds of thousands of false positives from iRefIndex\nComputational analysis: While the statistical techniques employed are appropriate in principle, CCProfiler performs relatively similarly with CORUM, hu.MAP and Complex Portal, but iRefIndex is again a large outlier (Figure 3D). I'm highly concerned about the effect of including ~170,000 protein interactions from iRefIndex as TRUE labels, specially on the calibration of the false discovery rates.\nThere is a quite low level of overlap between the predicted interactions from CCProfiler and PrInCE. Additionally, 1.4 million co-eluted protein pairs identified by PrInCE (above 20% FDR?) is unrealistically and extremely high. I would like to see a precision-recall curve for the predicted interactions, to understand how the predicted interactions overlap with the curated protein complex databases.\nThe use of isotonic regression is an appropriate approach to transform raw prediction scores to probability estimates, and linear interpolation is a reasonable approach to estimate precision from scores.\nComplex identification: It's not clear to me if the complexes presented in Figures 5 and 6 are the only complexes identified, as this is a very low number that does not seem to include major abundant protein complexes. Compared to other protein complex identification papers from CF-MS, this is a very low number of identified complexes.\nComplex modelling: Alphafold predicted protein complex images for PFAS/HK1/TGM2 are not clearly presented, and it is difficult to see where the predicted interfaces are, as both subunits are the same colors. Is it possible to combine these three individual alphafold predictions into a single prediction of the 3-protein complex? If there are major steric clashes between the pairwise subunit predictions when overlapped, it would decrease confidence that this is a true 3-protein complex. These candidate novel interactions could be supported with use of the ipTM score or the IPSAE score (PMID: 39990437).\nIn Figure 6B, PFAS/HK1/TGM2 are shown to have an extra peak in Nthy around 18m, however, this peak is not visible in the heatmap in figure 6A.\nRecommendations:\n- Remove iRefIndex from the training set\n- Provide a precision-recall curve for the predicted interactions, showing overlap with the curated protein complex databases\n- Clarify the number of interactions and complexes identified\n- Remove references to multi-database search strategy, as it is standard practice to use multiple databases to construct a gold-standard dataset of known interactions\n- Improve presentation of alphafold predicted protein complex images\n-Emphasize novelties in the computational strategy compared to past CF-MS papers.\nWhile the robotics approach to sample preparation is innovative and does shorten the experimental time, the computational analysis relies on existing tools and employs a questionable strategy for defining true interactions. The major issue is the use of a large number of PPIs from the iRefIndex database, which is an extreme outlier from other protein complex databases. With improvements to the computational methodology and more rigorous validation of predictions, this work could make a more substantial contribution to the field.", "reviewer_3": "Lyu et al. present ProteoAutoNet, a workflow combining robotics-assisted CF-MS sample processing and a multi-database search strategy. The workflow is applied to three thyroid cell lines to identify cancer specific protein-protein interactions. While the automated workflow provides clear benefits with respect to throughput and experimental reproducibility, the computational part of ProteoAutoNet is insufficiently described and benchmarked.\nMajor points:\n1) The computational method of ProteoAutoNet for integrating multiple database search results is only very briefly and superficially described in the methods section. The authors should provide more detailed information and a performance benchmark. It is not clear how ProteoAutoNet relates to the findings presented in Figure 3.\n2) If protein pairs rather than protein complexes are of interest, also alternative strategies, such as SECAT or PCprophet could be evaluated.\n3) Conceptually, I would generally advise to combine databases prior to CCprofiler or PrInCE analysis (rather than post-processing combination), which will keep statistical models of these tools intact. Have the authors tested this approach?\n4) While the reported results in the cancer cell lines are certainly interesting, it is not clear whether these were only possible because of the adjusted analysis approach in ProteoAutoNet or whether a traditional CCprofiler or PrInCE analysis would also report these findings.\nMinor points:\n5) Line 61 on page 2: check sentence structure “the performance of the in”"}, "author_response": "Point by point response to the reviewer's comments:\nReviewer #1 (Remarks to the Author):\nReview of NCOMMS-25-11440, “ProteoAutoNet: high-throughput co-eluted protein analysis with\nrobotics and a multi-database search strategy\" by Guo and colleagues.\nThis is an interesting automation workflow with potential broad applications for CFMS style\ninteractomic studies. Overall, the ProteoAutoNet offers the potential as a method for reducing\nintrasample variability and overall analysis time. However, there are some performance gaps\nwarranting a more thorough comparison between robotically and manually-prepared samples\nwould help to elucidate the relative advantages, and whether there are drawbacks in terms of\ncoverage and replicability.\nReply: We appreciate the reviewer's kind recognition of our work. We have added manually\nprepared samples and used the coefficient of variation (CV) of protein counts and abundance as\nbenchmarks. The results showed superior protein coverage and quantitative reproducibility in the\nrobotics-assisted platform. Please see our point-by-point response below for the detailed results.\nAdditionally, integration of the software component isn’t fully convincing, as iRefIndex, while\nwell benchmarked, has low overlap with other public databases. Notably, there is approximately a\n20% drop in IDs when only considering non-iRefIndex entries, which is worth exploring to\nimprove confidence in the computational aspects of the pipeline.\nReply: Thanks for your suggestions. We have removed iRefIndex based on your and other\nreviewers’ comments. We have employed XGBoost and data augmentation, and used AURPC and\nAUROC to evaluate the computational confidence. The detailed results are presented in our point-\nby-point response.\n1. Use of 9 unfractionated samples: protein abundance is used as QC measure, but overlap of\nprotein IDs and coverage between replicates as well as comparison to non-robotically prepped\nsamples would provide more objective QC measures\nReply: We appreciate the reviewer's insightful suggestion regarding more objective quality control\nmeasures. We have added new experiments that include a non-robotic group to compare protein\nidentification and intensity with the robotic platform. The added text reads as follows:\n“We compared the robotics-assisted platform to manual processing using unfractionated TPC-1\nsamples to validate the performance of the platform further. In the robotical processing group\nversus the manual processing group, the coefficient of variation (CV) of protein counts decreased\nfrom 1.08% to 1.01% (Fig. 1E). The number of proteins confirmed the reliability of the robotics-\nassisted platform (sFig. 1B). The mean protein counts were comparable robotic (3408 ± 34)\nand manual processing (3404 ± 37). Protein counts in the robotical group showed 25th and\n75th percentiles at 3398 and 3428 respectively, compared to 3381 and 3437 for manual\nprocessing. We also evaluated reproducibility by calculating the CV of protein abundance. The\nresults showed superior quantitative reproducibility in the robotic processing group, with a lower\nmedian coefficient of variation (20.1% versus 21.6% in manual processing) and a narrower\ninterquartile range (12.6% versus 14.4%) (Fig. 1F).”\nWe have added the details of the new experiment in “Automated desalting and digestion of co-\nfractionated proteins” of the Materials and Methods sections:\n1\n“To validate the performance of the automated platform, we compared the robotics-assisted\nprocessing against manual processing using eight unfractionated TPC-1 samples for each group.\nBoth sets of samples underwent identical processing conditions following the above protocol for\ndenaturation, reduction, alkylation, digestion, and desalting steps.”\n2. In general, comparison of robotically-prepped samples to standard workflow would give more\nrobust evidence of benefits of robotic workflow outside of time/effort savings\n-Page 4, line 151, authors state that they use 81 raw LC-DIA/MS files to generate their database. A\nfurther explanation of these files, species, fractionation, quantity etc. would help elucidate this\ndatabase. The methods page lists 576 DIA files.\nReply: Thanks for your suggestions. We have added new experiments, which are a non-robotic\ngroup to compare protein identification and intensity with the robotic platform as mentioned\nabove. The results showed superior protein coverage and quantitative reproducibility in the\nrobotics-assisted platform.\nThe 81 raw LC-DIA/MS files were from a public dataset, while the 576 DIA files were generated\nby us in this study.\nThe 81 files were obtained from published HEK293 datasets (PMID: 32690956), with fractions\ncollected under optimized conditions as follows: elution range of 10-28 min (0.19 min/fraction),\nflow rate of 0.5 mL/min, yielding 81 total fractions for quantification. However, to enable a more\npredictive and generalized framework for identifying co-eluted patterns, we have removed it from\nthe revised manuscript and built an XGBoost model in its place to predict co-eluted proteins,\nbenchmarking its performance with AUPRC and AUROC.\nWe added the composition of 576 files in “DIA-MS analysis of the cell lines” of Materials and\nMethods session:\n“We acquired a total of 576 analyzable DIA files, including 540 co-fractionated samples from\nthree cell lines with three SEC replicates (60 SEC fractions per replicate collected every 19\nseconds between 9-28 minutes), 27 unfractionated quality control samples of automated platform\n(three technical replicates per SEC replicate), and 9 batch control injections of pooled three cell\nline mixtures for MS acquisition.”\n3. iRefIndex identified many more PINs in Figure 3B than Figure 2C. Given that Figure 2 is\ngenerated from the full human proteome, this suggests that iRefIndex IDed many predictive\ncomplexes, and should not be weighted equally with the other three. Similarly, the overlap\nbetween databases other than iRefIndex in Figure 2 is much higher in 2C than with Figure 3B. It\nwould be more instructive and easier to visualize if Figure 3B and 2C were both shown as Venn\ndiagrams.\nReply: Thank you very much for your suggestions. As you pointed out, the iRefIndex database\nmay introduce substantial noise. Therefore, we have removed iRefIndex and merged the\nremaining three databases according to your suggestion. In the revised version, to establish a more\npredictive and generalized framework for identifying co-eluted patterns, we have developed an\nXGBoost model for co-eluted proteins.\nThe brief workflow of the XGBoost modeling is presented in the “A machine learning model for\nprotein interaction prediction” of Results session:\n2\n“We built an XGBoost model for co-eluted protein prediction and evaluated its performance using\nthe area under the receiver operating characteristic curve (AUROC) and the area under the\nprecision-recall curve (AUPRC).\nIn the initial phase, the TPC-1 cell line contained 27,794 positive interactions. The remaining\ninteractions were annotated as either negative (6,139,613) or unlabeled (8,548,311). FTC238\nshowed comparable positive interactions at 27,615 but exhibited 6,255,162 negative and\n8,723,199 unlabeled interactions. Notably, Nthy-ori 3-1 displayed 31,076 positive, 8,397,238\nnegative and 12,926,198 unlabeled interactions. The greater interaction counts in Nthy-ori 3-1\ndirectly reflect its larger proteome coverage (N), since the potential number of pairs scales with N\n²/2 (sFig. 2A). We evaluated different downsampling ratios and determined that a 1:5 negative-to-\npositive ratio (precision = 0.28, recall = 0.73) offered the optimal balance, minimizing the\nexcessive false negatives observed at higher ratios. The 1:5 ratio achieved stable precision in\nrandom forest models, as confirmed by five-fold cross-validation (sFig. 2B). Density plots verified\neffective enrichment of high-confidence interactions (PrInCE score > 0.5) under this ratio (sFig.\n2C). We selected 20,635 co-eluted proteins with positive labels from TPC-1, 20,373 from FTC238\nand 22,947 from Nthy-ori 3-1.\nTo address the scarcity and limited diversity of known protein-protein interactions (PPIs), we\nemployed data augmentation by introducing random perturbations to positive samples. The\nsecond phase involved data augmentation through random value perturbation and missing-value\nperturbation as described previously (sFig. 2D). The XGBoost model was trained using five key\nfeatures on group-stratified splits of 44.7 million co-elution traces from three cell lines: TPC-1\nwith 14.7 million traces (14,715,718), FTC238 with 15.0 million (15,005,976), and Nthy-ori 3-1\nwith 14.9 million (14,969,894). The final 1:1 balanced dataset was divided into training,\nvalidation and test sets, with additional FTC133 data for external validation.”\nAdditionally, your suggestion to present Figures 3B and 2C as Venn diagrams is well taken. Since\nwe have established a machine learning model in the revised version and no longer need to\ncompare the analysis results with existing software (CCprofiler and PrInCE). Venn diagrams are\nnot included in the updated version.\n4. Based on figure 2C, there are approx. 17,828 PINs that overlap in more than 2 non-iRefIndex\ndatabases. This number is 5,610 for the thyroid cell dataset. This number is lower than the 6665\nquoted in the text, but is worth considering as it could rule out interactions with less supporting\nexperimental data\nReply: We thank the reviewer for highlighting this important distinction. The lower number of\noverlapping PINs in the thyroid cell dataset (5610 vs. 17,828 in non-iRefIndex databases) likely\nstems from two key factors in our initial analysis:\n1) The limited predictive power of the original PrInCE (naive Bayes-based model) for identifying\nco-eluted proteins.\n2) Overly stringent data/cell strategy correction applied to the data analysis of the three cell lines.\nTo address these limitations, we have implemented significant improvements in the revised\nmanuscript:\n1) Combined three complex databases (no iRefIndex) as gold standard.\n2) Developed a more robust XGBoost-based model to enhance co-elution prediction accuracy.\nPPIs” of Materials and Methods:\n“We established a machine learning-based framework for predicting protein-protein interactions\nby integrating three reference databases as the gold standard and employing XGBoost algorithms.\nWe extracted the column names of databases using R (v4.3.0) and refined using the human FASTA\n(May 2024) to remove non-matching proteins and complexes, with the counts detailed in Fig. 2A.\nComplexes from CORUM, Complex Portal and hu.MAP were combined as a human interaction\nreference dataset. Protein interactions were derived from the set of identified proteins (N), yielding\na total of N × (N - 1) / 2 possible interactions. However, the set of known true interactions from\ngold-standard databases is sparse relative to this theoretical maximum.\nWe implemented a two-phase strategy to build a robust predictive model from these sparse\nannotations. In the initial phase, sensitivity-optimized downsampling (1:5 negative-to-positive\nratio) was performed across all three cell lines (TPC-1, FTC238, and Nthy-ori 3-1) using the\nPrInCE R package (v1.16.0, R4.3.0). Protein interactions were annotated against integrated\ndatabase from CORUM, Complex Portal and hu.MAP, with random forest classification (5-fold\nCV) identifying high-confidence interactions (score > 0.5) for data augmentation. The second\nphase involved two-track data augmentation of high-confidence interactions: (1) random value\nperturbation, where protein elution profile values were scaled by a random factor between 90%\nand 110%; and (2) missing-value perturbation, where the positions of missing values were\nrandomly either retained in place or shifted to an adjacent time point. This augmentation process\nwas implemented in Python 3.9.21. Representative augmented pairs were selected based on a\ncomposite score: score = (n_valid × 0.5) + (|Pearson_r| × 30) − (euc_dist × 0.01), from which\nthe top 40 and bottom 10 scoring pairs were retained. Final training sets combined original and\naugmented positives and matched negatives (1:1 ratio). The XGBoost classifier (Python 3.9.21)\nwas trained using stratified partitioning via GroupShuffleSplit by protein ID to maintain group\nintegrity (80% training, 10% validation, and 10% test sets). Five interaction features were\nemployed: smoothed and raw Pearson correlation coefficients, Euclidean distance, co-peak\nfrequency, and weighted cross-correlation (WCC) scores. Hyperparameter optimization was\nconducted through three-fold stratified grid search covering max_depth (4-8), learning_rate\n(0.01-0.1), subsample ratio (0.6-1.0), with early stopping after 20 rounds of no improvement.\nClass imbalance was addressed by setting scale_pos_weight to the negative-to-positive ratio. The\nfinal model was selected based on AUPRC performance during cross-validation, with optimal\nparameters retrained on the complete 80% training dataset. Final model performance was\nevaluated on 10% internal test set and an independent external validation set of FTC133 cell line.\nWe employed two methods for precision calculation. The first method treated all unknown samples\nas negative instances (0), and precision was calculated as TP / (TP + FP), where TP denotes true\npositives and FP denotes false positives. However, this approach may underestimate the true\nprecision, since unknown samples are likely to contain some true positives. Therefore, we also\napplied a weighted precision method, which assigns the predicted probability of each unknown\nsample as its positive-class weight. These weighted precisions were combined with labeled\npositive and negative samples, and all interactions were ranked by confidence to compute a\ncumulative precision curve.”\nWe also summarized the benchmarking and performance of the XGBoost model in the Results:\n4\n“Model evaluation showed robust internal performance with cross-validation AUROC of 0.81 (Fig.\n3A) and AUPRC of 0.82 (Fig. 3B). Hyperparameter optimization in the internal validation set\nyielded an AUROC of 0.77 (Fig. 3C) and an AUPRC of 0.78 (Fig. 3D). Hold-out test results\nachieved an AUROC of 0.78 (Fig. 3E) and an AUPRC of 0.79 (Fig. 3F). External validation on the\nFTC133 dataset at the native sample ratio yielded an AUROC of 0.68 (sFig. 2G). The reduced\nAUROC during external validation reflects both technical differences across cell lines and the\ninherent challenge of class imbalance in native proteomic data. For high-confidence predictions\nusing a strict probability threshold of 0.95, weighted precision achieved 0.5 in the FTC133 dataset\n(Fig. 3G), which contained 6558 labeled interactions out of 18,811 interactions (precision > 0.5)\nfrom integrated databases (Fig. 3H). The three-cell consensus set of 25,173 interactions exhibited\ncomparable performance with a probability higher than 0.95 and a weighted precision over 0.6\n(sFig. 2E), including 1197 interactions that were previously documented in CORUM (sFig. 2F). ”\nThe above systematic improvements have significantly enhanced the reliability of the model,\nproviding a robust computational framework for PPI study.\nThe authors present a robotics-assisted method for sample preparation in Co-Fractionation Mass\nSpectrometry (CF-MS) experiments. While CF-MS is a powerful technique for studying stable\nprotein interactions, sample preparation for a single experiment typically requires 3-5 days of\nmanual work. The authors use automated laboratory equipment to replace manual performance of\ntwo key steps: 1) enzymatic digestion of fractions and 2) peptide cleanup. This roboticization\nallows more plates to be processed than is feasible manually (performing complex digestion for\nsix plates for 18 hours continuously, and peptide cleanup for 12 hours continuously). The robotics\nimplementation for sample preparation addresses a genuine bottleneck in CF-MS workflows.\nHowever, my concerns are will the computational analysis. While the addition of robotics to the\nsample preparation workflow is useful, the data analysis and prediction of protein interactions uses\nexisting tools (CCprofiler and PrInCE). Additionally, I have serious concerns with the construction\nof the gold standard training set.\nGold standard dataset construction: I'm concerned about the \"multi-database search strategy\" for\nconstructing training sets. Firstly, it is not so novel to pull examples of known interactions from\nmultiple databases in order to construct a larger training set. It is fairly standard practice gather\nexamples of known interactions from multiple databases in order to construct a larger training set.\nReply: Thank you very much for your positive feedback on the ProteoAutoNet automated\nworkflow. We agree that integrating databases is not rare in the literature. Our improved approach\nintroduces several key methodological advancements that enhance the quality and reliability of the\nmachine-learning model for PPI prediction:\nWe developed an XGBoost model to predict co-eluted proteins using a two-stage sample\naugmentation strategy to address the scarcity and limited diversity of known true PPIs and\nenhance the model's ability to generalize. We employed a XGBoost-based model utilizing a two-\nstage sample augmentation strategy. This approach resulted in a 21.5% improvement in predictive\nperformance on a single dataset, achieving an AUROC of 0.78 in an internal hold-out test dataset.\nThe five inputs of the model as follows:\na. Pearson correlation of raw elution traces\nb. Pearson correlation of smoothed elution traces\nc. Euclidean distance\nd. Co-peak detection\ne. Weighted cross-correlation (WCC)\nThe first four inputs were calculated using Python implementations consistent with the PrInCE\npackage (PMID: 33471077), while the WCC was derived from a Python adaptation of the wccsom\nR package's algorithm (PMID: 22939629).\nIn summary, we have removed all “multi-database search strategy” from manuscript, including in\nthe title. The revised manuscript, now entitled “ProteoAutoNet: high-throughput co-eluted protein\nanalysis with robotics and machine learning” and the revised version not only strengthens the\nreliability of the gold standard but also enhances the performance of the model, providing an\neffective computational framework for PPI studies.\nWhile the authors pull protein interactions from four databases (CORUM, Complex Portal,\n6\nhu.MAP, and iRefIndex), iRefIndex is a large outlier in terms of size and overlap with other\nsources. Of the ~170,000 interactions in iRefIndex, only 12, 28, and 200 overlap with the other\ndatabases. This strongly suggests that the interactions in iRefIndex are not reflective of stable\nprotein interactions that CF-MS is designed to capture and should be removed from the training\nset. While the iRefIndex resource website is currently down, it does not make sense to me to me\nthat an interaction database that says it includes CORUM would only contain 28 PPIs in common\nwith CORUM. Additionally, the size of complexes in iRefIndex appears biologically unrealistic\n(too large) and distinct from curated protein complex databases (Figure 3A).\nThis strongly suggests to me that the interactions in iRefIndex are not reflective of stable protein\ninteractions, and should be removed from the training set. Additionally, while the iRefIndex\nresource website is currently down, it does not make sense to me to me that an interaction\ndatabase that says it includes CORUM would only contain 28 PPIs in common with CORUM.\nAdditionally, the size of complexes in the iRefIndex are biologically unrealistic (too large), and a\ndistinct outlier from the curated protein complex databases (Figure 3A). In the refinement stage,\n1754 proteins are removed from iRefIndex, which only removes 3 protein complexes.\nInteraction labeling approach: If protein pair was labeled as a PPI in any of the four databases, it is\nlabeled as true. This extremely permissive criterion likely incorporates hundreds of thousands of\nfalse positives from iRefIndex.\nReply: We sincerely appreciate the reviewer's critical observations regarding iRefIndex. Based on\nyour suggestions and our additional investigations, we have completely excluded iRefIndex from\nour revised analysis.\nWe agree that its inclusion introduced biologically unrealistic interactions and compromised the\nreliability of our gold standard. To directly address your concerns, we re-analyzed the data using\nonly the three curated databases (CORUM, hu.MAP, and Complex Portal). This refinement\nsubstantially improved the biological relevance. A key example is the novel PIN: HK-TGM2-\nPFAS.\nIn previous analysis using iRefIndex, the interaction between PFAS and HK1 is labeled as true. It\nwas likely spurious, originating from the iRefIndex entry “iRef_4137891”. This spurious link\nresulted in the prediction of a PIN: TGM2-HK1-PFAS. After removing iRefIndex, the PFAS-HK1\nlink was eliminated. The PFAS-HK1 interaction was predicted with high probability (0.936) in our\nXGBoost model, but it did not meet our stringent confidence threshold (prob > 0.95).\nConsequently, we now more accurately report novel interactions between TGM2 and HK1 (ipTM\n= 0.61, pTM = 0.66), whereas the removed PFAS-HK1 (ipTM = 0.25, pTM = 0.65) had low\nconfidence in Alphafold3.\nThis case exhibits that excluding iRefIndex effectively filtered out lower-quality interactions,\nleading to a more reliable biological relevance of PPIs. We have updated all the figures and results\nin the manuscript.\nComputational analysis: While the statistical techniques employed are appropriate in principle,\nCCProfiler performs relatively similarly with CORUM, hu.MAP and Complex Portal, but\niRefIndex is again a large outlier (Figure 3D). I'm highly concerned about the effect of including\n~170,000 protein interactions from iRefIndex as TRUE labels, especially on the calibration of the\nfalse discovery rates.\nPrInCE. Additionally, 1.4 million co-eluted protein pairs identified by PrInCE (above 20% FDR?)\nis unrealistically and extremely high. I would like to see a precision-recall curve for the predicted\ninteractions, to understand how the predicted interactions overlap with the curated protein\ncomplex databases.\nThe use of isotonic regression is an appropriate approach to transform raw prediction scores to\nprobability estimates, and linear interpolation is a reasonable approach to estimate precision from\nscores.\nReply: We sincerely appreciate the reviewer's thorough analysis and valid concerns regarding\ndatabase consistency and prediction reliability. We agree that the inclusion of iRefIndex and\nlimited diversity of CF-DIA-MS data caused challenges in predicting reliable PPIs.\nTo address these limitations, we have implemented significant improvements in the revised\nmanuscript:\n1) Combined three complex databases (no iRefIndex) as gold standard.\n2) Developed a more robust XGBoost-based model enhanced with data augmentation.\nBy introducing random perturbations to the known true PPIs, this strategy effectively rebalanced\nthe dataset and mitigated the scarcity and limited diversity of positive samples, thereby enhancing\nco-elution prediction accuracy. We described the machine learning framework in “Machine\nlearning framework for predicting PPIs” of Materials and Methods:\n“We established a machine learning-based framework for predicting protein-protein interactions\nby integrating three reference databases as the gold standard and employing XGBoost algorithms.\nWe extracted the column names of databases using R (v4.3.0) and refined using the human FASTA\n(May 2024) to remove non-matching proteins and complexes, with the counts detailed in Fig. 2A.\nComplexes from CORUM, Complex Portal and hu.MAP were combined as a human interaction\nreference dataset. Protein interactions were derived from the set of identified proteins (N), yielding\na total of N × (N - 1) / 2 possible interactions. However, the set of known true interactions from\ngold-standard databases is sparse relative to this theoretical maximum.\nWe implemented a two-phase strategy to build a robust predictive model from these sparse\nannotations. In the initial phase, sensitivity-optimized downsampling (1:5 negative-to-positive\nratio) was performed across all three cell lines (TPC-1, FTC238, and Nthy-ori 3-1) using the\nPrInCE R package (v1.16.0, R4.3.0). Protein interactions were annotated against integrated\ndatabase from CORUM, Complex Portal and hu.MAP, with random forest classification (5-fold\nCV) identifying high-confidence interactions (score > 0.5) for data augmentation. The second\nphase involved two-track data augmentation of high-confidence interactions: (1) random value\nperturbation, where protein elution profile values were scaled by a random factor between 90%\nand 110%; and (2) missing-value perturbation, where the positions of missing values were\nrandomly either retained in place or shifted to an adjacent time point. This augmentation process\nwas implemented in Python 3.9.21. Representative augmented pairs were selected based on a\ncomposite score: score = (n_valid × 0.5) + (|Pearson_r| × 30) − (euc_dist × 0.01), from which\nthe top 40 and bottom 10 scoring pairs were retained. Final training sets combined original and\naugmented positives and matched negatives (1:1 ratio). The XGBoost classifier (Python 3.9.21)\nwas trained using stratified partitioning via GroupShuffleSplit by protein ID to maintain group\n8\nintegrity (80% training, 10% validation, and 10% test sets). Five interaction features were\nemployed: smoothed and raw Pearson correlation coefficients, Euclidean distance, co-peak\nfrequency, and weighted cross-correlation (WCC) scores. Hyperparameter optimization was\nconducted through three-fold stratified grid search covering max_depth (4-8), learning_rate\n(0.01-0.1), subsample ratio (0.6-1.0), with early stopping after 20 rounds of no improvement.\nClass imbalance was addressed by setting scale_pos_weight to the negative-to-positive ratio. The\nfinal model was selected based on AUPRC performance during cross-validation, with optimal\nparameters retrained on the complete 80% training dataset. Final model performance was\nevaluated on 10% internal test set and an independent external validation set of FTC133 cell line.\nWe employed two methods for precision calculation. The first method treated all unknown samples\nas negative instances (0), and precision was calculated as TP / (TP + FP), where TP denotes true\npositives and FP denotes false positives. However, this approach may underestimate the true\nprecision, since unknown samples are likely to contain some true positives. Therefore, we also\napplied a weighted precision method, which assigns the predicted probability of each unknown\nsample as its positive-class weight. These weighted precisions were combined with labeled\npositive and negative samples, and all interactions were ranked by confidence to compute a\ncumulative precision curve.”\nWe also summarized the benchmarking and performance of the XGBoost model in the Results:\n“Model evaluation showed robust internal performance with cross-validation AUROC of 0.81 (Fig.\n3A) and AUPRC of 0.82 (Fig. 3B). Hyperparameter optimization in the internal validation set\nyielded an AUROC of 0.77 (Fig. 3C) and an AUPRC of 0.78 (Fig. 3D). Hold-out test results\nachieved an AUROC of 0.78 (Fig. 3E) and an AUPRC of 0.79 (Fig. 3F). External validation on the\nFTC133 dataset at the native sample ratio yielded an AUROC of 0.68 (sFig. 2G). The reduced\nAUROC during external validation reflects both technical differences across cell lines and the\ninherent challenge of class imbalance in native proteomic data. For high-confidence predictions\nusing a strict probability threshold of 0.95, weighted precision achieved 0.5 in the FTC133 dataset\n(Fig. 3G), which contained 6558 labeled interactions out of 18,811 interactions (precision > 0.5)\nfrom integrated databases (Fig. 3H). The three-cell consensus set of 25,173 interactions exhibited\ncomparable performance with a probability higher than 0.95 and a weighted precision over 0.6\n(sFig. 2E), including 1197 interactions that were previously documented in CORUM (sFig. 2F).”\nComplex identification: It's not clear to me if the complexes presented in Figures 5 and 6 are the\nonly complexes identified, as this is a very low number that does not seem to include major\nabundant protein complexes. Compared to other protein complex identification papers from CF-\nMS, this is a very low number of identified complexes.\nReply: We appreciate the reviewer's careful evaluation of our complex identification results. We\nwould like to clarify the key points regarding Figures 5-6 and the overall complex detection:\n1) In the previous manuscript, Figure 5 and Figure 6 specifically highlight differentially expressed\nprotein interactions between FTC238 vs Nthy-ori 3-1 (Figure 5) and TPC-1 vs Nthy-ori 3-1\n(Figure 6), rather than representing the complete complexes.\n2) The relatively low number of Figure 5 and Figure 6 is caused by employing the criteria in the\nprevious manuscript:\n“To enhance the reliability of identified interacting proteins, we established a linear relationship\n9\nbetween retention time and molecular weight using a standard protein mix. Proteins with\nmolecular weights smaller than those listed on UniProt were excluded. Only protein pairs found in\nat least five biological replicates across two cell lines were retained.”\n3) Figure 4A presents all interactions identified at 20% FDR across three cell lines. The relatively\nlower number compared to other CF-MS studies might stem from the fact that we used the default\nsettings of PrInCE, which employs a Naive Bayes classifier by default, resulting in suboptimal\nprediction performance.\nWe totally agree that the previous method identified a limited number of complexes. Therefore,\nwe constructed an XGBoost-based machine learning model to predict PPIs:\nWe identified over 20,000 interactions using the XGBoost model across three thyroid cell lines, as\ndescribed in the “A machine learning model for protein interaction prediction” of the Results:\n“The three-cell consensus set of 25,173 interactions exhibited comparable performance with a\nprobability higher than 0.95 and a weighted precision over 0.6 (sFig. 2E), including 1197\ninteractions that were previously documented in CORUM (sFig. 2F).”\nThe analysis of major complexes was conducted via the following approach.\nFirst, we enriched the biological functions from all interactions across the three cell lines. Our\nanalysis specifically highlighted the top ten KEGG pathways. The added text in “Protein\ninteraction landscape of thyroid cell lines” of the Results as follows:\n“The top ten significant KEGG pathways included ribosome, proteasome, biosynthesis of amino\nacids, DNA replication and cysteine and methionine metabolism (Fig. 4A). Ribosome and\nproteasome complexes as the top two pathways align with mass spectrometry-based PPI studies\n(PMID: 33961781, 26186194), further validating the reliability of ProteoAutoNet workflow. We\nprioritized pathways by combining the p-value and enrichment score. This approach better\ncaptures stable protein complexes, leading to focus on five significant protein interaction networks\n(PINs). The proteasome pathway ranked highest and followed by sulfur metabolism, DNA\nreplication, ribosome, and 2-oxocarboxylic acid metabolism.\nProteoAutoNet detected 37 out of 43 known proteasome components from the KEGG database\n(sFig. 3A), exhibiting the coverage of 20S core particle with its PSMA and PSMB families (PMID:\n12083007), and the 19S regulatory particle with its PSMC and PSMD families (PMID:\n19145068). We also detected partial components of regulatory factors including the PSME and\nPSMF families. The detected 20S-19S-regulatory factor architecture matches the known\nproteasome organization (PMID: 19145068, 24804812, 30700495). This methodological\nconvergence reinforces the validity of ProteoAutoNet for studying the proteasome complex. The\nsulfur metabolism (7 out of 10 known components) (sFig. 3B), DNA replication (22/36) (sFig.\n3C), ribosome pathway (72/131) (sFig. 3D, E) and 2-oxocarboxylic acid metabolism (10/18)\n(sFig. 3F) were detected based on KEGG database annotations. We constructed protein\ninteraction networks (PINs) for these pathways and performed Markov clustering (MCL) to\nidentify functional modules.”\nSecond, we focus on the conserved complexes by the using common interactions across three\nthyroid cell lines in the “Protein interaction landscape of thyroid cell lines” of the Results:\n“We displayed common PINs from three cell lines and highlighted edges with experimental\nevidence in STRING (sFig. 3G). We identified three functional modules with extensive overlap\n10\nwith the STRING: translation (Fig. 4B), proteasome-ubiquitin (Fig. 4C) and protein folding (Fig.\n4D). As the module with highest overlap of prior knowledge, the detection of 40S/55S/60S\nribosomal subunits and eukaryotic initiation factor 3 (eIF3) complex across all three cell lines\nvalidates the reliability of ProteoAutoNet results (Fig. 4B). Besides, the detection of proteasome\ncorroborates preclinical evidence for proteasome inhibitors in medullary thyroid cancer (MTC)\n(PMID: 37152939). The Prefoldin and chaperonin-containing TCP-1 (CCT) complexes\nparticipate in polypeptide folding, consistent with their canonical roles in eukaryotic protein\nfolding pathways (PMID: 35111762). Some subunits of these complexes have been reported as\nbiomarkers in gastric and lung cancer (PMID: 31957800, 27694898).”\nOur revised analysis identifies a comprehensive set of over 20,000 interactions that include major\nabundant complexes (e.g. ribosome, proteasome), effectively addressing the initial limitation.\nComplex modelling: Alphafold predicted protein complex images for PFAS/HK1/TGM2 are not\nclearly presented, and it is difficult to see where the predicted interfaces are, as both subunits are\nthe same colors. Is it possible to combine these three individual alphafold predictions into a single\nprediction of the 3-protein complex? If there are major steric clashes between the pairwise subunit\npredictions when overlapped, it would decrease confidence that this is a true 3-protein complex.\nThese candidate novel interactions could be supported with use of the ipTM score or the IPSAE\nscore (PMID: 39990437).\nReply: We sincerely thank the reviewer for prompting this deeper structural analysis. Following\nthe reviewer’s suggestions on our gold standard, we completely excluded the iRefIndex. This\nremoval led to a re-analysis of our findings. The interaction between PFAS and HK1, which was\nsupported by iRefIndex, was consequently filtered out.\nTherefore, we focus on the binary interaction between TGM2 and HK1, which is supported by\nprob > 0.95 in XGBoost model and weighted precision > 0.6 in thyroid cell lines. It had confident\nscores from AlphaFold3: ipTM = 0.61, pTM = 0.66.\nIn direct response to the reviewer’s request, we employed the IPSAE score (PMID: 39990437) to\nprovide a rigorous assessment of TGM2-HK1 interface.\nThe structural analysis yields a high confident model quality (ipTM = 0.61, pTM = 0.66) but a low\ninterface-specific score (ipSAE = 0.112). We suppose the discrepancy is not a false negative but a\n11\nreflection of the unique structural biology of Hexokinase 1 (HK1). HK1 is a dynamic allosteric\nenzyme (PMID: 10686099). In the previous structural study, it exists primarily as a monomer in\nsolution and undergoes large ligand-dependent conformational changes and rigid-body rotations\nbetween its N and C-terminal halves. High ipTM/pTM scores indicate HK1 and TGM2 are well-\nfolded proteins. However, the low IPSAE score suggests HK1 does not expose the necessary\ninterface for stable binary interaction with TGM2. HK1 might be induced by post-translational\nmodifications or metabolites to form a specific and transient interaction.\nWe thank the reviewer for prompting these critical improvements of the structural explanation.\nIn Figure 6B, PFAS/HK1/TGM2 are shown to have an extra peak in Nthy around 18m, however,\nthis peak is not visible in the heatmap in figure 6A.\nReply: We sincerely appreciate the reviewer’s insightful observation. In the original submission,\nthe heatmap was generated using data from one representative biological replicate of each cell\nline, while the line plots displayed the mean values across all three biological replicates. We\nacknowledge that this inconsistency in data presentation could lead to potential misinterpretation.\nIn the revised manuscript we used the mean values in heatmap and protein traces:\nheatmap. It is presented in sFig. 9 and 10 of the revised manuscript.\nRecommendations:\n- Remove iRefIndex from the training set\nReply: Thanks for your suggestions. We have removed iRefIndex from the gold standard.\n- Provide a precision-recall curve for the predicted interactions, showing overlap with the curated\nprotein complex databases\nReply: Thanks for your suggestions. We have added the AUPRC and AUROC plots (Fig. 3) to the\nrevised computational methods section and have included an analysis of the overlap between\nCORUM and our predicted interactions (sFig. 2F). The added text in Results as follows:\n13\n“Model evaluation showed robust internal performance with cross-validation AUROC of 0.81 (Fig.\n3A) and AUPRC of 0.82 (Fig. 3B). Hyperparameter optimization in the internal validation set\nyielded an AUROC of 0.77 (Fig. 3C) and an AUPRC of 0.78 (Fig. 3D). Hold-out test results\nachieved an AUROC of 0.78 (Fig. 3E) and an AUPRC of 0.79 (Fig. 3F). External validation on the\nFTC133 dataset at the native sample ratio yielded an AUROC of 0.68 (sFig. 2G). The reduced\nAUROC during external validation reflects both technical differences across cell lines and the\ninherent challenge of class imbalance in native proteomic data. For high-confidence predictions\nusing a strict probability threshold of 0.95, weighted precision achieved 0.5 in the FTC133 dataset\n(Fig. 3G), which contained 6558 labeled interactions out of 18,811 interactions (precision > 0.5)\nfrom integrated databases (Fig. 3H). The three-cell consensus set of 25,173 interactions exhibited\ncomparable performance with a probability higher than 0.95 and a weighted precision over 0.6\n(sFig. 2E), including 1197 interactions that were previously documented in CORUM (sFig. 2F).\nNotably, our approach overcomes a critical limitation of existing co-elution proteins prediction\nthat require multi-dataset integration for reliable performance (PMID: 34211188). Previous\nsingle-dataset implementations showed limited predictive accuracy with AUROC of 0.65 as\nreported (PMID: 34211188). Through systematic integration of data augmentation with XGBoost\nalgorithm, we achieved a 21.5% improvement in single-dataset predictive accuracy, while\nmaintaining robust external validation performance at the native class imbalance ratio.”\n- Clarify the number of interactions and complexes identified\nReply: Thanks for your suggestions. We have clarified that we identified 25,173 interactions in\nthe Results:\n“For high-confidence predictions using a strict probability threshold of 0.95, weighted precision\nachieved 0.5 in the FTC133 dataset (Fig. 3G), which contained 6558 labeled interactions out of\n18,811 interactions (precision > 0.5) from integrated databases (Fig. 3H). The three-cell\nconsensus set of 25,173 interactions exhibited comparable performance with a probability higher\nthan 0.95 and a weighted precision over 0.6 (sFig. 2E), including 1197 interactions that were\npreviously documented in CORUM (sFig. 2F).”\n- Remove references to multi-database search strategy, as it is standard practice to use multiple\ndatabases to construct a gold-standard dataset of known interactions\nReply: Thanks for your suggestions. We have removed all “multi-database search strategy” in the\nmanuscript. The revised manuscript, now entitled “ProteoAutoNet: high-throughput co-eluted\nprotein analysis with robotics and machine learning”\n- Improve presentation of alphafold predicted protein complex images\nReply: Thanks for your suggestions. We added X-Y and X-Y-Z perspectives of interactions in\nmain figure (Fig. 5D) The added text reads in “Identification of novel interactions and potential\ntherapeutic targets” as follows:\n“The structure of the novel interactions was further supported by AlphaFold 3 (Fig. 5D). Two\nstructural perspectives (X-Y and X-Y-Z) are displayed. The model shows moderate-to-high\nconfidence, with pTM and ipTM scores of 0.66 and 0.61, respectively. It indicates a valuable\npredicted interaction interface, providing strong structural support for the robustness of this\ninteraction.”\n14\n-Emphasize novelties in the computational strategy compared to past CF-MS papers.\nReply: Thanks for your suggestions. In revised manuscript, the computational part of\nProteoAutoNet overcomes a critical limitation in current co-elution protein prediction that requires\nmulti-dataset integration for reliable performance (PMID: 34211188).\nWe have clarified the advances and compared the performance of machine learning model in the\nDiscussion, and we have built the first XGBoost model in PPI studies:\n“The computational component of ProteoAutoNet addresses a fundamental challenge in co-\nelution-based protein prediction: the inherent scarcity and limited diversity of positive training\nsamples9. By combining data augmentation with the XGBoost algorithm, we achieved a 21.5%\nimprovement in single-dataset predictive accuracy, attaining an AUROC of 0.78 in internal testing\ndataset, while maintaining external validation performance (AUROC = 0.68). The XGBoost-based\nmachine learning model in ProteoAutoNet showed consistent predictive accuracy, with precision\nexceeding 0.6 for 25,173 interactions across three thyroid cell line, and predicted 18,811 protein\ninteractions in the independent FTC133 validation set.”\nWhile the robotics approach to sample preparation is innovative and does shorten the experimental\ntime, the computational analysis relies on existing tools and employs a questionable strategy for\ndefining true interactions. The major issue is the use of a large number of PPIs from the iRefIndex\ndatabase, which is an extreme outlier from other protein complex databases. With improvements\nto the computational methodology and more rigorous validation of predictions, this work could\nmake a more substantial contribution to the field.\nReply: We thank the reviewer for the comprehensive assessment of our computational approach.\nWe fully agree that we should remove the iRefIndex and integrated other three databases as gold\nstandard.\nAs detailed above, we have completely overhauled our computational methodology: (1) removing\niRefIndex; (2) combining CORUM, hu.MAP and Complex Portal as gold standard; (3) employing\na XGBoost model with data augmentation. With the improvements to our computational\nmethodology, including ipTM from AlphaFold 3 and IPSAE score, we are confident the revised\nmanuscript makes a more reliable contribution to the field.\nLyu et al. present ProteoAutoNet, a workflow combining robotics-assisted CF-MS sample\nprocessing and a multi-database search strategy. The workflow is applied to three thyroid cell lines\nto identify cancer specific protein-protein interactions. While the automated workflow provides\nclear benefits with respect to throughput and experimental reproducibility, the computational part\nof ProteoAutoNet is insufficiently described and benchmarked.\nMajor points:\n1) The computational method of ProteoAutoNet for integrating multiple database search results is\nonly very briefly and superficially described in the methods section. The authors should provide\nmore detailed information and a performance benchmark. It is not clear how ProteoAutoNet\nrelates to the findings presented in Figure 3.\nReply: We sincerely appreciate the reviewer's insightful comments regarding the computational\nmethodology of ProteoAutoNet. Below we provide a comprehensive description of the revised\nand original computational workflow:\nThe revised computational part in “Machine learning framework for predicting PPIs” of Materials\nand Methods is:\n“Protein interactions were derived from the set of identified proteins (N), yielding a total of N ×\n(N - 1) / 2 possible interactions. However, the set of known true interactions from gold-standard\ndatabases is sparse relative to this theoretical maximum.\nWe implemented a two-phase strategy to build a robust predictive model from these sparse\nannotations. In the initial phase, sensitivity-optimized downsampling (1:5 negative-to-positive\nratio) was performed across all three cell lines (TPC-1, FTC238, and Nthy-ori 3-1) using the\nPrInCE R package (v1.16.0, R4.3.0). Protein interactions were annotated against integrated\ndatabase from CORUM, Complex Portal and hu.MAP, with random forest classification (5-fold\nCV) identifying high-confidence interactions (score > 0.5) for data augmentation. The second\nphase involved two-track data augmentation of high-confidence interactions: (1) random value\nperturbation, where protein elution profile values were scaled by a random factor between 90%\nand 110%; and (2) missing-value perturbation, where the positions of missing values were\nrandomly either retained in place or shifted to an adjacent time point. This augmentation process\nwas implemented in Python 3.9.21. Representative augmented pairs were selected based on a\ncomposite score: score = (n_valid × 0.5) + (|Pearson_r| × 30) − (euc_dist × 0.01), from which\nthe top 40 and bottom 10 scoring pairs were retained. Final training sets combined original and\naugmented positives and matched negatives (1:1 ratio). The XGBoost classifier (Python 3.9.21)\nwas trained using stratified partitioning via GroupShuffleSplit by protein ID to maintain group\nintegrity (80% training, 10% validation, and 10% test sets). Five interaction features were\nemployed: smoothed and raw Pearson correlation coefficients, Euclidean distance, co-peak\nfrequency, and weighted cross-correlation (WCC) scores. Hyperparameter optimization was\nconducted through three-fold stratified grid search covering max_depth (4-8), learning_rate\n(0.01-0.1), subsample ratio (0.6-1.0), with early stopping after 20 rounds of no improvement.\nClass imbalance was addressed by setting scale_pos_weight to the negative-to-positive ratio. The\nfinal model was selected based on AUPRC performance during cross-validation, with optimal\nparameters retrained on the complete 80% training dataset. Final model performance was\n16\nevaluated on 10% internal test set and an independent external validation set of FTC133 cell line\nWe employed two methods for precision calculation. The first method treated all unknown samples\nas negative instances (0), and precision was calculated as TP / (TP + FP), where TP denotes true\npositives and FP denotes false positives. However, this approach may underestimate the true\nprecision, since unknown samples are likely to contain some true positives. Therefore, we also\napplied a weighted precision method, which assigns the predicted probability of each unknown\nsample as its positive-class weight. These weighted precisions were combined with labeled\npositive and negative samples, and all interactions were ranked by confidence to compute a\ncumulative precision curve.”\nWe used the area under the ROC curve (AUROC) and the area under the precision-recall curve\n(AUPRC) as benchmarks to evaluate the machine learning model in the Results:\n“Model evaluation showed robust internal performance with cross-validation AUROC of 0.81 (Fig.\n3A) and AUPRC of 0.82 (Fig. 3B). Hyperparameter optimization in the internal validation set\nyielded an AUROC of 0.77 (Fig. 3C) and an AUPRC of 0.78 (Fig. 3D). Hold-out test results\nachieved an AUROC of 0.78 (Fig. 3E) and an AUPRC of 0.79 (Fig. 3F). External validation on the\nFTC133 dataset at the native sample ratio yielded an AUROC of 0.68 (sFig. 2G). The reduced\nAUROC during external validation reflects both technical differences across cell lines and the\ninherent challenge of class imbalance in native proteomic data. For high-confidence predictions\nusing a strict probability threshold of 0.95, weighted precision achieved 0.5 in the FTC133 dataset\n(Fig. 3G), which contained 6558 labeled interactions out of 18,811 interactions (precision > 0.5)\nfrom integrated databases (Fig. 3H). The three-cell consensus set of 25,173 interactions exhibited\ncomparable performance with a probability higher than 0.95 and a weighted precision over 0.6\n(sFig. 2E), including 1197 interactions that were previously documented in CORUM (sFig. 2F).\nNotably, our approach overcomes a critical limitation of existing co-elution proteins prediction\nthat require multi-dataset integration for reliable performance (PMID: 34211188). Previous\nsingle-dataset implementations showed limited predictive accuracy with AUROC of 0.65 as\nreported (PMID: 34211188). Through systematic integration of data augmentation with XGBoost\nalgorithm, we achieved a 21.5% improvement in single-dataset predictive accuracy, while\nmaintaining robust external validation performance at the native class imbalance ratio.”\n2) If protein pairs rather than protein complexes are of interest, also alternative strategies, such as\nSECAT or PCprophet could be evaluated.\nReply: We thank the reviewer for suggesting alternative analytical tools such as SECAT or\nPCprophet for protein pair analysis. SECAT and PCprophet are indeed powerful methods for\nidentifying protein complexes and tracking their dynamics across conditions. While tools like\nSECAT and PCprophet excel at complex-level analysis, our goal was to develop a robust machine\nlearning model optimized for pairwise PPI prediction under a single dataset. We evaluated our\nmodel's performance against the benchmark set by previous machine learning efforts in the field.\nThe added text is in the “A machine learning model for protein interaction prediction” of Results:\n“Notably, our approach overcomes a critical limitation of existing co-elution proteins prediction\nthat require multi-dataset integration for reliable performance (PMID: 34211188). Previous\nsingle-dataset implementations showed limited predictive accuracy with AUROC of 0.65 as\nreported (PMID: 34211188). Through systematic integration of data augmentation with XGBoost\nalgorithm, we achieved a 21.5% improvement in single-dataset predictive accuracy, while\nmaintaining robust\n17\nexternal validation performance at the native class imbalance ratio.”\nWe believe our computational workflow provides a valuable and complementary approach to the\nexisting toolkit.\n3) Conceptually, I would generally advise to combine databases prior to CCprofiler or PrInCE\nanalysis (rather than post-processing combination), which will keep statistical models of these\ntools intact. Have the authors tested this approach?\nReply: We sincerely appreciate this valuable methodological suggestion.\n1) We removed iRefIndex and combined CORUM, huMAP, Complex Portal as gold standard:\nOur previous benchmarking using a Naive Bayes model in PrInCE R package on the TPC-1 cell\nline. The precision-recall curve is shown below, yielding an AUROC of 0.026.\nWe speculated the poor performance was due to both the limitations of the Naive Bayes and the\noverlooking of the class imbalance in the native dataset.\n2) We finally used an improved workflow:\nKey enhancements of the new workflow over the previous version included: removal of the\niRefIndex database and development of an XGBoost model enhanced with data augmentation. By\nintroducing random perturbations to the known true PPIs, this approach not only rebalanced the\npositive and negative samples but also effectively addressed their scarcity and limited diversity of\npositive samples.\nThe brief workflow of the XGBoost modeling is presented in the “A machine learning model for\nprotein interaction prediction” of Results:\n“We built an XGBoost model for co-eluted protein prediction and evaluated its performance using\nthe area under the receiver operating characteristic curve (AUROC) and the area under the\nprecision-recall curve (AUPRC).\nIn the initial phase, the TPC-1 cell line contained 27,794 positive interactions. The remaining\ninteractions were annotated as either negative (6,139,613) or unlabeled (8,548,311). FTC238\nshowed comparable positive interactions at 27,615 but exhibited 6,255,162 negative and\n8,723,199 unlabeled interactions. Notably, Nthy-ori 3-1 displayed 31,076 positive, 8,397,238\nnegative and 12,926,198 unlabeled interactions. The greater interaction counts in Nthy-ori 3-1\ndirectly reflect its larger proteome coverage (N), since the potential number of pairs scales with N\n²/2 (sFig. 2A). We evaluated different downsampling ratios and determined that a 1:5 negative-to-\npositive ratio (precision = 0.28, recall = 0.73) offered the optimal balance, minimizing the\nexcessive false negatives observed at higher ratios. The 1:5 ratio achieved stable precision in\nrandom forest models, as confirmed by five-fold cross-validation (sFig. 2B). Density plots verified\n18\neffective enrichment of high-confidence interactions (PrInCE score > 0.5) under this ratio (sFig.\n2C). We selected 20,635 co-eluted proteins with positive labels from TPC-1, 20,373 from FTC238\nand 22,947 from Nthy-ori 3-1.\nTo address the scarcity and limited diversity of known protein-protein interactions (PPIs), we\nemployed data augmentation by introducing random perturbations to positive samples. The\nsecond phase involved data augmentation through random value perturbation and missing-value\nperturbation as described previously (sFig. 2D). The XGBoost model was trained using five key\nfeatures on group-stratified splits of 44.7 million co-elution traces from three cell lines: TPC-1\nwith 14.7 million traces (14,715,718), FTC238 with 15.0 million (15,005,976), and Nthy-ori 3-1\nwith 14.9 million (14,969,894). The final 1:1 balanced dataset was divided into training,\nvalidation and test sets, with additional FTC133 data for external validation.”\nThe comprehensive steps of modeling are described in the Materials and Methods:\n“Protein interactions were derived from the set of identified proteins (N), yielding a total of N ×\n(N - 1) / 2 possible interactions. However, the set of known true interactions from gold-standard\ndatabases is sparse relative to this theoretical maximum.\nWe implemented a two-phase strategy to build a robust predictive model from these sparse\nannotations. In the initial phase, sensitivity-optimized downsampling (1:5 negative-to-positive\nratio) was performed across all three cell lines (TPC-1, FTC238, and Nthy-ori 3-1) using the\nPrInCE R package (v1.16.0, R4.3.0). Protein interactions were annotated against integrated\ndatabase from CORUM, Complex Portal and hu.MAP, with random forest classification (5-fold\nCV) identifying high-confidence interactions (score > 0.5) for data augmentation. The second\nphase involved two-track data augmentation of high-confidence interactions: (1) random value\nperturbation, where protein elution profile values were scaled by a random factor between 90%\nand 110%; and (2) missing-value perturbation, where the positions of missing values were\nrandomly either retained in place or shifted to an adjacent time point. This augmentation process\nwas implemented in Python 3.9.21. Representative augmented pairs were selected based on a\ncomposite score: score = (n_valid × 0.5) + (|Pearson_r| × 30) − (euc_dist × 0.01), from which\nthe top 40 and bottom 10 scoring pairs were retained. Final training sets combined original and\naugmented positives and matched negatives (1:1 ratio). The XGBoost classifier (Python 3.9.21)\nwas trained using stratified partitioning via GroupShuffleSplit by protein ID to maintain group\nintegrity (80% training, 10% validation, and 10% test sets). Five interaction features were\nemployed: smoothed and raw Pearson correlation coefficients, Euclidean distance, co-peak\nfrequency, and weighted cross-correlation (WCC) scores. Hyperparameter optimization was\nconducted through three-fold stratified grid search covering max_depth (4-8), learning_rate\n(0.01-0.1), subsample ratio (0.6-1.0), with early stopping after 20 rounds of no improvement.\nClass imbalance was addressed by setting scale_pos_weight to the negative-to-positive ratio. The\nfinal model was selected based on AUPRC performance during cross-validation, with optimal\nparameters retrained on the complete 80% training dataset. Final model performance was\nevaluated on 10% internal test set and an independent external validation set of FTC133 cell line\nWe employed two methods for precision calculation. The first method treated all unknown samples\nas negative instances (0), and precision was calculated as TP / (TP + FP), where TP denotes true\npositives and FP denotes false positives. However, this approach may underestimate the true\n19\nprecision, since unknown samples are likely to contain some true positives. Therefore, we also\napplied a weighted precision method, which assigns the predicted probability of each unknown\nsample as its positive-class weight. These weighted precisions were combined with labeled\npositive and negative samples, and all interactions were ranked by confidence to compute a\ncumulative precision curve.”\nWe used the area under the ROC curve (AUROC) and the area under the precision-recall curve\n(AUPRC) as performance benchmarks to evaluate the machine learning model in the Results:\n“Model evaluation showed robust internal performance with cross-validation AUROC of 0.81 (Fig.\n3A) and AUPRC of 0.82 (Fig. 3B). Hyperparameter optimization in the internal validation set\nyielded an AUROC of 0.77 (Fig. 3C) and an AUPRC of 0.78 (Fig. 3D). Hold-out test results\nachieved an AUROC of 0.78 (Fig. 3E) and an AUPRC of 0.79 (Fig. 3F). External validation on the\nFTC133 dataset at the native sample ratio yielded an AUROC of 0.68 (sFig. 2G). The reduced\nAUROC during external validation reflects both technical differences across cell lines and the\ninherent challenge of class imbalance in native proteomic data. For high-confidence predictions\nusing a strict probability threshold of 0.95, weighted precision achieved 0.5 in the FTC133 dataset\n(Fig. 3G), which contained 6558 labeled interactions out of 18,811 interactions (precision > 0.5)\nfrom integrated databases (Fig. 3H). The three-cell consensus set of 25,173 interactions exhibited\ncomparable performance with a probability higher than 0.95 and a weighted precision over 0.6\n(sFig. 2E), including 1197 interactions that were previously documented in CORUM (sFig. 2F).\nNotably, our approach overcomes a critical limitation of existing co-elution proteins prediction\nthat require multi-dataset integration for reliable performance (PMID: 34211188). Previous\nsingle-dataset implementations showed limited predictive accuracy with AUROC of 0.65 as\nreported (PMID: 34211188). Through systematic integration of data augmentation with XGBoost\nalgorithm, we achieved a 21.5% improvement in single-dataset predictive accuracy, while\nmaintaining robust external validation performance at the native class imbalance ratio.”\n4) While the reported results in the cancer cell lines are certainly interesting, it is not clear whether\nthese were only possible because of the adjusted analysis approach in ProteoAutoNet or whether a\ntraditional CCprofiler or PrInCE analysis would also report these findings.\nReply: We appreciate the reviewer's insightful question regarding the contribution of our\nimproved analytical approach. The identified interactions were uniquely recovered by our\nProteoAutoNet workflow and could not have been obtained using a traditional CCprofiler or\nPrInCE analysis on our dataset. As shown above, the Naive Bayes modules of PrInCE achieved an\nAUROC of only 0.026, preventing reliable pipeline execution. Therefore, we benchmarked the\nmajor complexes from over 20,000 high-confidence interactions against published AP-MS/CF-MS\ndata, as described above.\nThe added text in “Protein interaction landscape of thyroid cell lines” of the Results as follows:\n“The top ten significant KEGG pathways included ribosome, proteasome, biosynthesis of amino\nacids, DNA replication and cysteine and methionine metabolism (Fig. 4A). Ribosome and\nproteasome complexes as the top two pathways align with mass spectrometry-based PPI studies\n(PMID: 33961781, 26186194), further validating the reliability of ProteoAutoNet workflow. We\nprioritized pathways by combining the p-value and enrichment score. This approach better\ncaptures stable protein complexes, leading to focus on five significant protein interaction networks\n20\n(PINs). The proteasome pathway ranked highest and followed by sulfur metabolism, DNA\nreplication, ribosome, and 2-oxocarboxylic acid metabolism.\nProteoAutoNet detected 37 out of 43 known proteasome components from the KEGG database\n(sFig. 3A), exhibiting the coverage of 20S core particle with its PSMA and PSMB families (PMID:\n12083007), and the 19S regulatory particle with its PSMC and PSMD families (PMID:\n19145068). We also detected partial components of regulatory factors including the PSME and\nPSMF families. The detected 20S-19S-regulatory factor architecture matches the known\nproteasome organization (PMID: 19145068, 24804812, 30700495). This methodological\nconvergence reinforces the validity of ProteoAutoNet for studying the proteasome complex. The\nsulfur metabolism (7 out of 10 known components) (sFig. 3B), DNA replication (22/36) (sFig.\n3C), ribosome pathway (72/131) (sFig. 3D, E) and 2-oxocarboxylic acid metabolism (10/18)\n(sFig. 3F) were detected based on KEGG database annotations. We constructed protein\ninteraction networks (PINs) for these pathways and performed Markov clustering (MCL) to\nidentify functional modules.”\nSecond, we focus on the conserved complexes by the using common interactions across three\nthyroid cell lines in the “Protein interaction landscape of thyroid cell lines” of the Results:\n“We displayed common PINs from three cell lines and highlighted edges with experimental\nevidence in STRING (sFig. 3G). We identified three functional modules with extensive overlap\nwith the STRING: translation (Fig. 4B), proteasome-ubiquitin (Fig. 4C) and protein folding (Fig.\n4D). As the module with highest overlap of prior knowledge, the detection of 40S/55S/60S\nribosomal subunits and eukaryotic initiation factor 3 (eIF3) complex across all three cell lines\nvalidates the reliability of ProteoAutoNet results (Fig. 4B). Besides, the detection of proteasome\ncorroborates preclinical evidence for proteasome inhibitors in medullary thyroid cancer (MTC)\n(PMID: 37152939). The Prefoldin and chaperonin-containing TCP-1 (CCT) complexes\nparticipate in polypeptide folding, consistent with their canonical roles in eukaryotic protein\nfolding pathways (PMID: 35111762). Some subunits of these complexes have been reported as\nbiomarkers in gastric and lung cancer (PMID: 31957800, 27694898).”\nMinor points:\n5) Line 61 on page 2: check sentence structure “the performance of the in”\nReply: Thanks for your suggestions, we have revised the manuscript.\n21"}, {"round": 2, "reviewer_comments": {"reviewer_1": "Authors have addressed the main concerns, the revised paper is now suitable for publication.", "reviewer_4": "I was asked to evaluate the responses to Reviewers 2 and 3. These appear to be very thorough and well-thought-out responses. I also looked in detail at a few random claims made by the authors and these also checked out"}, "author_response": ""}]

Decision letter