BACKGROUND
the emergence of high-performance protein-extraction procedures , brute-force spectra-capture methods , and improved multiplexing technologies  <cit>  has transformed proteomics  from a relatively low-throughput technology to one with critical practical applications in biology.
the application of proteomics on clinical samples  is concerned with unraveling proteome changes associated with disease using actual clinical samples.
typically, two classes of samples---e.g., normal  and disease ---are compared against each other.
proteins exhibiting strong inter-class differences are marked as differential and analyzed for relevant functional roles.
statistics provides powerful means for differential protein selection based on the hypothesis-testing framework.
this process is commonly referred to as “feature selection” .
unfortunately, despite increasing ease in data generation, extracting knowledge from proteomics expression data is difficult  <cit> .
proper feature selection, if done correctly, should lead directly to drug-target and biomarker identification; but in practice, this is seldom the case  <cit> .
in theory, a strongly differential feature  should exhibit strong inter-class differences across samples.
however, real samples are intrinsically noisy.
this intrinsic noise is random  and obfuscates proper feature selection by masking true inter-class differences.
the manner in which the samples are prepared contributes towards a second type of variation, which unlike intrinsic random noise, is non-random  and not associated with class effects; i.e., they do not distinguish sample classes d and d* specifically.
this second source of variation, where features are more strongly correlated with technical factors  than with sample classes  , is referred to as batch effects.
it is not straightforward to distinguish batch and class effects: when the former is mild, it may lead to bias during feature selection; but when strong, lead to downright selection of irrelevant proteins that confound and mislead  and/or the loss of truly relevant proteins .
in other words, batch effects obfuscate analysis.
batch effects are known to be present in genomics assays .
however, they are a nuisance in proteomics assays, where multiplexing limits impose constraints on the number of samples for concurrent analysis; e.g., analyzing eight samples with the commonly used 4-plex itraq labeling system requires at least two separate experiments performed at different times, or on different instruments.
despite fairly recent work demonstrating that batch-effect correction may lead to substantial increase in feature-selection sensitivity  <cit> , a systematic exploration of batch effects in proteomics data, and proposal of feasible workarounds, is missing.
reasons include underestimating heterogeneity in practical usage , unsuitable data , and the erroneous belief that normalization eradicates batch effects.
normalization is a data processing technique that adjusts global properties of measurements for individual samples for appropriate comparisons.
examples include z- and quantile-normalization, and mean-scaling.
however, normalization cannot eradicate batch effects, as the latter does not affect all variables similarly  <cit> .
in cases where statistical assumptions are violated, normalization may affect data integrity instead.
batch effects are usually detected via principal component analysis , where the first two or three principal components  are plotted for each sample colored by the batch labels, and separation of colors taken as evidence of batch effects  <cit> .
when batch effects are dominant, the first n pcs are expected to be dominated by batch effects, and removal of these pcs may be an alternative yet effective means of batch-effect correction.
the remaining pcs---though these have lower contribution towards overall variation---may be dominated by small subsets of variables with good class-discrimination power  <cit> .
thus, feature selection at the level of pcs---i.e., using pcs, as opposed to proteins, as features---may be a viable batch effect-resistant feature-selection strategy.
protein complex-based analysis, as a new analytical paradigm, provides a powerful yet stable means of selecting features, at the level of protein complexes, from proteomics data .
protein complexes are strongly enriched for biological coherence signal  <cit> , beating any combinations of alternative measurements  using protein complex-based analysis, we have successfully recovered missing proteins  <cit>  and overcome consistency issues where patient samples present widely different protein sets .
protein complex-based analysis also exhibits unparalleled stability and reproducibility in feature selection  <cit> .
we hypothesize that this superior performance may stem in part, from innate resistance to batch effects.
we address the following gaps in batch effects, and its implications for feature selection in a proteomics setting.
first, we propose a simple technique for simulating batch effects in proteomics data, and recommend using it for evaluating feature-selection procedures, as well as checking whether batch effect-correction algorithms are working as intended.
second, while pca is the de facto approach for visualizing presence of batch effects, we investigate its feasibility as a feature-selection technique where features are principal components  instead of proteins.
and finally, as a potential new advantage , we check whether protein complex-based feature-selection algorithms are truly resistant to batch effects; and if so, whether they may supersede the need for batch effect-correction algorithms.
methods
simulated data --- d <dig>  and d <dig> h
we used part of the d <dig>  dataset  from the study of langley and mayr as a reference proteomics simulation dataset where differential variables are known a priori  <cit>  .
quantitation is based on spectral counts.
class effects and batch effects are inserted randomly, with the increase made in d* samples only.
simulated data with both class and batch effects inserted is referred to as d <dig> h, while the original data with only class effects is referred to as d <dig>  .
real data --- renal cancer
the renal cancer  study of guo et al.
<cit>  comprises a total of  <dig> swath runs originating from six pairs of non-tumorous and tumorous clear-cell renal cell carcinoma  tissues, in two batches, rep <dig> and rep <dig> .
batch effect-correction methods
for batch-effect correction, we used quantile normalization and linear-scaling as generic approaches .
quantile normalization and linear-scaling are not explicitly batch effect-correction methods.
so, we also used combat on d <dig> h to remove batch effects, and evaluate performance recovery against the original d <dig>  .
combat is a well-known batch effect-correction approach and employs empirical bayes frameworks for adjusting data for batch effects.
it is reported to be robust to outliers in small sample sizes  while maintaining comparable performance to existing methods for large samples  <cit> .
statistical feature-selection methods
four classes of feature-selection methods are tested to see whether they are robust against batch effects .
the standard single-protein t-test   <cit>  and hypergeometric enrichment   <cit>  test are the most commonly used comparative analysis methods.
we have also included two variants of rank-based network algorithms ---viz.
subnets   <cit>  and fuzzy-subnets   <cit> ---which were demonstrated to be highly stable and reliable  <cit>  .
on real data, he, snet and fsnet are tested using corum complexes  <cit>  as their protein complex-based feature vector  <cit> .
the performance of these feature-selection methods are evaluated on precision and recall .
on simulated data , these same methods are evaluated based on simulated complexes .
in simulated data, the differential proteins are known a priori.
we use these to create true-positive pseudo-complexes.
to achieve this, a euclidean distance is first determined for all differential protein pairs across all samples.
these are then clustered via ward’s linkage.
differential proteins are reordered such that those with similar expression pattern  are adjacent to each other.
this reordered list is then split at regular intervals to generate  <dig> true-positive pseudo-complexes.
an equal number of non-significant proteins is randomly selected, reordered based on expressional correlation, and then split to generate an equal number of true-negative pseudo-complexes.
we may alter the “purity” of the true-positive pseudo-complexes by reducing the proportion of differential proteins within them.
in practice, we seldom observe all complex members being differentially expressed simultaneously .
purity, therefore, is the proportion of differential proteins within each true-positive pseudo-complex.
at 100% purity, simulated complexes are comprised solely of significant proteins; at 75% purity, 25% of the constituent significant proteins are randomly replaced with non-significant ones; and so on.
reducing purity permits evaluation of the robustness and sensitivity of the complex-based analysis methods.
purity is tested at three levels:  <dig>   <dig> and 50%.
the true-positive and true-negative pseudo-complexes are combined into a single vector.
evaluation is based on the f-score.
RESULTS
batch effects cannot be completely eradicated via batch effect-correction algorithms
our method simulates batch effects in the following manner : in the first dimension, class-effect sizes are inserted based on the method of langley and mayr to distinguish classes d and d*  <cit> .
class-effect sizes are sampled randomly from five possibilities and inserted into randomly selected variables on samples belonging to class d*, constituting 20% of all variables.
this is repeated to generate  <dig> random datasets .
in the second dimension, for each of the  <dig> datasets, batch effects are inserted over all variables.
here, batch effects are simulated by taking half the members in d and d* to be batch  <dig>  and the remaining half as batch  <dig>  batch effects are not evenly applied across all variables, and thus cannot be eradicated via simple normalization  <cit> .
to simulate batch effects unevenly, heterogeneity/batch-effect sizes, like class-effect sizes, are also sampled randomly from the same five possibilities.
however, we make the assumption that batch effects influence all variables in a sample, and thus expect batch effects to account for a majority of variation.
these  <dig> simulated datasets, d <dig> .301h to d <dig> .400h are available for download from additional file  <dig> fig.
<dig>
a simulations: from single-class data, samples are split randomly into two classes d and d*.
class effects are randomly inserted into proteins in d*.
this is followed by insertion of batch effects.
b pc manipulation: proteins are sorted based on variance followed by a cutoff.
the retained expression data is analyzed via principal component analysis .
the first principal component  corresponds strongly to batch effects and may be removed.
c manipulation of pcs for clustering and class prediction.
when combined with data of unknown labels, class labels can be predicted based on co-clustering with samples of known labels
to understand how batch effects affect feature selection, we compare precision, recall and f-score based on the standard t-test, with and without multiple-test correction based on the benjamini-hochberg false discovery rate  .
the first two columns in fig.
<dig> reveal that incorporation of batch effects increases the variability of performance metrics, with particular impact on recall, and reduces overall performance .
while we expect different feature-selection methods, aside from the t-test, may respond differently to batch effects, it is useful to incorporate heterogeneity/noise simulations during performance evaluation.fig.
<dig> batch effect-correction does not work optimally in practice.
top row: feature selection without multiple-test correction.
bottom row: feature selection with fdr correction.
first column shows base performance with only class effects .
second column shows performance with batch effects incorporated.
third, fourth and fifth columns show recovery using various batch effect-correction methods, .
feature-selection test used here is the two-sample t-test
the commonly used batch effect-correction method, combat  <cit> , only partially recovers original test performance .
therefore, it does not completely eradicate heterogeneity.
additionally, while it improves overall performance, it also tends to reduce precision, incorporating false positives into the selected feature set.
this is cause for concern during selection of features for experimental validation.
in the non-fdr corrected scenario, combat also does not perform better than conventional data normalization methods, e.g., quantile normalization and linear-scaling .
however, when test requirements are more stringent given the 5% fdr cutoff, then it is clear that combat provides considerable advantage .
in spite of the simplicity of these simulations, it is noteworthy that batch effect-removal  methods are not a panacea.
we cannot declare combat is inferior, but rather, we will never know if batch effects have been effectively removed from real data, particularly when the data happens not to fit combat’s assumptions well.
thus, a naïve reliance on batch effect-correction algorithms, without conducting further downstream checks for remnant batch effects , may potentially worsen analytical outcome.
a relook at principal component analysis for detecting and removing batch effects
principal component analysis  yields linear combinations of each variable’s contribution to variance, but evidently not all variables are equally interesting or relevant : we may say the features that changed the most, i.e., exhibited the most variation, are likely more impactful .
although we may not know this first-up for every feature, we can still reduce the feature set size via variance-based pre-selection.
here, a cutoff is introduced to include only the top 20% proteins  and used in pca .
as stated earlier, pca is commonly used for visualizing batch effects.
a simple way to do this is to label samples by classes  and batches , and diagrammatize these as scatterplots across the first  <dig> or  <dig> principal components .
in fig.
3a, the pcs are based on the top 20% proteins  and evidently, this is sufficient for detecting batch effects.
it is therefore unnecessary to use all variables .
however, it is unclear whether this is sufficient for detecting class effects.fig.
<dig>
a 3d-principal component scatterplot analysis: batch effects dominate in the simulated data.
shown are two examples d <dig> .301h and d <dig> .302h.
b pc-paired boxplots: each pc may be split by class and batch, and compared against each other as side-by-side boxplots.
this allows easy evaluation of the contribution of class and batch effects to each pc
scatterplots, being rough visual guides, do not reveal well the contributions of class and batch effects to each pc.
in our opinion, paired boxplots  are more informative and, here, it is evident that pc <dig> and pc <dig> correspond to batch and class effects respectively .
when a batch effect is observed, it is common practice to apply a batch effect-removal or –correction method .
however, this does not necessarily work well in practice.
moreover, if the data does not fit the correction method’s assumptions, it may lead to false positives.
instead, we may opt for a more direct strategy by simply removing the first pc , and deploying the remaining pcs as features for analysis.
when pc <dig> contributes strongly to batch effects, its removal should allow class effects to become the dominating source of variation .fig.
<dig>
a 3d-principal component analysis: class effects dominate in the simulated data, given the removal of pc <dig> .
b heatmaps and hierarchical clustering : the remaining pcs may also be used as individual variables for clustering, and provide strong discrimination between classes d and d*.
c combining two datasets with different batch effects: datasets a and b have the same differential feature set but different batch effects.
combining these followed by analysis of all principal components  shows batch effects dominate.
however, removal of pc <dig> perfectly recovers class-effect discrimination without having to perform any feature selection
using two examples , we show that removal of the first pc  allows samples to cluster based on classes rather than batch .
a caveat is that removal of pc <dig> works here primarily because it is strongly correlated with batch effects; i.e., batch effects account for the majority of variance in the data.
on real data, it may necessitate the removal of several other pcs that are correlated with batch effects.
moreover, if incompletely eradicated or inseparable from class effects, batch effects may resurface in subsequent pcs  during analysis .
suppose that removal of the first n pcs results in good class separation in pca, it may be possible to use the remaining pcs for feature selection and non-projection-based clustering techniques, e.g., hierarchical clustering and k-means.
this may seem counter-intuitive, as during standard analysis involving pca, it is common to keep just the top n pcs accounting for the majority of variation.
but not all variation is attributable towards class effects .
moreover, pcs with large same-sign coefficients tend to represent non-class effect properties correlated with the variables; e.g., tsuchiya et al.
demonstrated that, for their dataset, pc <dig> is linearly correlated with the magnitude of average gene expression  <cit> .
on the other hand, subsequent pcs with lower contribution towards overall variation may be dominated by small subsets of variables with good class-discrimination power  <cit> .
thus, instead of discarding the lower ranked pcs, it is more reasonable to remove the top pcs that are non-correlated with class effects.
we find that subsequent pcs do correlate strongly with sample classes d and d* , and may be used as variables for clustering .
a pc-based feature-selection approach is viable and allows relevant proteins to be retraced.
this is executed by first identifying the pc of interest, and selecting proteins that contribute exclusively and strongly to it  <cit> .
as a simple test, we look at two scenarios using d <dig> .301h and d <dig> .302h.
in the first scenario, proteins strongly associated with pc <dig>  are selected.
we term this “pc1” .
in the second scenario, we removed the original pc <dig>  and looked at proteins strongly associated with the new pc <dig> .
this is termed “–pc1” .
since the original pc <dig> is strongly associated with batch effects, its associated proteins are therefore less relevant, and we expect that performing feature selection on these using the t-test, and evaluating the corresponding precision, recall and f-score should fare worse than those proteins associated with the new pc <dig>  we find the results following pc <dig> removal are similar to batch effect-correction algorithms, with concomitant increase in recall following removal of the first principal component .
as a parallel test, we calculate a t-statistic for each protein under consideration, rank them from the highest to lowest, and keep only the top n proteins, where n = same number of proteins associated with –pc <dig> .
interestingly, these do worse , and moreover, have limited overlaps with the –pc <dig> selected features.table  <dig> effects on precision and recall for d <dig> .301h and d <dig> .302h before and following removal of proteins with heavy loadings on the first principal component
tt refers to the standard t-test while tt/-pc <dig> are the top n-ranked tt-features .
the jaccard coefficient indicates limited overlaps amongst the top tt-features associated with –pc <dig> features
this procedure---viz.
rank proteins by variance, perform pca using the top 20%, discard pcs that are strongly correlated with technical variables, and perform e.g., clustering using the remaining pcs---may be used for class prediction on new batches with unknown class labels.
a schematic is provided in fig.
1c.
and, to test this, two different sets of batch effects are inserted into d <dig> .
<dig>  the first is , and the second is  , and the data combined.
expectedly, clustering based on all pcs show that batch effects dominate.
however, removal of pc <dig> recovers perfect class discrimination.
this suggests that even when combining several datasets with different batch effects, removal of pc <dig> retains class effects, and permits class prediction .
moreover, if we have multiple datasets of the same disease, properly dealing with batch effects makes it possible to pool these datasets for analysis.
this is useful when larger sample sizes are needed for ad hoc analysis.
using variance-based variable pre-selection and principal component manipulation to tackle real batch effects
to evaluate how the procedure above is applicable towards real data, we consider the renal cancer study of guo et al., which contains two technical replicates  <cit> .
this data, rc, has been carefully processed; and batch effects appear contained .
we insert batch effects into rc rep <dig>  .
however, pc <dig> does not purely contain variability from batch effects but also some signal from class effects .
as with d <dig> h, pc projection also reveals dominance of batch effects  that may be controlled via removal of pc <dig> .
however, it appears some remnant batch effect still persists .fig.
<dig>
a hierarchical clustering  based on protein expression: batch effects dominate here, as samples separate more strongly based on batch than by class.
no variable selection was performed prior.
b principal components   <dig> and  <dig> stratified by class and batch: although pc <dig> is still strongly associated with batch; there is also some association with class effects.
in contrast, pc <dig> is strongly associated with class rather than with batch effects.
c hcl based on all pcs: in this case, it is clear that batch effects dominate.
d hcl following pc <dig> removal: removal of pc <dig> appears to eradicate a large proportion of the batch effects.
but this is incomplete, suggesting that there may still be ambient batch effects remaining
feature-selection methods with resistance to batch/heterogeneity effects
there are remnant batch effects in rc  that are difficult to eradicate, and may lead to bias during subsequent feature selection.
when batch effects are strong, then removal of the first few pcs is a useful direct strategy, especially if information on batch and other potential confounding factors are not known a priori  and batch effect-correction methods cannot be effectively deployed.
on the other hand, it is not always straightforward to interpret the pcs and extract the proteins relevant for class effects.
pc-based removal and batch-effect correction also may not be able to remove subtle batch effects.
as an additional note of caution, batch effect-removal approaches---including the procedure described above---may at times be overkill: these corrections may unintentionally eliminate true biological heterogeneity amongst samples , which is informative  and should not be discarded from the data in the first place.
unfortunately, batch-effect and subpopulations are not easy to tell apart  <cit> .
and if we run the pc-removal or other batch effect-correction methods, subpopulation information is irrevocably lost.
on the other hand, high heterogeneity in the form of multiple subpopulations can make analysis very challenging, particularly in cancer proteomics  <cit> .
one way forward is to incorporate robust data normalization methods and biological context  directly into feature-selection approaches  <cit> .
recently, we expound on the advantages of protein complexes as suitable biological context in improving data analysis.
unlike analysis at the level of proteins as features, the use of protein complexes as features, leads to improve stability and reproducibility  <cit> .
we are curious if the high performance of protein complex-based methods belonging to the family known as rank-based network analysis  exhibit superior performance  due to innate resistance to batch effects  <cit> .
there are several reasons why we think rbnas may be robust against batch effects: its score function uses rank-based discretization instead of exact values, which is robust against various biases, e.g., test-set bias  <cit> .
use of biological context  increases biological signal over signal from other spurious correlations  as only signal from same-complex members are summated.
we already know that use of protein complexes increases power, and we believe the signal amplification is phenotypically relevant  <cit> .
previous tests have already demonstrated that complex-based features are specifically predictive for phenotype classes and that false-positive rates are low.
however, a specific investigation into batch-effect resistance has not been done.
hence, we test two members: subnets   <cit>  and fuzzy-snet   <cit> .
to test whether rbnas are effective in overcoming batch effects, as opposed to simply eliminating them, we performed two sets of tests; the first on simulated data  as a proof-of-concept and the second on real data .
besides snet  <cit>  and fsnet  <cit>  , we also include the standard single-protein t-test   <cit>  and the hypergeometric enrichment test   <cit> .
sp is a control based on the standard univariate t-test at the level of individual proteins.
he is an over-representation-based technique meant to determine if the differential proteins are significantly enriched in some protein complex based on the hypergeometric test; i.e., it uses the same protein complexes, but not the same statistical test as the rbnas.
using d <dig>   and d <dig> h , we compare the f-scores across three purity levels , at both complex and individual protein levels .
we observe that the f-score for the protein-based t-test  is very low to begin with, but progressively worsens when batch effects are introduced.
the hypergeometric enrichment  pipeline uses the same pseudo-complexes as the rbnas, but also appears to be sensitive to batch effects .
the rbnas, snet and fsnet, are resistant to batch effects.
at the complex level, there is almost no difference in f-scores regardless of purity.
this suggests that the rbnas are robust not only against the batch effects, but also decreasing differential signal as purity decreases.
this finding is also corroborated at the level of individual proteins as well.fig.
<dig> complex-based methods are robust against batch effects : distribution of f-scores for sp, he, snet and fsnet in simulated data where batch effects are absent or present based on the simulated complexes  and individual proteins.
note that for sp, the f-scores shown are always based on individual proteins
our findings are a positive indication that the rbnas are highly robust against weaker differential signal and batch effects.
additionally, given he’s poor performance, we assert that use of complexes alone is insufficient; the statistical test setup is also critical.
this result is important as it is the first, to the best of our knowledge, to demonstrate that complex-based feature-selection approaches are resilient against batch effects.
however, there are caveats: firstly, these batch effects are simulated, and unreflective of true batch effects.
secondly, these pseudo-complexes may not be good approximations of true protein complexes.
although we cannot test real data directly , we may still evaluate these methods  on real data.
using the original rc , we perform pca on significant features selected by the four feature-selection methods.
for the first  <dig> pcs, side-by-side boxplots stratified by class and batch  demonstrates that rbnas  are very powerful, and appear to capture variation stemming only from class effects.
this is in contrast to sp, where batch effects still persist in pc <dig>  he is commonly used as a post-hoc test following sp.
it also takes advantage of the same set of protein complexes as snet and fsnet; but clearly, this is insufficient : the method used for statistical testing also matters.fig.
<dig> pc side-by-side boxplots stratified by class and batch tested on real data:, rank-based network algorithms snet and fsnet are against batch effects, and only seems to capture variation stemming from class effects
this finding is critical: since rbnas are robust against batch effects, this obviates the need for performing data transformations .
this also means that if subpopulations do exist in the data, this information is retained.
it should be noted that dealing with subpopulations is difficult and outside the scope of this work, although we may also use complexes for detailed in-depth study of subpopulations  <cit> .
while rbnas are evidently powerful against batch effects, especially against subtle ones that cannot be easily removed via removal of the first n pcs or via batch effect-correction algorithms, they are not perfect solutions.
e.g.
methods such as fsnet weigh each protein in a protein complex by the fraction of subjects  where the protein is highly ranked.
this fraction may be unstable from subsample to subsample, particularly in the presence of hidden subpopulations.
this may reduce class-specific signals, making it difficult to identify good-quality and relevant features.
downstream considerations post-analysis
discarding batch effect-laden pcs is a transformation that provides a transformed dataset with much reduced batch effects.
it helps identify strong class discriminatory features.
yet, at the same time, new batches are not directly comparable to this transformed dataset.
so it is difficult to extract directly clinical guideline/thresholds for future diagnostic use.
for example, as a simple diagnostic tool, one wants a protein x such that: if x’s abundance is above a threshold y, then the patient is sick.
but, in the presence of batch effects, different thresholds are needed for different batches.
on the other hand, once the good features are found, one may apply more reliable technologies---i.e., ones that are less susceptible to batch effects---to measure only those features specifically.
to test this empirically, the analysis in fig.
<dig> is repeated.
we plot the boxplots based on log-normalized expression of significant proteins for sp and he, and scores of significant subnets for snet and fsnet, for the top three features .
for sp and he, both class and batch variation are detected amongst the top three features.
so it is reasonable to conclude that where these approaches are concerned, it is difficult to identify class discriminatory signal.
the top three features selected by snet and fsnet  capture class effects better, and appear robust against batch effects.
thus, we expect that use of these features for diagnostics will yield better results.fig.
<dig> expression or network score-based  side-by-side boxplots stratified by class and batch for top three variables: rank-based network algorithms, snet and fsnet, can capture the class effects while robust against batch effects.
both class and batch variability is present in the top variables selected by sp and he
CONCLUSIONS
the impact of batch effects in proteomics cannot be understated, and has key implications in clinical and translational research.
we have shown that batch effect-correction algorithms are not a panacea, and that the corrected data may be erroneous.
moreover, with the development of any novel feature-selection approach, it is worthwhile to test their robustness against simulated batch effects.
we have illustrated that side-by-side barplots are better for visually detecting batch effects than the standard pca scatterplot-based representation format.
moreover, the pcs themselves may be used as features, which may also be effectively traced back to relevant differential proteins.
this is also a viable strategy complementary to batch effect-correction methods.
unfortunately, subtle batch effects cannot be easily removed or detected, and can lead to bias in the analysis of real data.
moreover, data transformation may lead to the loss of valuable subpopulation information.
we confirm that one of the reasons complex-based algorithms like the rbnas are successful is because they have innate resistance against batch effects.
this resistance stems from amplification of phenotypic-relevant signal from same-complex members and rank-based discretization of expression values .
as rbnas require no prior data transformations, the integrity of the data is preserved .
finally individual check on the top features selected by rbnas confirms that they are strongly associated with class and not batch effects.
this means that features selected in this manner are more likely to be clinically useful.
additional files
additional file 1: descriptions of supplementary methods  <cit> .
additional file 2: simulated data d <dig> h.
additional file 3: 3d-principal components analysis  scatterplots for all variables in samples d <dig> .301h and d. <dig> .302h.
b: 3d-principal components analysis  scatterplots for top 20% variables  in samples d <dig> .301h and d. <dig> .302h.
additional file 4: batch-effects in rc appears to be limited .

 the emergence of high-performance protein-extraction procedures, brute-force spectra-capture methods, and improved multiplexing technologies has transformed proteomics from a relatively low-throughput technology to one with critical practical applications in biology. the application of proteomics on clinical samples is concerned with unraveling proteome changes associated with disease using actual clinical samples. typically, two classes of samples---e.g., normal and disease ---are compared against each other. proteins exhibiting strong inter-class differences are marked as differential and analyzed for relevant functional roles. statistics provides powerful means for differential protein selection based on the hypothesis-testing framework. this process is commonly referred to as “feature selection”. unfortunately, despite increasing ease in data generation, extracting knowledge from proteomics expression data is difficult. proper feature selection, if done correctly, should lead directly to drug-target and biomarker identification; but in practice, this is seldom the case. in theory, a strongly differential feature should exhibit strong inter-class differences across samples. however, real samples are intrinsically noisy. this intrinsic noise is random and obfuscates proper feature selection by masking true inter-class differences. the manner in which the samples are prepared contributes towards a second type of variation, which unlike intrinsic random noise, is non-random and not associated with class effects; i.e., they do not distinguish sample classes d and d* specifically. this second source of variation, where features are more strongly correlated with technical factors than with sample classes, is referred to as batch effects. it is not straightforward to distinguish batch and class effects: when the former is mild, it may lead to bias during feature selection; but when strong, lead to downright selection of irrelevant proteins that confound and mislead and/or the loss of truly relevant proteins. in other words, batch effects obfuscate analysis. batch effects are known to be present in genomics assays. however, they are a nuisance in proteomics assays, where multiplexing limits impose constraints on the number of samples for concurrent analysis; e.g., analyzing eight samples with the commonly used 4-plex itraq labeling system requires at least two separate experiments performed at different times, or on different instruments. despite fairly recent work demonstrating that batch-effect correction may lead to substantial increase in feature-selection sensitivity, a systematic exploration of batch effects in proteomics data, and proposal of feasible workarounds, is missing. reasons include underestimating heterogeneity in practical usage, unsuitable data, and the erroneous belief that normalization eradicates batch effects. normalization is a data processing technique that adjusts global properties of measurements for individual samples for appropriate comparisons. examples include z- and quantile-normalization, and mean-scaling. however, normalization cannot eradicate batch effects, as the latter does not affect all variables similarly. in cases where statistical assumptions are violated, normalization may affect data integrity instead. batch effects are usually detected via principal component analysis, where the first two or three principal components are plotted for each sample colored by the batch labels, and separation of colors taken as evidence of batch effects. when batch effects are dominant, the first n pcs are expected to be dominated by batch effects, and removal of these pcs may be an alternative yet effective means of batch-effect correction. the remaining pcs---though these have lower contribution towards overall variation---may be dominated by small subsets of variables with good class-discrimination power. thus, feature selection at the level of pcs---i.e., using pcs, as opposed to proteins, as features---may be a viable batch effect-resistant feature-selection strategy. protein complex-based analysis, as a new analytical paradigm, provides a powerful yet stable means of selecting features, at the level of protein complexes, from proteomics data. protein complexes are strongly enriched for biological coherence signal, beating any combinations of alternative measurements using protein complex-based analysis, we have successfully recovered missing proteins and overcome consistency issues where patient samples present widely different protein sets. protein complex-based analysis also exhibits unparalleled stability and reproducibility in feature selection. we hypothesize that this superior performance may stem in part, from innate resistance to batch effects. we address the following gaps in batch effects, and its implications for feature selection in a proteomics setting. first, we propose a simple technique for simulating batch effects in proteomics data, and recommend using it for evaluating feature-selection procedures, as well as checking whether batch effect-correction algorithms are working as intended. second, while pca is the de facto approach for visualizing presence of batch effects, we investigate its feasibility as a feature-selection technique where features are principal components instead of proteins. and finally, as a potential new advantage, we check whether protein complex-based feature-selection algorithms are truly resistant to batch effects; and if so, whether they may supersede the need for batch effect-correction algorithms. methods simulated data --- d and d h we used part of the d dataset from the study of langley and mayr as a reference proteomics simulation dataset where differential variables are known a priori. quantitation is based on spectral counts. class effects and batch effects are inserted randomly, with the increase made in d* samples only. simulated data with both class and batch effects inserted is referred to as d h, while the original data with only class effects is referred to as d. real data --- renal cancer the renal cancer study of guo et al. comprises a total of swath runs originating from six pairs of non-tumorous and tumorous clear-cell renal cell carcinoma tissues, in two batches, rep and rep. batch effect-correction methods for batch-effect correction, we used quantile normalization and linear-scaling as generic approaches. quantile normalization and linear-scaling are not explicitly batch effect-correction methods. so, we also used combat on d h to remove batch effects, and evaluate performance recovery against the original d. combat is a well-known batch effect-correction approach and employs empirical bayes frameworks for adjusting data for batch effects. it is reported to be robust to outliers in small sample sizes while maintaining comparable performance to existing methods for large samples. statistical feature-selection methods four classes of feature-selection methods are tested to see whether they are robust against batch effects. the standard single-protein t-test and hypergeometric enrichment test are the most commonly used comparative analysis methods. we have also included two variants of rank-based network algorithms ---viz. subnets and fuzzy-subnets ---which were demonstrated to be highly stable and reliable. on real data, he, snet and fsnet are tested using corum complexes as their protein complex-based feature vector. the performance of these feature-selection methods are evaluated on precision and recall. on simulated data, these same methods are evaluated based on simulated complexes. in simulated data, the differential proteins are known a priori. we use these to create true-positive pseudo-complexes. to achieve this, a euclidean distance is first determined for all differential protein pairs across all samples. these are then clustered via ward’s linkage. differential proteins are reordered such that those with similar expression pattern are adjacent to each other. this reordered list is then split at regular intervals to generate true-positive pseudo-complexes. an equal number of non-significant proteins is randomly selected, reordered based on expressional correlation, and then split to generate an equal number of true-negative pseudo-complexes. we may alter the “purity” of the true-positive pseudo-complexes by reducing the proportion of differential proteins within them. in practice, we seldom observe all complex members being differentially expressed simultaneously. purity, therefore, is the proportion of differential proteins within each true-positive pseudo-complex. at 100% purity, simulated complexes are comprised solely of significant proteins; at 75% purity, 25% of the constituent significant proteins are randomly replaced with non-significant ones; and so on. reducing purity permits evaluation of the robustness and sensitivity of the complex-based analysis methods. purity is tested at three levels: and 50%. the true-positive and true-negative pseudo-complexes are combined into a single vector. evaluation is based on the f-score.  batch effects cannot be completely eradicated via batch effect-correction algorithms our method simulates batch effects in the following manner: in the first dimension, class-effect sizes are inserted based on the method of langley and mayr to distinguish classes d and d*. class-effect sizes are sampled randomly from five possibilities and inserted into randomly selected variables on samples belonging to class d*, constituting 20% of all variables. this is repeated to generate random datasets. in the second dimension, for each of the datasets, batch effects are inserted over all variables. here, batch effects are simulated by taking half the members in d and d* to be batch and the remaining half as batch batch effects are not evenly applied across all variables, and thus cannot be eradicated via simple normalization. to simulate batch effects unevenly, heterogeneity/batch-effect sizes, like class-effect sizes, are also sampled randomly from the same five possibilities. however, we make the assumption that batch effects influence all variables in a sample, and thus expect batch effects to account for a majority of variation. these simulated datasets, d.301h to d.400h are available for download from additional file fig. a simulations: from single-class data, samples are split randomly into two classes d and d*. class effects are randomly inserted into proteins in d*. this is followed by insertion of batch effects. b pc manipulation: proteins are sorted based on variance followed by a cutoff. the retained expression data is analyzed via principal component analysis. the first principal component corresponds strongly to batch effects and may be removed. c manipulation of pcs for clustering and class prediction. when combined with data of unknown labels, class labels can be predicted based on co-clustering with samples of known labels to understand how batch effects affect feature selection, we compare precision, recall and f-score based on the standard t-test, with and without multiple-test correction based on the benjamini-hochberg false discovery rate. the first two columns in fig.  reveal that incorporation of batch effects increases the variability of performance metrics, with particular impact on recall, and reduces overall performance. while we expect different feature-selection methods, aside from the t-test, may respond differently to batch effects, it is useful to incorporate heterogeneity/noise simulations during performance evaluation.fig. batch effect-correction does not work optimally in practice. top row: feature selection without multiple-test correction. bottom row: feature selection with fdr correction. first column shows base performance with only class effects. second column shows performance with batch effects incorporated. third, fourth and fifth columns show recovery using various batch effect-correction methods,. feature-selection test used here is the two-sample t-test the commonly used batch effect-correction method, combat, only partially recovers original test performance. therefore, it does not completely eradicate heterogeneity. additionally, while it improves overall performance, it also tends to reduce precision, incorporating false positives into the selected feature set. this is cause for concern during selection of features for experimental validation. in the non-fdr corrected scenario, combat also does not perform better than conventional data normalization methods, e.g., quantile normalization and linear-scaling. however, when test requirements are more stringent given the 5% fdr cutoff, then it is clear that combat provides considerable advantage. in spite of the simplicity of these simulations, it is noteworthy that batch effect-removal methods are not a panacea. we cannot declare combat is inferior, but rather, we will never know if batch effects have been effectively removed from real data, particularly when the data happens not to fit combat’s assumptions well. thus, a naïve reliance on batch effect-correction algorithms, without conducting further downstream checks for remnant batch effects, may potentially worsen analytical outcome. a relook at principal component analysis for detecting and removing batch effects principal component analysis yields linear combinations of each variable’s contribution to variance, but evidently not all variables are equally interesting or relevant: we may say the features that changed the most, i.e., exhibited the most variation, are likely more impactful. although we may not know this first-up for every feature, we can still reduce the feature set size via variance-based pre-selection. here, a cutoff is introduced to include only the top 20% proteins and used in pca. as stated earlier, pca is commonly used for visualizing batch effects. a simple way to do this is to label samples by classes and batches, and diagrammatize these as scatterplots across the first or principal components. in fig. 3a, the pcs are based on the top 20% proteins and evidently, this is sufficient for detecting batch effects. it is therefore unnecessary to use all variables. however, it is unclear whether this is sufficient for detecting class effects.fig. a 3d-principal component scatterplot analysis: batch effects dominate in the simulated data. shown are two examples d.301h and d.302h. b pc-paired boxplots: each pc may be split by class and batch, and compared against each other as side-by-side boxplots. this allows easy evaluation of the contribution of class and batch effects to each pc scatterplots, being rough visual guides, do not reveal well the contributions of class and batch effects to each pc. in our opinion, paired boxplots are more informative and, here, it is evident that pc and pc correspond to batch and class effects respectively. when a batch effect is observed, it is common practice to apply a batch effect-removal or –correction method. however, this does not necessarily work well in practice. moreover, if the data does not fit the correction method’s assumptions, it may lead to false positives. instead, we may opt for a more direct strategy by simply removing the first pc, and deploying the remaining pcs as features for analysis. when pc contributes strongly to batch effects, its removal should allow class effects to become the dominating source of variation.fig. a 3d-principal component analysis: class effects dominate in the simulated data, given the removal of pc. b heatmaps and hierarchical clustering: the remaining pcs may also be used as individual variables for clustering, and provide strong discrimination between classes d and d*. c combining two datasets with different batch effects: datasets a and b have the same differential feature set but different batch effects. combining these followed by analysis of all principal components shows batch effects dominate. however, removal of pc perfectly recovers class-effect discrimination without having to perform any feature selection using two examples, we show that removal of the first pc allows samples to cluster based on classes rather than batch. a caveat is that removal of pc works here primarily because it is strongly correlated with batch effects; i.e., batch effects account for the majority of variance in the data. on real data, it may necessitate the removal of several other pcs that are correlated with batch effects. moreover, if incompletely eradicated or inseparable from class effects, batch effects may resurface in subsequent pcs during analysis. suppose that removal of the first n pcs results in good class separation in pca, it may be possible to use the remaining pcs for feature selection and non-projection-based clustering techniques, e.g., hierarchical clustering and k-means. this may seem counter-intuitive, as during standard analysis involving pca, it is common to keep just the top n pcs accounting for the majority of variation. but not all variation is attributable towards class effects. moreover, pcs with large same-sign coefficients tend to represent non-class effect properties correlated with the variables; e.g., tsuchiya et al. demonstrated that, for their dataset, pc is linearly correlated with the magnitude of average gene expression. on the other hand, subsequent pcs with lower contribution towards overall variation may be dominated by small subsets of variables with good class-discrimination power. thus, instead of discarding the lower ranked pcs, it is more reasonable to remove the top pcs that are non-correlated with class effects. we find that subsequent pcs do correlate strongly with sample classes d and d*, and may be used as variables for clustering. a pc-based feature-selection approach is viable and allows relevant proteins to be retraced. this is executed by first identifying the pc of interest, and selecting proteins that contribute exclusively and strongly to it. as a simple test, we look at two scenarios using d.301h and d.302h. in the first scenario, proteins strongly associated with pc are selected. we term this “pc1”. in the second scenario, we removed the original pc and looked at proteins strongly associated with the new pc. this is termed “–pc1”. since the original pc is strongly associated with batch effects, its associated proteins are therefore less relevant, and we expect that performing feature selection on these using the t-test, and evaluating the corresponding precision, recall and f-score should fare worse than those proteins associated with the new pc we find the results following pc removal are similar to batch effect-correction algorithms, with concomitant increase in recall following removal of the first principal component. as a parallel test, we calculate a t-statistic for each protein under consideration, rank them from the highest to lowest, and keep only the top n proteins, where n = same number of proteins associated with –pc. interestingly, these do worse, and moreover, have limited overlaps with the –pc selected features.table effects on precision and recall for d.301h and d.302h before and following removal of proteins with heavy loadings on the first principal component tt refers to the standard t-test while tt/-pc are the top n-ranked tt-features. the jaccard coefficient indicates limited overlaps amongst the top tt-features associated with –pc features this procedure---viz. rank proteins by variance, perform pca using the top 20%, discard pcs that are strongly correlated with technical variables, and perform e.g., clustering using the remaining pcs---may be used for class prediction on new batches with unknown class labels. a schematic is provided in fig. 1c. and, to test this, two different sets of batch effects are inserted into d. the first is, and the second is, and the data combined. expectedly, clustering based on all pcs show that batch effects dominate. however, removal of pc recovers perfect class discrimination. this suggests that even when combining several datasets with different batch effects, removal of pc retains class effects, and permits class prediction. moreover, if we have multiple datasets of the same disease, properly dealing with batch effects makes it possible to pool these datasets for analysis. this is useful when larger sample sizes are needed for ad hoc analysis. using variance-based variable pre-selection and principal component manipulation to tackle real batch effects to evaluate how the procedure above is applicable towards real data, we consider the renal cancer study of guo et al., which contains two technical replicates. this data, rc, has been carefully processed; and batch effects appear contained. we insert batch effects into rc rep. however, pc does not purely contain variability from batch effects but also some signal from class effects. as with d h, pc projection also reveals dominance of batch effects that may be controlled via removal of pc. however, it appears some remnant batch effect still persists.fig. a hierarchical clustering based on protein expression: batch effects dominate here, as samples separate more strongly based on batch than by class. no variable selection was performed prior. b principal components and stratified by class and batch: although pc is still strongly associated with batch; there is also some association with class effects. in contrast, pc is strongly associated with class rather than with batch effects. c hcl based on all pcs: in this case, it is clear that batch effects dominate. d hcl following pc removal: removal of pc appears to eradicate a large proportion of the batch effects. but this is incomplete, suggesting that there may still be ambient batch effects remaining feature-selection methods with resistance to batch/heterogeneity effects there are remnant batch effects in rc that are difficult to eradicate, and may lead to bias during subsequent feature selection. when batch effects are strong, then removal of the first few pcs is a useful direct strategy, especially if information on batch and other potential confounding factors are not known a priori and batch effect-correction methods cannot be effectively deployed. on the other hand, it is not always straightforward to interpret the pcs and extract the proteins relevant for class effects. pc-based removal and batch-effect correction also may not be able to remove subtle batch effects. as an additional note of caution, batch effect-removal approaches---including the procedure described above---may at times be overkill: these corrections may unintentionally eliminate true biological heterogeneity amongst samples, which is informative and should not be discarded from the data in the first place. unfortunately, batch-effect and subpopulations are not easy to tell apart. and if we run the pc-removal or other batch effect-correction methods, subpopulation information is irrevocably lost. on the other hand, high heterogeneity in the form of multiple subpopulations can make analysis very challenging, particularly in cancer proteomics. one way forward is to incorporate robust data normalization methods and biological context directly into feature-selection approaches. recently, we expound on the advantages of protein complexes as suitable biological context in improving data analysis. unlike analysis at the level of proteins as features, the use of protein complexes as features, leads to improve stability and reproducibility. we are curious if the high performance of protein complex-based methods belonging to the family known as rank-based network analysis exhibit superior performance due to innate resistance to batch effects. there are several reasons why we think rbnas may be robust against batch effects: its score function uses rank-based discretization instead of exact values, which is robust against various biases, e.g., test-set bias. use of biological context increases biological signal over signal from other spurious correlations as only signal from same-complex members are summated. we already know that use of protein complexes increases power, and we believe the signal amplification is phenotypically relevant. previous tests have already demonstrated that complex-based features are specifically predictive for phenotype classes and that false-positive rates are low. however, a specific investigation into batch-effect resistance has not been done. hence, we test two members: subnets and fuzzy-snet. to test whether rbnas are effective in overcoming batch effects, as opposed to simply eliminating them, we performed two sets of tests; the first on simulated data as a proof-of-concept and the second on real data. besides snet and fsnet, we also include the standard single-protein t-test and the hypergeometric enrichment test. sp is a control based on the standard univariate t-test at the level of individual proteins. he is an over-representation-based technique meant to determine if the differential proteins are significantly enriched in some protein complex based on the hypergeometric test; i.e., it uses the same protein complexes, but not the same statistical test as the rbnas. using d and d h, we compare the f-scores across three purity levels, at both complex and individual protein levels. we observe that the f-score for the protein-based t-test is very low to begin with, but progressively worsens when batch effects are introduced. the hypergeometric enrichment pipeline uses the same pseudo-complexes as the rbnas, but also appears to be sensitive to batch effects. the rbnas, snet and fsnet, are resistant to batch effects. at the complex level, there is almost no difference in f-scores regardless of purity. this suggests that the rbnas are robust not only against the batch effects, but also decreasing differential signal as purity decreases. this finding is also corroborated at the level of individual proteins as well.fig. complex-based methods are robust against batch effects: distribution of f-scores for sp, he, snet and fsnet in simulated data where batch effects are absent or present based on the simulated complexes and individual proteins. note that for sp, the f-scores shown are always based on individual proteins our findings are a positive indication that the rbnas are highly robust against weaker differential signal and batch effects. additionally, given he’s poor performance, we assert that use of complexes alone is insufficient; the statistical test setup is also critical. this result is important as it is the first, to the best of our knowledge, to demonstrate that complex-based feature-selection approaches are resilient against batch effects. however, there are caveats: firstly, these batch effects are simulated, and unreflective of true batch effects. secondly, these pseudo-complexes may not be good approximations of true protein complexes. although we cannot test real data directly, we may still evaluate these methods on real data. using the original rc, we perform pca on significant features selected by the four feature-selection methods. for the first pcs, side-by-side boxplots stratified by class and batch demonstrates that rbnas are very powerful, and appear to capture variation stemming only from class effects. this is in contrast to sp, where batch effects still persist in pc he is commonly used as a post-hoc test following sp. it also takes advantage of the same set of protein complexes as snet and fsnet; but clearly, this is insufficient: the method used for statistical testing also matters.fig. pc side-by-side boxplots stratified by class and batch tested on real data:, rank-based network algorithms snet and fsnet are against batch effects, and only seems to capture variation stemming from class effects this finding is critical: since rbnas are robust against batch effects, this obviates the need for performing data transformations. this also means that if subpopulations do exist in the data, this information is retained. it should be noted that dealing with subpopulations is difficult and outside the scope of this work, although we may also use complexes for detailed in-depth study of subpopulations. while rbnas are evidently powerful against batch effects, especially against subtle ones that cannot be easily removed via removal of the first n pcs or via batch effect-correction algorithms, they are not perfect solutions. e.g. methods such as fsnet weigh each protein in a protein complex by the fraction of subjects where the protein is highly ranked. this fraction may be unstable from subsample to subsample, particularly in the presence of hidden subpopulations. this may reduce class-specific signals, making it difficult to identify good-quality and relevant features. downstream considerations post-analysis discarding batch effect-laden pcs is a transformation that provides a transformed dataset with much reduced batch effects. it helps identify strong class discriminatory features. yet, at the same time, new batches are not directly comparable to this transformed dataset. so it is difficult to extract directly clinical guideline/thresholds for future diagnostic use. for example, as a simple diagnostic tool, one wants a protein x such that: if x’s abundance is above a threshold y, then the patient is sick. but, in the presence of batch effects, different thresholds are needed for different batches. on the other hand, once the good features are found, one may apply more reliable technologies---i.e., ones that are less susceptible to batch effects---to measure only those features specifically. to test this empirically, the analysis in fig.  is repeated. we plot the boxplots based on log-normalized expression of significant proteins for sp and he, and scores of significant subnets for snet and fsnet, for the top three features. for sp and he, both class and batch variation are detected amongst the top three features. so it is reasonable to conclude that where these approaches are concerned, it is difficult to identify class discriminatory signal. the top three features selected by snet and fsnet capture class effects better, and appear robust against batch effects. thus, we expect that use of these features for diagnostics will yield better results.fig. expression or network score-based side-by-side boxplots stratified by class and batch for top three variables: rank-based network algorithms, snet and fsnet, can capture the class effects while robust against batch effects. both class and batch variability is present in the top variables selected by sp and he  the impact of batch effects in proteomics cannot be understated, and has key implications in clinical and translational research. we have shown that batch effect-correction algorithms are not a panacea, and that the corrected data may be erroneous. moreover, with the development of any novel feature-selection approach, it is worthwhile to test their robustness against simulated batch effects. we have illustrated that side-by-side barplots are better for visually detecting batch effects than the standard pca scatterplot-based representation format. moreover, the pcs themselves may be used as features, which may also be effectively traced back to relevant differential proteins. this is also a viable strategy complementary to batch effect-correction methods. unfortunately, subtle batch effects cannot be easily removed or detected, and can lead to bias in the analysis of real data. moreover, data transformation may lead to the loss of valuable subpopulation information. we confirm that one of the reasons complex-based algorithms like the rbnas are successful is because they have innate resistance against batch effects. this resistance stems from amplification of phenotypic-relevant signal from same-complex members and rank-based discretization of expression values. as rbnas require no prior data transformations, the integrity of the data is preserved. finally individual check on the top features selected by rbnas confirms that they are strongly associated with class and not batch effects. this means that features selected in this manner are more likely to be clinically useful. additional files additional file 1: descriptions of supplementary methods. additional file 2: simulated data d h. additional file 3: 3d-principal components analysis scatterplots for all variables in samples d.301h and d..302h. b: 3d-principal components analysis scatterplots for top 20% variables in samples d.301h and d..302h. additional file 4: batch-effects in rc appears to be limited. 
the principal components themselves may be used as differential features, from which relevant differential proteins may be effectively traced. worse still, they may alter data integrity, and introduce false positives. to address this, we introduce the concept of batch-effect resistant methods and demonstrate how such methods incorporating protein complexes are particularly resistant to batch effect without compromising data integrity. as complex-based analyses require no prior data transformation, data integrity is protected. individual checks on top-ranked protein complexes confirm strong association with phenotype classes and not batch. however, neither pc-based nor existing batch effect-correction methods address well subtle batch effects, which are difficult to eradicate, and involve data transformation and/or projection which is error-prone. in proteomics, batch effects are technical sources of variation that confounds proper analysis, preventing effective deployment in clinical and translational research. using simulated and real data, we demonstrate existing batch effect-correction methods do not always eradicate all batch effects. moreover, although principal component analysis is commonly used for detecting batch effects. batch effect are removable by identifying pcs highly correlated with batch but not class effect. we demonstrate for the first time their innate resistance against batch effects, even subtle ones. proteomicsbioinformaticsprincipal component analysisheterogeneitybatch effectsthe fifteenth asia pacific bioinformatics conference apbc 2017shenzhen, china 16- january issue-copyright-statement© the author 2017
 in proteomics, batch effects are technical sources of variation that confounds proper analysis, preventing effective deployment in clinical and translational research.  using simulated and real data, we demonstrate existing batch effect-correction methods do not always eradicate all batch effects. worse still, they may alter data integrity, and introduce false positives. moreover, although principal component analysis is commonly used for detecting batch effects. the principal components themselves may be used as differential features, from which relevant differential proteins may be effectively traced. batch effect are removable by identifying pcs highly correlated with batch but not class effect. however, neither pc-based nor existing batch effect-correction methods address well subtle batch effects, which are difficult to eradicate, and involve data transformation and/or projection which is error-prone. to address this, we introduce the concept of batch-effect resistant methods and demonstrate how such methods incorporating protein complexes are particularly resistant to batch effect without compromising data integrity.  protein complex-based analyses are powerful, offering unparalleled differential protein-selection reproducibility and high prediction accuracy. we demonstrate for the first time their innate resistance against batch effects, even subtle ones. as complex-based analyses require no prior data transformation, data integrity is protected. individual checks on top-ranked protein complexes confirm strong association with phenotype classes and not batch. therefore, the constituent proteins of these complexes are more likely to be clinically relevant. electronic supplementary material the online version of this article contains supplementary material, which is available to authorized users. keywords proteomicsbioinformaticsprincipal component analysisheterogeneitybatch effectsthe fifteenth asia pacific bioinformatics conference apbc 2017shenzhen, china 16- january issue-copyright-statement© the author 2017 