# Filter-free exhaustive odds ratio-based genome-wide interaction approach pinpoints evidence for interaction in the HLA region in psoriasis.

Laura Grange, Jean-François Bureau, Iryna Nikolayeva, Richard Paul, Kristel van Steen, Benno Schwikowski, Anavaj Sakuntabhai

► To cite this version:

Laura Grange, Jean-François Bureau, Iryna Nikolayeva, Richard Paul, Kristel van Steen, et al.. Filter-free exhaustive odds ratio-based genome-wide interaction approach pinpoints evidence for interaction in the HLA region in psoriasis.. BMC Genetics, 2015, 16, pp.11. 10.1186/s12863-015-0174-3. pasteur-01370988

HAL Id: pasteur-01370988

https://pasteur.hal.science/pasteur-01370988

Submitted on 23 Sep 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

Public Domain
# Filter-free exhaustive odds ratio-based genome-wide interaction approach pinpoints evidence for interaction in the HLA region in psoriasis

Laura Grange<sup>1,2,3</sup>, Jean-François Bureau<sup>1,2</sup>, Iryna Nikolayeva<sup>1,4,5</sup>, Richard Paul<sup>1,2</sup>, Kristel Van Steen<sup>6,7</sup>, Benno Schwikowski<sup>4*</sup> and Anavaj Sakuntabhai<sup>1,2*</sup>

## Abstract

**Background:** Deciphering the genetic architecture of complex traits is still a major challenge for human genetics. In most cases, genome-wide association studies have only partially explained the heritability of traits and diseases. Epistasis, one potentially important cause of this missing heritability, is difficult to explore at the genome-wide level. Here, we develop and assess a tool based on interactive odds ratios ($I_{OR}$), Fast Odds Ratio-based sCan for Epistasis (FORCE), as a novel approach for exhaustive genome-wide epistasis search. $I_{OR}$ is the ratio between the multiplicative term of the odds ratio (OR) of having each variant over the OR of having both of them. By definition, an $I_{OR}$ that significantly deviates from 1 suggests the occurrence of an interaction (epistasis). As the $I_{OR}$ is fast to calculate, we used the $I_{OR}$ to rank and select pairs of interacting polymorphisms for P value estimation, which is more time consuming.

**Results:** FORCE displayed power and accuracy similar to existing parametric and non-parametric methods, and is fast enough to complete a filter-free genome-wide epistasis search in a few days on a standard computer. Analysis of psoriasis data uncovered novel epistatic interactions in the HLA region, corroborating the known major and complex role of the HLA region in psoriasis susceptibility.

**Conclusions:** Our systematic study revealed the ability of FORCE to uncover novel interactions, highlighted the importance of exhaustiveness, as well as its specificity for certain types of interactions that were not detected by existing approaches. We therefore believe that FORCE is a valuable new tool for decoding the genetic basis of complex diseases.

**Keywords:** Genome-wide interaction studies, Epistasis, Plink, MBMDR, IOR

## Background

During the past decade, many genome-wide association studies (GWAS) have aimed to identify new genetic factors determining susceptibility to a variety of diseases [1,2]. Although promising and sometimes successful, these large-scale studies have only led to modest advances [3]. One explanation is that the underlying model that single SNPs contribute independently to the complex trait may frequently be too simple. Rather, complex

traits are likely to result from a complex interplay between genes, notably epistatic gene-environment and gene-gene interactions [4].

The principal obstacles in a genome-wide search for epistasis are statistical power to overcome the limitations of multiple testing and the computational time of the search itself. Over the past decades, many tools have been developed for epistasis detection using various statistical methods [5,6], including those based on regression [7-11], linkage disequilibrium and haplotypes [12,13], and Bayesian approaches [14,15]. Alternative approaches are based on data-filtering, machine-learning and data mining [16-19]. Here, we present an approach that detects pairwise epistasis on a genome-wide scale based on the

<footnote>* Correspondence: benno@pasteur.fr; anavaj@pasteur.fr</footnote>

<footnote n="4"><sup>4</sup>Department of Genomes and Genetics, Institut Pasteur, Systems Biology Lab, Paris 75015, France</footnote>

<footnote n="1"><sup>1</sup>Department of Genomes and Genetics, Institut Pasteur, Functional Genetics of Infectious Diseases Unit, Paris 75015, France</footnote>

<footnote>Full list of author information is available at the end of the article</footnote>
classical interaction odds ratio (I<sub>OR</sub>). Introduced by Piegorsch et al. in 1994 [20], this approach has mainly been used for the detection of gene-environment interactions in case-only designs [21]. VanderWeele et al. [22] showed how the use of I<sub>OR</sub> can help reveal mechanistic interactions in case-only datasets.

Firstly, we report on the first efficient implementation of an approach for genome-wide epistasis detection, which we call FORCE (Fast Odds Ratio sCan for Epistasis). Due to its mathematical simplicity, the approach is suitable for exhaustive unfiltered epistasis analysis; i.e., the exact value of the I<sub>OR</sub> statistic can be evaluated for all pairs of genotyped SNPs in reasonable time on a standard computer. We introduce the mathematics to compute exact P-values for the most extreme values of I<sub>OR</sub>.

Secondly, we describe the application of FORCE to the Welcome Trust Case Control Consortium (WTCCC) data on psoriasis, and analyze the previously unknown statistical interactions we found in the light of already-known results.

Lastly we ask whether the statistical interactions detected by FORCE were found due to its exhaustiveness and/or its underlying genetic model, and we present evidence for both. We show that the restriction of FORCE to analyzing only certain SNPs selected according to their marginal effect on psoriasis (as previously described by Knight et al. [23]) strongly limits the statistical significance of the results. We then benchmark the performance of FORCE and other popular methods to detect simulated epistatic interactions, always using exhaustive search. Under different common models for interaction and noise, FORCE consistently detects certain types of interactions better than other approaches.

## Methods

### Definition of interaction odds ratio (I<sub>OR</sub>)

For any given pair of SNPs, the interaction odds-ratio statistic I<sub>OR</sub> is calculated from a pair of 2x2 contingency tables. These tables are derived from 3×3 tables of all allele combinations, by collapsing them according to a dominant or recessive model (see Table 1). Following preliminary evidence that the dominant model allowed more efficient detection of epistasis (Table 2), all analyses were performed using this dominant genetic model.

<table><caption>Table 1 Contingency table under a dominant model</caption><thead><tr><th>SNP1</th><th>SNP2</th><th>Cases</th><th>Controls</th></tr></thead><tbody><tr><td>AA</td><td>BB</td><td>α</td><td>β</td></tr><tr><td>AA</td><td>Bb or bb</td><td>γ</td><td>δ</td></tr><tr><td>Aa or aa</td><td>BB</td><td>ε</td><td>ζ</td></tr><tr><td>Aa or aa</td><td>Bb or bb</td><td>η</td><td>θ</td></tr></tbody></table>

Major alleles are respectively A and B for each SNP and minor alleles a and b. The risk allele is assumed to be the minor allele.

We define the following odds ratios:

$$OR_1 = \frac{\beta\epsilon}{\alpha\zeta}, OR_2 = \frac{\beta\gamma}{\alpha\delta}, OR_{1+2} = \frac{\beta\eta}{\alpha\theta}, \text{ and } I_{OR} = \frac{OR_{1+2}}{OR_1 \cdot OR_2} = \frac{\alpha\delta\zeta\eta}{\beta\gamma\epsilon\theta}$$

Note that I<sub>OR</sub> is undefined when the denominator of this expression becomes zero. For formal consistency, we therefore added a pseudocount of 1 to each cell of the two contingency tables.

### Statistical significance: Empirical and exact P-values

Note that an I<sub>OR</sub> of x equals an I<sub>OR</sub> of 1/x after exchanging counts between cases and controls. We define universal I<sub>OR</sub>, u(I<sub>OR</sub>):

$$u(I_{OR}) = \frac{1}{I_{OR}} \text{ if } I_{OR} \le 1 \text{ and } u(I_{OR}) = I_{OR} \text{ if } I_{OR} > 1.$$

This definition allows us to express significant deviations of u(I<sub>OR</sub>) from the expectation of 1 using a one-tailed P-value.

Pairs with high u(I<sub>OR</sub>) were identified by the straightforward algorithm that computes u(I<sub>OR</sub>) for each pair of given SNPs. Our C implementation encodes, in a pre-processing step, all data related to any given SNP into a bit string, and then uses fast logical and bit-counting functions to compute u(I<sub>OR</sub>) for all pairs.

Marginal empirical P-values for any given pair of SNPs were calculated as the proportion of u(I<sub>OR</sub>) values from randomly generated permutations of case-control labels that were larger than or equal to the value of u(I<sub>OR</sub>) obtained for the same pair in real data. The number of permutations performed (1000 for simulated data, 100,000 for real data) was adapted to the number of tests performed in these two scenarios.

Exact P-values were calculated using

$$\begin{align*} p &= \Pr(u(I_{OR}) \ge x) \\ &= 2 \cdot \sum_{(\alpha', \gamma', \epsilon', \eta') : I_{OR} \ge x, \alpha' + \gamma' + \epsilon' + \eta' = \alpha + \gamma + \epsilon + \eta} \frac{\binom{\alpha + \beta}{\alpha'} \binom{\gamma + \delta}{\gamma'} \binom{\epsilon + \zeta}{\epsilon} \binom{\eta + \theta}{\eta'}}{\binom{\alpha + \beta + \gamma + \delta + \epsilon + \zeta + \eta + \theta}{\alpha' + \gamma' + \epsilon' + \eta'}} \end{align*}$$

and computed by the straightforward algorithm with four nested loops to cover all required parameter tuples (α',γ',ε',η'). Each inner loop only visits those parameter values that correspond to possible tuples with α' + γ' + ε' + η' = α + γ + ε + η, given the parameter values in the outer loop. Summed are those terms with u(I<sub>OR</sub>) ≥ x.

### Application of FORCE to psoriasis data

To evaluate FORCE, we assessed its performance on the WTCCC psoriasis dataset. Initial GWAS and further analyses performed on these data are described in [24]. Following general practice for pre-processing, we excluded potentially low-quality SNP data from further analysis.
<table><caption>Table 2 Power and Family-wise error rate (FWER) for detection of the functional pair using a dominant or recessive transmission assumption in 6 different epistasis models</caption><thead><tr><th>Genetic model</th><th>Test</th><th>Model 1</th><th>Model 2</th><th>Model 3</th><th>Model 4</th><th>Model 5</th><th>Model 6</th></tr></thead><tbody><tr><td rowspan="2">Dominant</td><td>Power</td><td>1</td><td>0.97</td><td>0.96</td><td>1</td><td>0.93</td><td>0.99</td></tr><tr><td>FWER</td><td>0.05</td><td>0.02</td><td>0.02</td><td>0.06</td><td>0.05</td><td>0.04</td></tr><tr><td rowspan="2">Recessive</td><td>Power</td><td>0.93</td><td>0.96</td><td>0.01</td><td>0.01</td><td>0</td><td>0</td></tr><tr><td>FWER</td><td>0.04</td><td>0.07</td><td>0.02</td><td>0.02</td><td>0.03</td><td>0.01</td></tr></tbody></table>

Specifically, we discarded i) any individual whose total missing rate was above 0.05, ii) any SNP with a frequency of missing data above 0.05, and iii) any SNP with minor allele frequency below 0.05. After pre-processing, our dataset consisted of 2,618 cases, 2,737 controls and 491,191 SNPs, corresponding to approximately $1.2 \times 10^{11}$ SNP pairs. We excluded pairs with a genomic distance of less than 100 kb to avoid pairs in linkage disequilibrium. In addition, we found that low row and cell counts in the contingency table (Table 1) can lead to extreme but frequently not significant values of $u(I_{OR})$. For the purposes of this study, we excluded 3,521,114 SNP pairs with a total count of less than 50 in any row, or less than 5 in any cell of the contingency table. In addition to FORCE, we performed PLINK (FastEpistasis mode) on the top-ranked 500 pairs to compare the results obtained with both methods.

## Comparison of exhaustive FORCE with semi-exhaustive and conditional search

To assess the utility of exhaustive search, we constructed a reference dataset of SNPs previously implicated in psoriasis. We started with a set of 34 SNPs from two previous reviews on psoriasis genetics [25,26] that were part of our psoriasis dataset. After applying quality control thresholds (described above), 18 SNPs remained.

Following general practice for genome-wide approaches, for exhaustive and semi-exhaustive searches, we used a genome-wide significance threshold of $p = \frac{2 \times 0.05}{(10^6)^2} = 10^{-13}$, which is based on a model of the human genome with $10^6$ independent SNPs [27].

## Comparison of FORCE with other approaches on simulated datasets

We simulated datasets of 10 biallelic SNPs over 200 cases and 200 controls following the Hardy-Weinberg equilibrium model. Interactions were simulated according to six different previously described models without main effect [28] (Table 3). These models represent pure epistasis effects, and not confounding main effects. Model 1 is an interaction effect in which high risk of disease occurs when inheriting heterozygous genotypes at either locus (Aa or Bb) but not both. Model 2 represents high risk of disease when inheriting two high-risk

alleles that could be A and/or B. Models 3–6 correspond to the epistasis model discovery method described by Moore et al. [29]. Each of these models represents an interaction effect without any main effects. Allele frequencies are $p = 0.25$ and $q = 0.75$ for model 3 and 4, $p = 0.1$ and $q = 0.9$ for model 5 and 6.

For each of the six models, we generated 100 datasets in each of the 16 conditions of the presence or absence of four of the most commonly encountered sources of noise: missing data (MS), genotyping errors (GE), genetic heterogeneity (GH), and phenocopy (PC).

For GH, two independent interactions were simulated instead of one, each interaction being risk-associated in half of the affected cases. When PC was simulated, interaction affected the trait for half of the cases, emulating an unknown environmental effect. GE and MS were simulated at 5%, as previously described [28].

An epistatic pair of SNPs was considered as detected if the empirical P-value was below 0.001, i.e., below 0.05 after Bonferroni correction. Power was estimated as $n/100$, where n is the number of datasets with detection (s). When two pairs ($P_1$, $P_2$) of SNPs were simulated, detection was counted under one of three different conditions: D1) when $P_1$ and $P_2$ were detected, D2) when $P_1$ was detected, or D3) when $P_1$ or $P_2$ was detected. Family-wise error rate (FWER) was calculated as $m/100$, where m

<table><caption>Table 3 Penetrances and allele frequencies (p,q) used to simulate the interaction models – from Ritchie [28]</caption><thead><tr><th rowspan="2"></th><th colspan="3">Model 1</th><th colspan="3">Model 2</th><th colspan="3">Model 3</th></tr><tr><th>BB</th><th>Bb</th><th>bb</th><th>BB</th><th>Bb</th><th>bb</th><th>BB</th><th>Bb</th><th>bb</th></tr></thead><tbody><tr><td>AA</td><td>0</td><td>0.10</td><td>0</td><td>AA</td><td>0</td><td>0</td><td>0.10</td><td>AA</td><td>0.08</td><td>0.07</td><td>0.05</td></tr><tr><td>Aa</td><td>0.10</td><td>0</td><td>0.10</td><td>Aa</td><td>0</td><td>0.05</td><td>0</td><td>Aa</td><td>0.10</td><td>0</td><td>0.10</td></tr><tr><td>aa</td><td>0</td><td>0.10</td><td>0</td><td>aa</td><td>0.10</td><td>0</td><td>0</td><td>aa</td><td>0.03</td><td>0.10</td><td>0.04</td></tr><tr><td>p = 0.5, q = 0.5</td><td colspan="3"></td><td>p = 0.5, q = 0.5</td><td colspan="3"></td><td>p = 0.25, q = 0.75</td><td colspan="3"></td></tr><tr><th colspan="4">Model 4</th><th colspan="4">Model 5</th><th colspan="4">Model 6</th></tr><tr><th></th><th>BB</th><th>Bb</th><th>bb</th><th>BB</th><th>Bb</th><th>bb</th><th>BB</th><th>Bb</th><th>bb</th><th>BB</th><th>Bb</th><th>bb</th></tr><tr><td>AA</td><td>0</td><td>0.01</td><td>0.09</td><td>AA</td><td>0.07</td><td>0.05</td><td>0.02</td><td>AA</td><td>0.09</td><td>0.001</td><td>0.02</td></tr><tr><td>Aa</td><td>0.04</td><td>0.01</td><td>0.08</td><td>Aa</td><td>0.05</td><td>0.09</td><td>0.01</td><td>Aa</td><td>0.08</td><td>0.07</td><td>0.005</td></tr><tr><td>aa</td><td>0.07</td><td>0.09</td><td>0.03</td><td>aa</td><td>0.02</td><td>0.01</td><td>0.03</td><td>aa</td><td>0.003</td><td>0.007</td><td>0.02</td></tr><tr><td>p = 0.25, q = 0.75</td><td colspan="3"></td><td>p = 0.10, q = 0.9</td><td colspan="3"></td><td>p = 0.10, q = 0.9</td><td colspan="3"></td></tr></tbody></table>

Marginal penetrances for each genotype are identical as we simulate pure epistasis effects.
is the number of datasets for which at least one pair other than the simulated pair was detected.

## Results

### FORCE enables exhaustive unfiltered epistasis analysis

The FORCE method for epistasis detection is based on the choice of a dominant or recessive model that collapses combinations of allele counts into two 2x2 incidence tables (see Methods). Interactions are then detected as extreme values of the I<sub>OR</sub> statistic. We implemented the FORCE method for epistasis in C language [30]. Due to its mathematical simplicity and efficient implementation, the computation of I<sub>OR</sub> could be performed rapidly, compared to other approaches (4.3 days on a single core of a stand-ard computer). Table 4 shows running times of different methods selected for this study.

### Identification of statistically strong interactions requires exhaustive search

To assess the value of exhaustive search, we first evaluated the performance of a conventional, non-exhaustive approach of constraining the analysis to pairs of SNPs that were previously shown to have main effects associated with the phenotype. We therefore performed a constrained analysis on all pairs of 18 high-quality SNPs that had main effects on psoriasis in previous GWA studies (see Methods). Table 5 gives the best 25 hits obtained through this approach when evaluated on the WTCCC dataset on psoriasis [24] (the results of all pairs are shown in Additional file 1: Table S1). None of the 153 pairs reached a significant interaction P-value below a genome-wide significance threshold of 10<sup>-13</sup>.

A more comprehensive approach, to which we will here refer to as semi-exhaustive, constrains only one of the SNPs in a pair to a set of previously identified SNPs [8]. Table 6 shows, for each of the 18 previously identified “main effects” SNPs, the highest-scoring interactors, according to the FORCE and PLINK FastEpistasis statistics. Note that FORCE and PLINK identified a few genome-wide significant interactions with P-values as low as 10<sup>-20</sup>.

<table><caption>**Table 4** Average time needed to exhaustively test one/all 1.25×10<sup>11</sup> pairs among 500,000 SNPs using a single-core CPU computer</caption><thead><tr><td>Software</td><td>Time for one/all SNPs (single core)</td></tr></thead><tbody><tr><td>MB-MDR [16]</td><td>5×10<sup>-3</sup> s/20 years [31]</td></tr><tr><td>PLINK Epistasis [7]</td><td>2×10<sup>-4</sup> s/289 days [5]</td></tr><tr><td>PLINK FastEpistasis [8]</td><td>2×10<sup>-5</sup> s/29 days [32]</td></tr><tr><td>FORCE</td><td>3×10<sup>-6</sup> s/4.3 days</td></tr><tr><td>GWIS - 3 filters [33]</td><td>1.6×10<sup>-6</sup> s/2.2 days [33]</td></tr><tr><td>GWIS - 1 filter [33]</td><td>3.8×10<sup>-7</sup> s/0.5 days [33]</td></tr></tbody></table>

We included the recent GWIS approach that is described as ‘exhaustive’, but uses filtering to avoid computing test statistics for all pairs of SNPs.

Finally, the relatively low computational complexity required for the FORCE statistic allowed us to perform exhaustive analysis of all SNP pairs in the psoriasis dataset. The results are shown in Table 7 (100 best hits shown in Additional file 1: Table S2). Strikingly, the best resulting P-values are another 20 orders of magnitude lower than the P-values identified by semi-exhaustive search. This shows that a large number of the most significant interactions are missed by the semi-exhaustive approach, and hence that the possibility of discovering the statistically best-supported interactions requires an exhaustive approach. Interestingly, FORCE and PLINK identify distinct interactions.

### FORCE pinpoints interactions beyond main effects in the HLA region

We also analyzed the exhaustive FORCE results with regard to previous studies, which have detected numerous main effects [24-26], but only few weak statistical interactions [24,34,35]. We assessed the performance of FORCE using the WTCCC psoriasis dataset, which contains 2,618 cases, 2,737 controls and 491,191 SNPs. Table 7 shows the 25 best FORCE hits. Twenty-one out of 25 SNP pairs involve SNPs located in the HLA region on chromosome 6, which is consistent with the known strong involvement of the HLA region in psoriasis. Interestingly, certain SNP pairs found to be statistically significant by FORCE did not reach genome-wide significance when using PLINK FastEpistasis.

It is well known that SNPs with main effects may falsely appear to be interacting [36]. To avoid such artifacts in our analysis, we removed those SNPs that displayed a univariate statistical association P-value of 10<sup>-5</sup> or less [24]. The results show three highly significant interactions involving SNPs from the HLA region that display no main effect (Table 8). In the absence of correlation between the SNPs we claim that these findings provide evidence of interactive effects involved in psoriasis susceptibility. This confirms that FORCE is able to uncover novel statistical interactions in the HLA region that have not been detected before using conventional approaches.

### FORCE systematically detects interactions missed by other approaches

Besides its exhaustiveness, the other characteristic feature of the FORCE approach is the use of the I<sub>OR</sub> statistic for genome-wide epistasis analysis. To study the extent to which the choice of this statistic contributed to the identification of novel statistical interactions, we used datasets that contained different simulated epistatic interactions between SNPs without main effects, according one of six models of Ritchie [28], and none or one of the four sources of noise: Genotyping Error (GE), Missing Data (MS), Genetic Heterogeneity (GH), Phenocopy (PC) (see
<table><caption>Table 5 Results from conditional search, restricted to pairs of previously implicated SNPs</caption><thead><tr><th colspan="2">First GWAS SNP</th><th colspan="2">Second GWAS SNP</th><th colspan="2">FORCE</th><th>PLINK FastEpistasis</th></tr><tr><th>rs number</th><th>Chromosomal location</th><th>rs number</th><th>Chromosomal location</th><th>I<sub>OR</sub></th><th>Empirical p-value</th><th>p-value</th></tr></thead><tbody><tr><td>rs10484554</td><td>6p21.33</td><td>rs27524</td><td>5q15</td><td>6.846</td><td>0.008882</td><td>0.003095</td></tr><tr><td>rs10484554</td><td>6p21.33</td><td>rs3134792</td><td>6p21.33</td><td>1.068</td><td>0.3014</td><td>0.007746</td></tr><tr><td>rs2201841</td><td>1p31.3</td><td>rs3213094</td><td>5q33.3</td><td>4.737</td><td>0.02952</td><td>0.012373</td></tr><tr><td>rs3134792</td><td>6p21.33</td><td>rs4795067</td><td>17q11</td><td>3.188</td><td>0.07419</td><td>0.012783</td></tr><tr><td>rs20541</td><td>5q31</td><td>rs17716942</td><td>2q24</td><td>6.987</td><td>0.008212</td><td>0.013389</td></tr><tr><td>rs702873</td><td>2p16</td><td>rs4795067</td><td>17q11</td><td>3.414</td><td>0.06466</td><td>0.014129</td></tr><tr><td>rs10484554</td><td>6p21.33</td><td>rs4795067</td><td>17q11</td><td>2.597</td><td>0.1071</td><td>0.018096</td></tr><tr><td>rs610604</td><td>6q23</td><td>rs17716942</td><td>2q24</td><td>6.591</td><td>0.01025</td><td>0.023261</td></tr><tr><td>rs3213094</td><td>5q33.3</td><td>rs12580100</td><td>12q13.2</td><td>5.132</td><td>0.02349</td><td>0.028993</td></tr><tr><td>rs4649203</td><td>1p36</td><td>rs240993</td><td>6q21</td><td>2.270</td><td>0.1319</td><td>0.037791</td></tr><tr><td>rs4649203</td><td>1p36</td><td>rs702873</td><td>2p16</td><td>1.237</td><td>0.266</td><td>0.041136</td></tr><tr><td>rs3134792</td><td>6p21.33</td><td>rs27524</td><td>5q15</td><td>11.840</td><td>0.000581</td><td>0.041483</td></tr><tr><td>rs702873</td><td>2p16</td><td>rs2546890</td><td>5q33.3</td><td>0.804</td><td>0.37</td><td>0.041729</td></tr><tr><td>rs27524</td><td>5q15</td><td>rs17716942</td><td>2q24</td><td>5.280</td><td>0.02158</td><td>0.045812</td></tr><tr><td>rs610604</td><td>6q23</td><td>rs6701216</td><td>1q21</td><td>4.289</td><td>0.03837</td><td>0.057701</td></tr><tr><td>rs2201841</td><td>1p31.3</td><td>rs2546890</td><td>5q33.3</td><td>2.587</td><td>0.1077</td><td>0.059206</td></tr><tr><td>rs27524</td><td>5q15</td><td>rs7993214</td><td>13q14.11</td><td>3.596</td><td>0.05793</td><td>0.059609</td></tr><tr><td>rs3134792</td><td>6p21.33</td><td>rs3213094</td><td>5q33.3</td><td>2.610</td><td>0.1062</td><td>0.072999</td></tr><tr><td>rs702873</td><td>2p16</td><td>rs2201841</td><td>1p31.3</td><td>1.669</td><td>0.1964</td><td>0.083717</td></tr><tr><td>rs10484554</td><td>6p21.33</td><td>rs12580100</td><td>12q13.2</td><td>2.535</td><td>0.1113</td><td>0.086631</td></tr><tr><td>rs4649203</td><td>1p36</td><td>rs6701216</td><td>1q21</td><td>3.518</td><td>0.06072</td><td>0.086671</td></tr><tr><td>rs2201841</td><td>1p31.3</td><td>rs27524</td><td>5q15</td><td>1.666</td><td>0.1968</td><td>0.088785</td></tr><tr><td>rs4112788</td><td>1q21.3</td><td>rs7993214</td><td>13q14.11</td><td>1.546</td><td>0.2137</td><td>0.090419</td></tr><tr><td>rs240993</td><td>6q21</td><td>rs7993214</td><td>13q14.11</td><td>1.896</td><td>0.1685</td><td>0.096038</td></tr><tr><td>rs6701216</td><td>1q21</td><td>rs8016947</td><td>14q13</td><td>1.087</td><td>0.2971</td><td>0.100508</td></tr></tbody></table>

Methods for details). We then evaluated the power of FORCE and three other popular epistasis detection methods (PLINK Epistasis [7] and PLINK FastEpistasis [8] using default parameters, and MB-MDR [16], using recommended parameters [37]) to detect the simulated interactions. We used a significance threshold of 0.001. Figure 1 shows the results for all epistatic models for the case of no noise.

Under all six models, FORCE and MB-MDR consistently showed power close to 1. The situation became more interesting in the presence of noise. Figure 2 shows the power of the tested methods for all six models in the presence of one type of noise (numerical values for are given in Tables 9, 10, 11 and 12). While the results for Genotyping Errors (GE) and Missing Data (MS) were very similar to the no-noise scenario, the presence of Genetic Heterogeneity (GH, independent of the definition of “detection”) or Phenocopy (PC) revealed larger differences among the different approaches. Firstly, we noted

that, with GH and PC, all approaches lose power. Secondly, we observed that different approaches worked consistently better than others, depending on the interaction model. For interaction models 1 and 2, MB-MDR dominated all other approaches; FORCE dominated the other approaches for interaction models 3–6.

## Discussion

This study introduces the FORCE approach for genome-wide epistasis analysis. On the basis of the Interaction Odds Ratio ($I_{OR}$) statistic, it performs a genome-wide search for epistatic interactions between pairs of SNPs in a reasonable time on a standard laptop computer. The search is exhaustive and filter-free; i.e., the result is guaranteed to reflect the most extreme $I_{OR}$ values over all possible interactions. Exhaustive search using FORCE is possible because of the computational simplicity of the $I_{OR}$ statistic.
<table><caption>Table 6 Semi-exhaustive search among SNP pairs containing a GWAS-identified SNP</caption><thead><tr><th colspan="4">GWAS-identified SNP</th><th colspan="5">Highest-scoring interactor with GWAS-identified SNP</th><th colspan="3">PLINK FastEpistasis Z-score</th></tr><tr><th rowspan="2">rs number</th><th rowspan="2">Chromosomal location</th><th rowspan="2">Risk allele OR</th><th rowspan="2">Single association p-value</th><th colspan="4">FORCE u(I<sub>OR</sub>)</th><th colspan="3">FORCE p-value<sup>a</sup></th></tr><tr><th>rs number</th><th>Chromosomal location</th><th>u(I<sub>OR</sub>)</th><th>Empirical p-value</th><th>Exact p-value<sup>a</sup></th><th>rs number</th><th>Chromosomal location</th><th>Exact p-value</th></tr></thead><tbody><tr><td>rs10484554</td><td>6p21.33</td><td>4.66</td><td>4.0E-214</td><td>rs4151664</td><td>6p21.33</td><td>2.97</td><td>&lt;10E-06</td><td>7.86E-10</td><td>rs28615950</td><td>6p21.3</td><td><strong>2.12E-14</strong></td></tr><tr><td>rs2546890</td><td>5q33.3</td><td>1.54</td><td>1.0E-20</td><td>rs7525345</td><td>1p31.1</td><td>2.53</td><td>7.10E-04</td><td>2.17E-06</td><td>rs4796093</td><td>17q12</td><td>1.24E-06</td></tr><tr><td>rs6701216</td><td>1q21</td><td>1.45</td><td>6.2E-05</td><td>rs2156892</td><td>22q11.22</td><td>2.5</td><td>&lt;10E-06</td><td>1.30E-13</td><td>rs10853580</td><td>18q21.1</td><td>4.99E-07</td></tr><tr><td>rs4112788</td><td>1q21.3</td><td>1.41</td><td>6.5E-09</td><td colspan="5">No pair meeting all inclusion criteria</td><td>rs4459983</td><td>4q21.1</td><td>3.35E-08</td></tr><tr><td>rs7993214</td><td>13q14.11</td><td>1.41</td><td>2.0E-06</td><td colspan="5">No pair meeting all inclusion criteria</td><td>rs10800559</td><td>1q23.3</td><td>4.04E-08</td></tr><tr><td>rs3213094</td><td>5q33.3</td><td>1.39</td><td>5.0E-11</td><td colspan="5">No pair meeting all inclusion criteria</td><td>rs10512686</td><td>5p13.1</td><td>8.12E-06</td></tr><tr><td>rs17716942</td><td>2q24</td><td>1.29</td><td>1.1E-13</td><td>rs16928722</td><td>10q22.1</td><td>2.69</td><td>4.10E-06</td><td>2.78E-06</td><td>rs2553680</td><td>8q13.2</td><td>5.67E-07</td></tr><tr><td>rs20541</td><td>5q31.1</td><td>1.27</td><td>5.0E-15</td><td colspan="5">No pair meeting all inclusion criteria</td><td>rs17171818</td><td>5q31.2</td><td>5.63E-07</td></tr><tr><td>rs240993</td><td>6q21</td><td>1.25</td><td>5.3E-20</td><td>rs4727157</td><td>7q21.12</td><td>2.78</td><td>&lt;10E-06</td><td><strong>1.88E-20</strong></td><td>rs2877327</td><td>22q12.1</td><td>1.26E-07</td></tr><tr><td>rs4795067</td><td>17q11</td><td>1.19</td><td>4.0E-11</td><td colspan="5">No pair meeting all inclusion criteria</td><td>rs3819847</td><td>3q27.3</td><td>4.59E-07</td></tr><tr><td>rs8016947</td><td>14q13</td><td>1.19</td><td>1.5E-11</td><td colspan="5">No pair meeting all inclusion criteria</td><td>rs11071746</td><td>15q22.31</td><td>1.58E-08</td></tr><tr><td>rs610604</td><td>6q23</td><td>1.19</td><td>7.0E-07</td><td>rs17585537</td><td>3p26.2</td><td>2.69</td><td>&lt;10E-06</td><td><strong>6.47E-19</strong></td><td>rs4794888</td><td>17q11.1</td><td>1.11E-06</td></tr><tr><td>rs12580100</td><td>12q13.2</td><td>1.17</td><td>1.0E-06</td><td>rs7565742</td><td>2q31.2</td><td>3.39</td><td>&lt;10E-06</td><td><strong>6.80E-20</strong></td><td>rs2992154</td><td>13q21.31</td><td>2.07E-06</td></tr><tr><td>rs4649203</td><td>1p36</td><td>1.13</td><td>6.8E-08</td><td colspan="5">No pair meeting all inclusion criteria</td><td>rs7661684</td><td>4q28.1</td><td>1.70E-06</td></tr><tr><td>rs2201841</td><td>1p31.3</td><td>1.13</td><td>3.0E-08</td><td colspan="5">No pair meeting all inclusion criteria</td><td>rs12783252</td><td>10q26.11</td><td>3.79E-06</td></tr><tr><td>rs27524</td><td>5q15</td><td>1.13</td><td>2.6E-11</td><td colspan="5">No pair meeting all inclusion criteria</td><td>rs7849719</td><td>9q21.31</td><td>1.37E-08</td></tr><tr><td>rs702873</td><td>2p16</td><td>1.12</td><td>3.6E-09</td><td colspan="5">No pair meeting all inclusion criteria</td><td>rs10897897</td><td>11q13.4</td><td>1.79E-06</td></tr><tr><td>rs3134792</td><td>6p21.33</td><td>NR</td><td>1.0E-09</td><td>rs1062070</td><td>6p21.32</td><td>2.88</td><td>&lt;10E-06</td><td>2.85E-10</td><td>rs1062070</td><td>6p21.32</td><td><strong>5.25E-14</strong></td></tr></tbody></table>

<footnote><sup>a</sup>Bold data are genome-wide significant interactions.</footnote>
<table><caption>Table 7 FORCE Exhaustive search top hits, and PLINK FastEpistasis results in WTCCC psoriasis data</caption><thead><tr><th colspan="2">SNP pair description</th><th colspan="4">Epistasis search results</th></tr><tr><th colspan="2">First SNP</th><th colspan="2">Second SNP</th><th colspan="2">FORCE</th></tr><tr><th>rs number</th><th>Chromosomal location (position)</th><th>rs number</th><th>Chromosomal location (position)</th><th>u(I<sub>OR</sub>)</th><th>p-value<sup>a</sup></th></tr></thead><tbody><tr><td>rs4151664</td><td>6p21.33 (31,920,873)</td><td>rs9267532</td><td>6p21.33 (31,639,979)</td><td>10.588</td><td><strong>3.32E-33</strong></td></tr><tr><td>rs4151664</td><td>6p21.33 (31,920,873)</td><td>rs2227956</td><td>6p21.33 (31,778,272)</td><td>9.662</td><td><strong>2.02E-26</strong></td></tr><tr><td>rs3132468</td><td>6p21.33 (31,475,486)</td><td>rs4151664</td><td>6p21.33 (31,920,873)</td><td>9.571</td><td><strong>3.14E-25</strong></td></tr><tr><td>rs9267546</td><td>6p21.33 (31,673,436)</td><td>rs4151664</td><td>6p21.33 (31,920,873)</td><td>8.340</td><td><strong>1.08E-31</strong></td></tr><tr><td>rs4151664</td><td>6p21.33 (31,920,873)</td><td>rs2260000</td><td>6p21.33 (31,593,476)</td><td>7.749</td><td><strong>3.74E-18</strong></td></tr><tr><td>rs2523608</td><td>6p21.33 (31,322,559)</td><td>rs4151664</td><td>6p21.33 (31,920,873)</td><td>7.695</td><td><strong>1.08E-18</strong></td></tr><tr><td>rs4151664</td><td>6p21.33 (31,920,873)</td><td>rs2855807</td><td>6p21.33 (31,469,323)</td><td>7.444</td><td><strong>3.88E-17</strong></td></tr><tr><td>rs2596464</td><td>6p21.33 (31,416,156)</td><td>rs4151664</td><td>6p21.33 (31,920,873)</td><td>7.379</td><td><strong>2.67E-15</strong></td></tr><tr><td>rs3129939</td><td>6p21.32 (31,412,961)</td><td>rs3131296</td><td>6p21.32 (32,172,993)</td><td>7.376</td><td><strong>6.43E-41</strong></td></tr><tr><td>rs2516464</td><td>6p21.33 (31,416,156)</td><td>rs12663103</td><td>6p21.32 (32,161,324)</td><td>7.229</td><td>4.25E-13</td></tr><tr><td>rs6906662</td><td>6p21.32 (32,266,506)</td><td>rs9267649</td><td>6p21.33 (31,824,828)</td><td>7.187</td><td><strong>1.59E-25</strong></td></tr><tr><td>rs12153855</td><td>6p21.33 (32,074,804)</td><td>rs2523608</td><td>6p21.33 (31,322,559)</td><td>7.181</td><td><strong>1.59E-23</strong></td></tr><tr><td>rs4149013</td><td>12p12.2 (21,282,410)</td><td>rs9356206</td><td>6q27 (164,818,834)</td><td>6.485</td><td>9.82E-09</td></tr><tr><td>rs535586</td><td>6p21.33 (31,860,337)</td><td>rs2523589</td><td>6p21.33 (31,327,334)</td><td>6.299</td><td><strong>1.84E-44</strong></td></tr><tr><td>rs2523589</td><td>6p21.33 (31,327,334)</td><td>rs659445</td><td>6p21.33 (31,864,304)</td><td>6.268</td><td><strong>4.08E-45</strong></td></tr><tr><td>rs408359</td><td>6p21.32 (32,141,883)</td><td>rs4151664</td><td>6p21.33 (31,920,873)</td><td>6.038</td><td><strong>4.30E-21</strong></td></tr><tr><td>rs2164182</td><td>chr11q21 (95,981,029)</td><td>rs16864296</td><td>1q24.3 (171,236,326)</td><td>5.945</td><td>8.34E-08</td></tr><tr><td>rs2227956</td><td>6p21.32 (31,778,272)</td><td>rs2523589</td><td>6p21.33 (31,327,334)</td><td>5.851</td><td><strong>1.39E-42</strong></td></tr><tr><td>rs12050395</td><td>14q31.3 (86,210,504)</td><td>rs2301092</td><td>5q14.3 (83,363,112)</td><td>5.831</td><td>1.67E-08</td></tr><tr><td>rs12663103</td><td>6p21.32 (32,161,324)</td><td>rs9267649</td><td>6p21.33 (31,824,828)</td><td>5.827</td><td><strong>4.99E-15</strong></td></tr><tr><td>rs535586</td><td>6p21.33 (31,860,337)</td><td>rs12663103</td><td>6p21.32 (32,161,324)</td><td>5.810</td><td>6.66E-11</td></tr><tr><td>rs9267532</td><td>6p21.33 (31,639,979)</td><td>rs9267487</td><td>6p21.33 (31,511,350)</td><td>5.806</td><td><strong>6.48E-19</strong></td></tr><tr><td>rs9267487</td><td>6p21.33 (31,511,350)</td><td>rs9501587</td><td>6p21.33 (31,346,937)</td><td>5.804</td><td><strong>1.75E-24</strong></td></tr><tr><td>rs12663103</td><td>6p21.32 (32,161,324)</td><td>rs3130637</td><td>6p21.33 (31,488,145)</td><td>5.800</td><td><strong>4.52E-16</strong></td></tr><tr><td>rs2948369</td><td>8p22 (12,736,387)</td><td>rs4077920</td><td>8q22.1 (98,893,864)</td><td>5.800</td><td>6.05E-09</td></tr></tbody></table>

Wu et al. [38] introduced a haplotype-based measure based on the following term:

$$I_{GH} = \frac{OR_{G_1H_1}}{OR_{G_1H_2} OR_{G_2H_1}}$$

where $OR_{G_1H_1}$ is the odds ratio for both risk haplotypes when carried together, compared to the baseline

haplotypes; $OR_{G_1H_2}$ and $OR_{G_2H_1}$ are the odds ratios for each risk haplotype, respectively, compared to the baseline haplotype.

Although both methods are based on odds ratios, the methods differ in several respects. First, and most significantly, Wu's method uses haplotypes, which typically require the statistical inference of haplotypes. Even though this design was shown to be better powered than classical

<table><caption>Table 8 Most significant interactions detected through exhaustive search after main effect SNPs removal</caption><thead><tr><th colspan="2">rs number</th><th colspan="2">Chromosome</th><th colspan="2">Position</th><th colspan="2">Marginal effect</th><th rowspan="2">p-value<sup>a</sup></th><th rowspan="2">I<sub>OR</sub></th><th rowspan="2">R<sup>2b</sup></th></tr><tr><th>SNP1</th><th>SNP2</th><th>SNP1</th><th>SNP2</th><th>SNP1</th><th>SNP2</th><th colspan="2">p-value</th></tr><tr><th></th><th></th><th></th><th></th><th></th><th></th><th>SNP1</th><th>SNP2</th><th></th><th></th><th></th></tr></thead><tbody><tr><td>rs2254556</td><td>rs9267532</td><td>6</td><td>6</td><td>31,374,854</td><td>31,672,202</td><td>0.008</td><td>0.076</td><td><strong>1.22E-22</strong></td><td>5.23</td><td>0.002</td></tr><tr><td>rs9267532</td><td>rs2523518</td><td>6</td><td>6</td><td>31,672,202</td><td>31,373,351</td><td>0.076</td><td>0.006</td><td><strong>3.15E-22</strong></td><td>5.15</td><td>0.002</td></tr><tr><td>rs2596437</td><td>rs9267532</td><td>6</td><td>6</td><td>31,371,309</td><td>31,672,202</td><td>0.006</td><td>0.076</td><td><strong>7.56E-22</strong></td><td>5.1</td><td>0.002</td></tr></tbody></table>

<footnote><sup>a</sup>Bold data are genome-wide significant interactions.</footnote>

<footnote><sup>a</sup>Bold data are genome-wide significant interactions. <sup>b</sup>R<sup>2</sup> were calculated using controls only.</footnote>
<figure><img src="image_1.png" /><figcaption>**Figure 1** Power of different approaches to detect simulated epistatic interactions across the six epistasis models by Ritchie [28]. Purple: FORCE – Green: MB-MDR – Blue: PLINK Epistasis – Red: PLINK FastEpistasis. Refer to Table 3 for the definitions of the 6 interaction models.</figcaption></figure>

<figure><img src="image_2.png" /><figcaption>**Figure 2** Power of different approaches to detect simulated epistatic interactions across the six epistasis models by Ritchie [28], in the presence of noise. Comparison of the power of four methods to detect interaction in the presence of one source of noise. GH: Genetic heterogeneity – GE: Genotyping errors – MS: Missing data – PC: Phenocopy. When GH is simulated, three different ways of calculating power are employed: the power of detecting both pairs in the same dataset, the power of detecting the first (fixed) pair and the power to detect either of the two epistatic pairs. Purple: FORCE – Green: MB-MDR – Blue: PLINK Epistasis – Red: PLINK FastEpistasis.</figcaption></figure>
<table><caption>Table 9 Power and family-wise error rate (FWER) of FORCE, MBMDR, Plink Epistasis and Plink FastEpistasis on 6 epistasis models with or without noise</caption><thead><tr><th></th><th></th><th></th><th>Model 1</th><th>Model 2</th><th>Model 3</th><th>Model 4</th><th>Model 5</th><th>Model 6</th></tr></thead><tbody><tr><td rowspan="6">No noise</td><td rowspan="2">FORCE</td><td>Power<sup>a</sup></td><td>1</td><td>0.97</td><td>0.96</td><td>1</td><td>0.93</td><td>0.99</td></tr><tr><td>FWER<sup>b</sup></td><td>0.05</td><td>0.02</td><td>0.02</td><td>0.06</td><td>0.05</td><td>0.04</td></tr><tr><td rowspan="2">MBMDR</td><td>Power</td><td>1</td><td>1</td><td>1</td><td>0.98</td><td>0.87</td><td>0.97</td></tr><tr><td>FWER</td><td>0.02</td><td>0</td><td>0</td><td>0.01</td><td>0.01</td><td>0.01</td></tr><tr><td rowspan="2">Plink Epistasis</td><td>Power</td><td>0</td><td>1</td><td>0.32</td><td>0.98</td><td>0.9</td><td>0.97</td></tr><tr><td>FWER</td><td>0.04</td><td>0.05</td><td>0.02</td><td>0.02</td><td>0</td><td>0.03</td></tr><tr><td rowspan="6">GE</td><td rowspan="2">FORCE</td><td>Power</td><td>0.07</td><td>0.07</td><td>0.02</td><td>0.02</td><td>0.01</td><td>0.05</td></tr><tr><td>FWER</td><td>0.03</td><td>0.03</td><td>0.08</td><td>0.03</td><td>0.03</td><td>0.04</td></tr><tr><td rowspan="2">MBMDR</td><td>Power</td><td>1</td><td>1</td><td>1</td><td>0.99</td><td>0.85</td><td>1</td></tr><tr><td>FWER</td><td>0</td><td>0</td><td>0.01</td><td>0.02</td><td>0</td><td>0</td></tr><tr><td rowspan="2">Plink Epistasis</td><td>Power</td><td>0</td><td>1</td><td>0.28</td><td>0.99</td><td>0.87</td><td>0.99</td></tr><tr><td>FWER</td><td>0</td><td>0</td><td>0.07</td><td>0.02</td><td>0</td><td>0</td></tr><tr><td rowspan="6">MS</td><td rowspan="2">FORCE</td><td>Power</td><td>0.01</td><td>1</td><td>0.31</td><td>1</td><td>0.74</td><td>0.92</td></tr><tr><td>FWER</td><td>0.04</td><td>0</td><td>0.09</td><td>0.05</td><td>0</td><td>0</td></tr><tr><td rowspan="2">MBMDR</td><td>Power</td><td>0.07</td><td>0.02</td><td>0.02</td><td>0.03</td><td>0.03</td><td>0.06</td></tr><tr><td>FWER</td><td>1</td><td>1</td><td>1</td><td>0.99</td><td>0.8</td><td>0.94</td></tr><tr><td rowspan="2">Plink Epistasis</td><td>Power</td><td>0</td><td>1</td><td>0.26</td><td>1</td><td>0.8</td><td>0.97</td></tr><tr><td>FWER</td><td>0.03</td><td>0.02</td><td>0.08</td><td>0.02</td><td>0</td><td>0</td></tr><tr><td rowspan="6">PC</td><td rowspan="2">FORCE</td><td>Power</td><td>0.06</td><td>0.01</td><td>0.1</td><td>0.04</td><td>0</td><td>0</td></tr><tr><td>FWER</td><td>0.08</td><td>0.05</td><td>0.13</td><td>0.33</td><td>0.23</td><td>0.37</td></tr><tr><td rowspan="2">MBMDR</td><td>Power</td><td>0.03</td><td>0.05</td><td>0.05</td><td>0.06</td><td>0.04</td><td>0.07</td></tr><tr><td>FWER</td><td>0.72</td><td>0.98</td><td>0.12</td><td>0.09</td><td>0.11</td><td>0.16</td></tr><tr><td rowspan="2">Plink Epistasis</td><td>Power</td><td>0</td><td>0.95</td><td>0.01</td><td>0.21</td><td>0.19</td><td>0.28</td></tr><tr><td>FWER</td><td>0.04</td><td>0.03</td><td>0.05</td><td>0.06</td><td>0.01</td><td>0</td></tr><tr><td rowspan="6">PC</td><td rowspan="2">FORCE</td><td>Power</td><td>0.07</td><td>0.03</td><td>0.04</td><td>0.06</td><td>0.01</td><td>0.01</td></tr><tr><td>FWER</td><td>0.07</td><td>0.03</td><td>0.04</td><td>0.06</td><td>0.01</td><td>0.01</td></tr></tbody></table>

Genotype errors (GE), missing data (MS) or phenocopy (PC). <sup>a</sup>In bold, power higher than 50%. <sup>b</sup>In bold, FWER lower than 5%.

genotype-based statistics, the additional calculations are computationally costly. As a result, FORCE can perform an exhaustive genome-wide epistasis search in a few days on a single compute core while, in practice, Wu's method only allows a limited number of SNP pairs to be tested.

In addition to the different statistics themselves, the approaches to calculating significance differ. FORCE relies on an exact P-value that requires too much time to be calculated exhaustively for all SNP pairs. Instead, P-values are calculated only for pairs with the highest I<sub>OR</sub>. Conversely, Wu et al. used an approximate, chi-

square distribution-based, P-value which can be applied to each investigated pair of the search.

Our study on WTCCC psoriasis data suggests that the computational effort for exhaustive testing is currently not just a luxury. The popular class of conditional analyses focuses only on possible interactions of previously implicated SNPs – often the only option to perform large-scale analysis in reasonable time. When comparing conditional and exhaustive FORCE analyses, we found that the conditional approach only detects interactions of vastly weaker statistical significance.
<table><caption>Table 10 Power and family-wise error rate (FWER) of FORCE, MBMDR, Plink Epistasis and Plink FastEpistasis on 6 epistasis models without noise or with simulated genetic heterogeneity (GH)</caption><thead><tr><th rowspan="2"></th><th rowspan="2"></th><th colspan="3">Model 1</th><th colspan="3">Model 2</th><th colspan="3">Model 3</th><th colspan="3">Model 4</th><th colspan="3">Model 5</th><th colspan="3">Model 6</th></tr><tr><th>Both</th><th>First</th><th>Either</th><th>Both</th><th>First</th><th>Either</th><th>Both</th><th>First</th><th>Either</th><th>Both</th><th>First</th><th>Either</th><th>Both</th><th>First</th><th>Either</th><th>Both</th><th>First</th><th>Either</th></tr></thead><tbody><tr><td rowspan="2">FORCE</td><td>Power<sup>a</sup></td><td>0.01</td><td>0.08</td><td>0.14</td><td>0.38</td><td>0.6</td><td><strong>0.82</strong></td><td>0.03</td><td>0.19</td><td>0.34</td><td>0.16</td><td>0.39</td><td><strong>0.62</strong></td><td>0.04</td><td>0.21</td><td>0.38</td><td>0.1</td><td>0.34</td><td><strong>0.57</strong></td></tr><tr><td>FWER<sup>b</sup></td><td></td><td></td><td></td><td>0.07</td><td></td><td></td><td>0.07</td><td></td><td></td><td><strong>0.04</strong></td><td></td><td></td><td><strong>0.02</strong></td><td></td><td></td><td></td><td><strong>0.02</strong></td><td></td><td></td></tr><tr><td rowspan="2">MBMDR</td><td>Power</td><td><strong>0.75</strong></td><td><strong>0.86</strong></td><td><strong>0.97</strong></td><td><strong>0.96</strong></td><td><strong>0.98</strong></td><td>1</td><td>0</td><td>0.09</td><td>0.17</td><td>0.01</td><td>0.07</td><td>0.13</td><td>0</td><td>0.07</td><td>0.13</td><td>0.03</td><td>0.16</td><td>0.28</td></tr><tr><td>FWER</td><td></td><td></td><td></td><td>0</td><td></td><td></td><td>0</td><td></td><td></td><td>0</td><td></td><td></td><td>0</td><td></td><td></td><td>0</td><td></td><td></td><td>0</td></tr><tr><td rowspan="2">Plink Epistasis</td><td>Power</td><td>0</td><td>0</td><td>0</td><td><strong>0.91</strong></td><td><strong>0.96</strong></td><td>1</td><td>0</td><td>0.02</td><td>0.03</td><td>0.05</td><td>0.29</td><td><strong>0.52</strong></td><td>0</td><td>0.13</td><td>0.26</td><td>0.02</td><td>0.25</td><td>0.47</td></tr><tr><td>FWER</td><td><strong>0.01</strong></td><td></td><td></td><td><strong>0.02</strong></td><td></td><td></td><td><strong>0.01</strong></td><td></td><td></td><td><strong>0.05</strong></td><td></td><td></td><td>0</td><td></td><td></td><td>0</td><td></td><td></td><td>0</td></tr><tr><td rowspan="2">FastEpistasis</td><td>Power</td><td>0</td><td>0</td><td>0</td><td><strong>0.96</strong></td><td><strong>0.98</strong></td><td>1</td><td>0</td><td>0.02</td><td>0.04</td><td>0.08</td><td>0.31</td><td><strong>0.54</strong></td><td>0</td><td>0.07</td><td>0.14</td><td>0.02</td><td>0.18</td><td>0.34</td></tr><tr><td>FWER</td><td><strong>0.02</strong></td><td></td><td></td><td><strong>0.03</strong></td><td></td><td></td><td><strong>0.01</strong></td><td></td><td></td><td>0.07</td><td></td><td></td><td><strong>0.01</strong></td><td></td><td></td><td>0</td><td></td><td></td><td>0</td></tr></tbody></table>

<footnote>ªIn bold, power higher than 50%. ᵇIn bold, FWER lower than 5%.</footnote>
<table><caption>Table 11 Power of FORCE detection method, impact of various sources of noise and combinations of them for the 6 epistatic models</caption><thead><tr><th>Type of noise</th><th>Model 1<sup>a</sup></th><th>Model 2</th><th>Model 3</th><th>Model 4</th><th>Model 5</th><th>Model 6</th></tr></thead><tbody><tr><td>No noise</td><td><strong>1</strong></td><td><strong>0.97</strong></td><td><strong>0.96</strong></td><td><strong>1</strong></td><td><strong>0.93</strong></td><td><strong>0.99</strong></td></tr><tr><td>Genotype errors (GE)</td><td><strong>0.99</strong></td><td><strong>0.99</strong></td><td><strong>0.97</strong></td><td><strong>1</strong></td><td><strong>0.95</strong></td><td><strong>0.99</strong></td></tr><tr><td>Phenocopy (PC)</td><td>0.08</td><td><strong>0.55</strong></td><td>0.13</td><td>0.33</td><td>0.23</td><td>0.37</td></tr><tr><td>Missing data (MS)</td><td><strong>0.99</strong></td><td><strong>0.96</strong></td><td><strong>0.95</strong></td><td><strong>1</strong></td><td><strong>0.93</strong></td><td><strong>0.99</strong></td></tr><tr><td>GE + PC</td><td>0.05</td><td><strong>0.62</strong></td><td>0.18</td><td>0.3</td><td>0.31</td><td>0.35</td></tr><tr><td>GE + MS</td><td><strong>0.95</strong></td><td><strong>0.98</strong></td><td><strong>0.96</strong></td><td><strong>1</strong></td><td><strong>0.91</strong></td><td><strong>0.99</strong></td></tr><tr><td>PC + MS</td><td>0.06</td><td><strong>0.52</strong></td><td>0.21</td><td>0.31</td><td>0.21</td><td>0.26</td></tr><tr><td>GE + PC + MS</td><td>0.09</td><td><strong>0.55</strong></td><td>0.21</td><td>0.46</td><td>0.13</td><td>0.35</td></tr><tr><td></td><td><strong>both</strong></td><td><strong>first</strong></td><td><strong>either</strong></td><td><strong>both</strong></td><td><strong>first</strong></td><td><strong>either</strong></td><td><strong>both</strong></td><td><strong>first</strong></td><td><strong>either</strong></td><td><strong>both</strong></td><td><strong>first</strong></td><td><strong>either</strong></td></tr><tr><td>Genetic heterogeneity (GH)</td><td>0.01</td><td>0.08</td><td>0.14</td><td>0.38</td><td>0.6</td><td><strong>0.82</strong></td><td>0</td><td>0.19</td><td>0.34</td><td>0.2</td><td>0.39</td><td><strong>0.62</strong></td><td>0</td><td>0.21</td><td>0.38</td><td>0.1</td><td>0.34</td><td><strong>0.57</strong></td></tr><tr><td>GH + GE</td><td>0.01</td><td>0.09</td><td>0.16</td><td>0.34</td><td>0.6</td><td><strong>0.85</strong></td><td>0.03</td><td>0.18</td><td>0.32</td><td>0.17</td><td>0.4</td><td><strong>0.62</strong></td><td>0.04</td><td>0.23</td><td>0.41</td><td>0.14</td><td>0.31</td><td><strong>0.50</strong></td></tr><tr><td>GH + PC</td><td>0</td><td>0.015</td><td>0.03</td><td>0.02</td><td>0.09</td><td>0.16</td><td>0</td><td>0.01</td><td>0.02</td><td>0</td><td>0.02</td><td>0.04</td><td>0.01</td><td>0.035</td><td>0.06</td><td>0.01</td><td>0.04</td><td>0.07</td></tr><tr><td>GH + MS</td><td>0.01</td><td>0.04</td><td>0.07</td><td>0.37</td><td>0.57</td><td><strong>0.77</strong></td><td>0.02</td><td>0.145</td><td>0.27</td><td>0.18</td><td>0.385</td><td><strong>0.59</strong></td><td>0.03</td><td>0.19</td><td>0.35</td><td>0.07</td><td>0.28</td><td><strong>0.50</strong></td></tr><tr><td>GH + GE + PC</td><td>0</td><td>0.01</td><td>0.02</td><td>0.03</td><td>0.105</td><td>0.18</td><td>0</td><td>0.02</td><td>0.04</td><td>0</td><td>0.025</td><td>0.05</td><td>0</td><td>0.025</td><td>0.05</td><td>0</td><td>0.03</td><td>0.06</td></tr><tr><td>GH + GE + MS</td><td>0</td><td>0.05</td><td>0.1</td><td>0.33</td><td>0.665</td><td><strong>0.80</strong></td><td>0.02</td><td>0.155</td><td>0.29</td><td>0.13</td><td>0.385</td><td><strong>0.64</strong></td><td>0.03</td><td>0.23</td><td>0.43</td><td>0.13</td><td>0.305</td><td>0.48</td></tr><tr><td>GH + PC + MS</td><td>0</td><td>0.005</td><td>0.01</td><td>0.01</td><td>0.095</td><td>0.18</td><td>0</td><td>0.025</td><td>0.05</td><td>0</td><td>0.04</td><td>0.08</td><td>0</td><td>0.005</td><td>0.01</td><td>0</td><td>0.035</td><td>0.07</td></tr><tr><td>GH + GE + PC + MS</td><td>0</td><td>0.015</td><td>0.03</td><td>0.01</td><td>0.08</td><td>0.15</td><td>0</td><td>0.015</td><td>0.03</td><td>0</td><td>0.035</td><td>0.07</td><td>0</td><td>0.015</td><td>0.03</td><td>0</td><td>0.03</td><td>0.06</td></tr></tbody></table>

GE: Genotyping errors – GH: Genetic heterogeneity – MS: Missing data – PC: Phenocopy. In case of GH, power is calculated in 3 different ways as the proportion of datasets in which both, the first or either of the interacting pairs are detected. <sup>a</sup>In bold, power higher than 50%.

<table><caption>Table 12 Family-wise error rate (FWER) of FORCE for the 6 epistatic models and 16 noise conditions tested</caption><thead><tr><th>Family-wise error rate</th><th>Model 1<sup>a</sup></th><th>Model 2</th><th>Model 3</th><th>Model 4</th><th>Model 5</th><th>Model 6</th></tr></thead><tbody><tr><td>No noise</td><td>0.05</td><td>0.02</td><td>0.02</td><td><strong>0.06</strong></td><td>0.05</td><td>0.04</td></tr><tr><td>Genotype errors (GE)</td><td>0.03</td><td>0.03</td><td><strong>0.08</strong></td><td>0.03</td><td>0.03</td><td>0.04</td></tr><tr><td>Genetic heterogeneity (GH)</td><td>0.02</td><td><strong>0.07</strong></td><td><strong>0.07</strong></td><td>0.04</td><td>0.02</td><td>0.02</td></tr><tr><td>Phenocopy (PC)</td><td>0.03</td><td>0.05</td><td>0.05</td><td><strong>0.06</strong></td><td>0.04</td><td><strong>0.07</strong></td></tr><tr><td>Missing data (MS)</td><td><strong>0.07</strong></td><td>0.02</td><td>0.02</td><td>0.03</td><td>0.03</td><td><strong>0.06</strong></td></tr><tr><td>GE + GH</td><td>0.05</td><td><strong>0.07</strong></td><td>0.04</td><td>0.03</td><td>0.01</td><td><strong>0.07</strong></td></tr><tr><td>GE + PC</td><td>0.05</td><td>0.05</td><td>0.01</td><td>0.02</td><td>0.03</td><td>0.02</td></tr><tr><td>GE + MS</td><td>0.02</td><td>0.01</td><td><strong>0.06</strong></td><td>0.04</td><td>0.04</td><td><strong>0.07</strong></td></tr><tr><td>GH + PC</td><td>0.05</td><td>0.05</td><td>0.03</td><td>0.02</td><td>0.03</td><td>0.03</td></tr><tr><td>GH + MS</td><td>0.04</td><td><strong>0.07</strong></td><td>0.05</td><td>0.03</td><td>0.03</td><td>0.01</td></tr><tr><td>PC + MS</td><td>0.03</td><td>0.02</td><td>0.03</td><td><strong>0.06</strong></td><td>0.04</td><td>0.03</td></tr><tr><td>GE + GH + PC</td><td><strong>0.07</strong></td><td>0.03</td><td><strong>0.06</strong></td><td>0.03</td><td>0.02</td><td>0.04</td></tr><tr><td>GE + GH + MS</td><td><strong>0.07</strong></td><td>0.05</td><td>0.04</td><td>0.01</td><td>0.05</td><td>0.05</td></tr><tr><td>GH + PC + MS</td><td>0.04</td><td>0.02</td><td><strong>0.06</strong></td><td>0.02</td><td>0.03</td><td>0.03</td></tr><tr><td>GE + PC + MS</td><td>0.05</td><td><strong>0.08</strong></td><td><strong>0.06</strong></td><td><strong>0.06</strong></td><td>0.05</td><td>0.05</td></tr><tr><td>GE + GH + PC + MS</td><td>0.02</td><td><strong>0.07</strong></td><td><strong>0.06</strong></td><td>0.04</td><td>0.05</td><td><strong>0.06</strong></td></tr></tbody></table>

GE: Genotyping errors – GH: Genetic heterogeneity – MS: Missing data – PC: Phenocopy. <sup>a</sup>In bold, FWER > 0.05.
Our systematic study on small simulated datasets indicates that FORCE not only “goes farther” than existing approaches because of its exhaustive search, but also detects fundamentally different types of interactions, in particular in the biologically more relevant models 3–6. In two out of six models of epistatic interaction described by Ritchie [28], and across the different sources of noise in the data, FORCE consistently displayed a good power of detection compared to other approaches. Interestingly, each of the four approaches is always less efficient than another for at least one model associated with one type of noise.

Finally, by applying FORCE to WTCCC psoriasis data, we were able to detect statistical interactions between SNPs in the HLA region, even after the exclusion of all SNPs with main effects. To our knowledge this constitutes the first demonstration that the genetic structure of the HLA region cannot be understood by the analysis of main effects alone and that more than one interacting locus exists in that region.

## Conclusions

Together, the different elements of our study suggest that FORCE represents a valuable new addition to the arsenal of genome-wide epistasis detection approaches for case-control studies. As with other approaches, the additionally detected interactions are a priori of a statistical nature, and require detailed analysis and follow-up.

Beyond this, our study has provided an example for the need for exhaustive epistasis analysis. In the future, exhaustive analysis will be facilitated by the ever-increasing computational power available to biological research. On one hand, this may enable the exhaustive calculation of FORCE P-values, which can be expected to lead to a potentially much enlarged set of statistically significant interactions. On the other hand, more computational power, as well as algorithmic improvements, may also render exhaustive analysis under those models of interactions feasible for which running times are prohibitive today. Finally, we believe that these improvements are necessary for the integration of different types of interactions and other types of large-scale data, which may ultimately be key to understanding the genetic basis of complex diseases.

## Additional file

**Additional file 1: Table S1.** Epistasis analysis among GWAS hits: all 153 pairs of the conditional search. **Table S2.** FORCE Exhaustive search top 100 hits on psoriasis data.

### Abbreviations

OR: Odds ratio; I<sub>OR</sub>: Interaction odds ratio; HLA: Human leukocyte antigen; GWAS: Genome wide association study; SNP: Single nucleotide polymorphism; WTCCC: The welcome trust case control consortium;

GE: Genotyping error; MS: Missing data; GH: Genetic heterogeneity; PC: Phenocopy; FWER: Family-wise error rate.

### Competing interests

The authors declare that they have no competing interests.

### Authors' contributions

LG participated in the study design, carried out the analyses, interpreted results and drafted the manuscript. JFB participated in the study design, interpreted result and drafted the manuscript. IN contributed to the program, RP participated in interpreting results and was involved in drafting and revising the manuscript, KVS participated in the study design and revising the manuscript, BS conceived and implemented the FORCE method, analyzed and interpreted data, and was involved in drafting and revising the manuscript, AS conceived of the study, participated in its design and coordination and helped to draft the manuscript. All authors read and approved the final manuscript.

### Acknowledgements

We would like to thank the WTCCC for permission to use the psoriasis genome-wide data set. This article is linked to a project funded by “ANR-11-BSV1-027-01”. LG is supported by Ministère de la Recherche of France, IN is supported by the French Government’s Investissement d’Avenir program, Laboratoire d’Excellence “Integrative Biology of Emerging Infectious Diseases” (grant n°ANR-10-LABX-62-IBEID).

### Author details

¹Department of Genomes and Genetics, Institut Pasteur, Functional Genetics of Infectious Diseases Unit, Paris 75015, France. ²CNRS URA3012, Paris 75015, France. ³Université Paris Diderot, Paris 75013, France. ⁴Department of Genomes and Genetics, Institut Pasteur, Systems Biology Lab, Paris 75015, France. ⁵Université Paris-Descartes, Sorbonne Paris Cité, Paris, France.

⁶Systems and Modeling Unit, Montefiore institute, University of Liège, Liège, Belgium. ⁷Bioinformatics and Modeling, GiGA-R, University of Liège, Liège, Belgium.

Received: 29 September 2014 Accepted: 23 January 2015

Published online: 06 February 2015

### References

1. Visscher PM, Brown MA, McCarthy MI, Yang J. Five years of GWAS discovery. Am J Hum Genet. 2012;90(1):7–24.

2. Welter D, MacArthur J, Morales J, Burdett T, Hall P, Junkins H, et al. The NHGRI GWAS Catalog, a curated resource of SNP-trait associations. Nucleic Acids Res. 2014;42 part 1:1–100.

3. Manolio TA, Collins FS, Cox NJ, Goldstein DB, Hindorff LA, Hunter DJ, et al. Finding the missing heritability of complex diseases. Nature. 2009;461 (7265):747–53.

4. Mackay TF. Epistasis and quantitative traits: using model organisms to study gene-gene interactions. Nat Rev Genet. 2014;15(1):22–33.

5. Steen KV. Travelling the world of gene-gene interactions. Brief Bioinform. 2012;13(1):1–19.

6. Wei WH, Hemani G, Haley CS. Detecting epistasis in human complex traits. Nat Rev Genet. 2014;15(11):722–33.

7. Purcell S, Neale B, Todd-Brown K, Thomas L, Ferreira MA, Bender D, et al. PLINK: a tool set for whole-genome association and population-based linkage analyses. Am J Hum Genet. 2007;81(3):559–75.

8. Schupbach T, Xenarios I, Bergmann S, Kapur K. FastEpistasis: a high performance computing solution for quantitative trait epistasis. Bioinformatics. 2010;26(11):1468–9.

9. Hemani G, Theocharidis A, Wei W, Haley C. EpiGPU: exhaustive pairwise epistasis scans parallelized on consumer level graphics cards. Bioinformatics. 2011;27(11):1462–5.

10. Lange K, Papp JC, Sinsheimer JS, Sripracha R, Zhou H, Sobel EM. Mendel: the Swiss army knife of genetic analysis programs. Bioinformatics. 2013;29(12):1568–70.

11. Wan X, Yang C, Yang Q, Xue H, Fan X, Tang NL, et al. BOOST: A fast approach to detecting gene-gene interactions in genome-wide case-control studies. Am J Hum Genet. 2010;87(3):325–40.
12. Kam-Thong T, Czamara D, Tsuda K, Borgwardt K, Lewis CM, Erhardt-Lehmann A, et al. EPIBLASTER-fast exhaustive two-locus epistasis detection strategy using graphical processing units. Eur J Hum Genet. 2011;19(4):465–71.

13. Prabhu S, Pe'er I. Ultrafast genome-wide scan for SNP-SNP interactions in common complex disease. Genome Res. 2012;22(11):2230–40.

14. Yi N, Kaklamani VG, Pasche B. Bayesian analysis of genetic interactions in case-control studies, with application to adiponectin genes and colorectal cancer risk. Ann Hum Genet. 2011;75(1):90–104.

15. Zhang Y, Liu JS. Bayesian inference of epistatic interactions in case-control studies. Nat Genet. 2007;39(9):1167–73.

16. Luz Calle ML, Urrea V, Van Steen K. MB-MDR: Model-Based Multifactor Dimensionality Reduction for detecting interactions in high-dimensional genomic data. Vic, Spain: Universitat de Vic; 2008.

17. Schwarz DF, Konig IR, Ziegler A. On safari to Random Jungle: a fast implementation of Random Forests for high-dimensional data. Bioinformatics. 2010;26(14):1752–8.

18. Ueki M, Tamiya G. Ultrahigh-dimensional variable selection method for whole-genome gene-gene interaction analysis. BMC Bioinformatics. 2012;13:72.

19. Xie M, Li J, Jiang T. Detecting genome-wide epistases based on the clustering of relatively frequent items. Bioinformatics. 2012;28(1):5–12.

20. Piegorsch WW, Weinberg CR, Taylor JA. Non-hierarchical logistic models and case-only designs for assessing susceptibility in population-based case-control studies. Stat Med. 1994;13(2):153–62.

21. Thomas D. Gene-environment-wide association studies: emerging approaches. Nat Rev Genet. 2010;11(4):259–72.

22. VanderWeele TJ, Hernandez-Diaz S, Hernan MA. Case-only gene-environment interaction studies: when does association imply mechanistic interaction? Genet Epidemiol. 2010;34(4):327–34.

23. Knight J, Spain SL, Capon F, Hayday A, Nestle FO, Clop A, et al. Conditional analysis identifies three novel major histocompatibility complex loci associated with psoriasis. Hum Mol Genet. 2012;21(23):S185–92.

24. Strange A, Capon F, Spencer CC, Knight J, Weale ME, Allen MH, et al. A genome-wide association study identifies new psoriasis susceptibility loci and an interaction between HLA-C and ERAP1. Nat Genet. 2010;42(11):985–90.

25. Chandran V. The genetics of psoriasis and psoriatic arthritis. Clin Rev Allergy Immunol. 2013;44(2):149–56.

26. Oka A, Mabuchi T, Ozawa A, Inoko H. Current understanding of human genetics and genetic analysis of psoriasis. J Dermatol. 2012;39(3):231–41.

27. Ziegler A, Konig IR, Thompson JR. Biostatistical aspects of genome-wide association studies. Biom J. 2008;50(1):8–28.

28. Ritchie MD, Hahn LW, Moore JH. Power of multifactor dimensionality reduction for detecting gene-gene interactions in the presence of genotyping error, missing data, phenocopy, and genetic heterogeneity. Genet Epidemiol. 2003;24(2):150–7.

29. Moore JH, Hahn LW, Ritchie MD, Thornton TA, White BC. Application of Genetic Algorithms to the Discovery of Complex Models for Simulation Studies in Human Genetics. Proc Genet Evol Comput Conf. 2002;2002:1150–5.

30. Kernighan BW, Ritchie DM. The C programming language. Second edition, ed. Prentice Hall; 1988.

31. Cattaert T, Calle ML, Dudek SM, Mahachie John JM, Van Lishout F, Urrea V, et al. Model-based multifactor dimensionality reduction for detecting epistasis in case-control data in the presence of noise. Ann Hum Genet. 2011;75(1):78–89.

32. Cordell HJ. Detecting gene-gene interactions that underlie human diseases. Nat Rev Genet. 2009;10(6):392–404.

33. Goudey B, Rawlinson D, Wang Q, Shi F, Ferra H, Campbell RM, et al. GWIS-model-free, fast and exhaustive search for epistatic interactions in case-control GWAS. BMC Genomics. 2013;14 Suppl 3:S10.

34. Riveira-Munoz E, He SM, Escaramis G, Stuart PE, Huffmeier U, Lee C, et al. Meta-analysis confirms the LCE3C\_LCE3B deletion as a risk factor for psoriasis in several ethnic groups and finds interaction with HLA-Cw6. J Invest Dermatol. 2011;131(5):1105–9.

35. Veal CD, Clough RL, Barber RC, Mason S, Tillman D, Ferry B, et al. Identification of a novel psoriasis susceptibility locus at 1p and evidence of epistasis between PSORS1 and candidate loci. J Med Genet. 2001;38(1):7–13.

36. Ueki M, Cordell HJ. Improved statistics for genome-wide interaction analysis. PLoS Genet. 2012;8(4):e1002625.

37. Mahachie John JM, Van Lishout F, Van Steen K. Model-Based Multifactor Dimensionality Reduction to detect epistasis for quantitative traits in the presence of error-free and noisy data. Eur J Hum Genet. 2011;19(6):696–703.

38. Wu X, Dong H, Luo L, Zhu Y, Peng G, Reveille JD, et al. A novel statistic for genome-wide interaction analysis. PLoS Genet. 2010;6(9):e1001131.

Submit your next manuscript to BioMed Central and take full advantage of:

• Convenient online submission

• Thorough peer review

• No space constraints or color figure charges

• Immediate publication on acceptance

• Inclusion in PubMed, CAS, Scopus and Google Scholar

• Research which is freely available for redistribution

Submit your manuscript at
www.biomedcentral.com/submit

BioMed Central