The novel coronavirus disease, Covid-19, caused by the virus SARS-coV-2 has caused a major pandemic worldwide. No vaccines or specific treatment were available until June 2020. Many existing medicines have been tested to treat patients, mostly in hospitalized patients under more severe clinical conditions. Hydroxychloroquine is the most discussed of these drugs and several studies have pointed to different outcomes. 3 Although some randomized clinical trials have shown the inefficacy of hydroxychloroquine treatment to prevent death in hospitalized patients, 4 the benefits to less severe patients in the beginning of the disease and as a pre-exposure or post-exposure prophylaxis is still under discussion. Boulware et al. 1 was the first randomized clinical trial which has tested hydroxychloroquine as a prophylaxis treatment to . The authors of the study concluded there is no statistical evidence of hydroxychloroquine's efficacy when compared to placebo results. Besides some limitations in their experimental design, as pointed by Cohen, 2 up to 23th of June 2020, when our study was finished, their results were the most reliable information available about hydroxychloroquine treatment for Covid-19 as a prophylaxis. However, correctable inaccuracies in their estimates remained without revision.
In the section Methods, we discuss some of these inaccuracies and present the corrections. Furthermore, in the section Results, we provide an original analysis of a more detailed time series data, available at their supplementary appendix. 1 The randomized trial of Boulware et al. 1 aims to test the treatment of Covid-19 infected patients for 5 days with hydroxychloroquine as a prophylaxis, measuring its effect by a possible reduction on the incidence of symptomatic outcomes when compared to results from a placebo group. Adult patients who had epidemiological linkage with Covid-19 confirmed patients were included if exposure was within 4 days at the beginning of the experiment. Initially 921 asymptomatic participants were randomly assigned to treatment or placebo groups but 100 presented symptoms at the day of beginning of experiment (day 1) and were excluded. The primary measure of effect was incidence of Covid-19 compatible symptoms with laboratory or clinical confirmation within 14 days. In the placebo group, 58 of 407 participants (14.3%) presented symptoms from day 2 to day 14, while 49 of 414 participants (11.8%) of the treatment group presented symptoms in the same period. No serious adverse reactions were reported. A two-tailed Fisher exact test was used to obtain a p-value of 0.351 and at 95% confidence level, they have concluded hydroxychloroquine did not prevent illness compatible with Covid-19 when used as a postexposure prophylaxis. Next, we make some considerations on their statistical analysis to corroborate with the different choices we make at section Results. Limitations on the design of their experiment are discussed by Cohen. 2
The drug in test is known to have antiviral and anti-inflammatory activities and to effectively inhibit SARS-coV-2 in vitro. 5 When applying a hypothesis test, the statistician responsible for the data analysis must define whether to use a two-tailed test or a one-tailed test. This definition typically depends on the alternative hypothesis to be tested. In the present problem, the question to be answered is whether the treatment is able to reduce the incidence of symptomatic patients in contrast to the null hypothesis H0 that it will be as good as placebo. Thus, a one-tailed test is the natural choice to the alternative hypothesis H1 in this problem, in contrast to the two-tailed test adopted in their study. 1 We emphasize this choice has serious implications in both interpretation and quantitative results. In the adopted two-tailed test, if the test result leads to H0 rejection, the conclusion would be that the treatment with hydroxychloroquine presents a different result than the placebo, where the true incidence could be either higher or lower. In this two-tailed case, it would not be possible to conclude directly that the rejection of H0 implies the efficacy of the treatment, despite the observed incidences favoring the hydroxychloroquine group. Another important aspect to consider is that the use of the twotailed test favors null hypothesis to be not rejected in comparison with the one-tailed test. As an example, the p-value obtained in the article by a two-tailed test is 35.1% while the p-value obtained by the same method but with a one-tailed test is 17.8% (for complete data see Boulware et al 1 ).
Precise treatment group definition is necessary to a proper conduction of the statistical analysis in a clinical trial. In Boulware et al., the treatment group has been apparently defined as the group of randomly selected patients which received hydroxychloroquine for five consecutive days in the following scheme: 800mg once, 600mg 6 to 8 hours later, then 600mg daily for 4 days. 1 Treatment effect has been calculated from the incidence differences of symptomatic patients between day 14 and day 1. Note that of 921 eligible patients on day 0, 100 became symptomatic on day 1 and were excluded. However, patients who became symptomatic on days 2, 3, 4 and 5 were included in the calculation of the treatment effect. Note this subgroup of patients, although included in the analysis by the authors, had not completed the entire treatment when the response variable (symptomatic or asymptomatic) had been defined and once a patient had symptoms it is no longer possible to change his status.
Note that a large proportion of the symptomatic patients in the study have presented first symptoms between day 2 and day 5, the period of treatment application (see Table 1 below). These patients did not have the full effect of the treatment. Using a similar logic that had made the authors exclude symptomatic patients in day 1, we should exclude patients with symptoms in days 2, 3, 4 and 5, otherwise treatment definition and statistical method should be changed. Therefore, in order to have a homogeneous sample of patients taking the same dose, an adequate analysis should measure the differences in incidences from days 14 to 5, and not from 14 to day 1. The high sensitivity of the response variable chosen by the researchers (symptomatic or asymptomatic) is another reason why people who had symptoms before finishing treatment should be excluded from measuring the effects on the test.
The sample obtained considering only patients who presented first symptoms after the fifth day is far from ideal because there can be many patients for whom a long time has passed from the day of infection to the day of the end of treatment. Unfortunately, the best way to avoid this problem should have been adopted in the design of the experiment. An alternative solution to include patients who have not used full treatment before the onset of symptoms is to use a statistical method more appropriate to a heterogeneous sample such as a regression method (see section 3).
Hence, to test the effect of a complete 5 days treatment we should compare only the incidence differences between the groups from day 14 to day 5 (patients who have presented symptoms only after the end of complete prophylaxis). Raw data are not available, but with figure 2 of their article we can obtain the following approximation 1 : 
Their study uses absolute difference of the proportions of symptomatic patients between treatment and placebo groups as its response variable. However, this variable is not explanatory because it is not robust to inclusion of asymptomatic uninfected patients. First, observe that there are many more asymptomatic than symptomatic patients in the sample (714 and 107). Second, most patients in the study were not tested, so a significant proportion of this sample should not be infected, decreasing a possible reduction in the absolute incidence of treatment effect and artificially increasing the sample size and, as a consequence, the reliability of the statistics in the study.
The real number of asymptomatic infected patients is unknown in each group, and Fisher's exact test cannot be applied to this data, because the test must be applied to symptomatic infected versus asymptomatic infected in each group, since asymptomatic uninfected are not sensitive to any treatment, no matter the treatment is effective or not.
Hence, in order to verify whether the difference between rates of symptomatic patients in the two groups is significant, the ideal sample should have included only confirmed infected patients. This when compared to placebo group in these conditions.
The treatment efficacy including only patients who had symptoms after day 5, which is the appropriate group to measure the effect of complete treatment, is -Rd = -(4.1-6.9) / 6.9 = 40.6% (see subsection 2.2, in Methods) (for complete data see Boulware et al. 1 ).
In this section, we consider the same treatment of Boulware et al, 1 (0.81%). Thus with 99% of confidence, we reject the hypothesis that the slope is greater than or equal to 0. This is a strong statistical evidence that hydroxychloroquine treatment reduces the proportion of symptomatic patients when used as a prophylaxis right after exposure, especially if treatment starts within 2 days. Next, we verify linear model suppositions. First, we assess the effect of heteroscedasticity by using weights in linear regression fit proportional to the sample sizes for x=1,2,3,4. The estimated slope is -21.63 and predicted relative difference at day 0 is 73.40%. One-tailed p-value is 0.88% in this case, which maintains the conclusions from homogeneous simple linear regression.
To assess for autocorrelation in the residuals we perform a two-tailed Durbin-Watson test using both Pan algorithm with 1000 iterations and bootstrap resampling algorithm with 10.000 replicates.
Autocorrelation = -0.46, DW statistic = 2.59 and p-value = 89.8%. Thus, at 95% confidence level, the hypothesis H0 (autocorrelation = 0) is not rejected.
To assess the normality of the residuals, we obtain kurtosis = 2.09, skewness = 0.77 and perform the Shapiro-Wilkson test. The W statistic = 0.91, p-value = 46.3% and at 95% confidence level the normality hypothesis is not rejected.
In this study, we discussed some inaccuracies in the statistical analysis of Boulware et al. 1 Moreover, our results show that the elapsed time between the exposure to the virus and the beginning of treatment is vital to the effectiveness of the antiviral use. We expect the treatment will be more effective when applied to patients in the viral replication period, before viral load reaches its peak which occurs around 5 days after symptom onset. 6 Meanwhile, if disease reaches the inflammatory period, typically after 8 days of symptoms onset and after viral load reaches its peak, 6 we can expected no or little benefit with the antiviral treatment.
Therefore, the mean time elapsed from exposure to the virus and the start of treatment in the sample may act as a lurking variable, influencing in a hidden way the efficacy of treatment. This might explain why many studies have found no statistical evidence of effectiveness of hydroxychloroquine treatment when used in hospitalized patients as most of this more severe cases had probably started treatment long after 4 days from their exposure to the virus. 3, 4, 6 In addition, it helps to understand why some studies have shown some positive results of hydroxychloroquine treatment as we can expect this when the proportion of patients in the beginning of the infection is higher in the sample. 3 Hence, as described by Boulware et al., 1 two possible applications would be to apply prophylaxis to health professionals and to contacts of positive patients, since these two groups would have a greater probability to benefit from treatment.
Our results suggest there is probably little or no benefit if the treatment is used in patients infected for too long, like hospitalized severe patients. On the contrary, they also suggest infected patients may have a large benefit if treated as early as possible, mostly as pre-prophylaxis treatment where symptoms appear will have an estimated relative reduction of at least 72%.
Another important aspect is that the variable of the study, be asymptomatic or be symptomatic, is quite time sensitive. Future trials should adopt less time sensitive variables, such as the number of days each patient is symptomatic, which could measure the possible benefit of treatment for patients that have been exposed for more than 3 days before the beginning of treatment. Another common possibility is to adopt some score system to measure severity of symptoms. 1 However, score systems are difficult to be scientifically validated because they typically depend on personal judgment, whereas the variable number of days with symptoms can be more easily replicated by other studies.
Furthermore, the hydroxychloroquine prophylaxis should also be investigated with concomitant use of azithromycin and zinc 7 , as also other antivirals should also be tested as prophylaxis, measuring the relative efficacy as a measure of the elapsed time after exposure to the virus to the beginning of treatment. 8 