Repository: scikit-hep/pyhf

Issue:
Inconsistency in CLs definition in hypotest
If I look at the [hypotest docs](https://diana-hep.org/pyhf/_generated/pyhf.utils.hypotest.html), it seems to me that what is called `p_b` is incorrect. In the docs just linked to, you can see that `CLb = 1 - p_b = 1 - \int_{-infinity}^{q_obs} f(q|b) dq`, which appears to be defining `p_b` as if it were a _left _tailed__ probability, which it is not. All p-values, including `p_b`, should be defined, I believe, as `p_b = \int_{q_obs}^{+infinity} f(q|b) dq`. 

If I look at the implementation of the `pvals_from_teststat` methods [here](https://github.com/diana-hep/pyhf/blob/d9a9ad13fbb6336a73ce9a6d53a6c044f8807a49/src/pyhf/utils.py#L171),

https://github.com/diana-hep/pyhf/blob/d9a9ad13fbb6336a73ce9a6d53a6c044f8807a49/src/pyhf/utils.py#L171

 the quantity with variable name `CLb` is given as `1 - CDF(q)`, which is actually `p_b` (using the right-tailed p-value definition). And so, the [CLs value](https://github.com/diana-hep/pyhf/blob/d9a9ad13fbb6336a73ce9a6d53a6c044f8807a49/src/pyhf/utils.py#L172) 

https://github.com/diana-hep/pyhf/blob/d9a9ad13fbb6336a73ce9a6d53a6c044f8807a49/src/pyhf/utils.py#L172

should be corrected as `CLs = CLsb / (1-CLb)`, where I use the same variable name of `CLb` as [here](https://github.com/diana-hep/pyhf/blob/d9a9ad13fbb6336a73ce9a6d53a6c044f8807a49/src/pyhf/utils.py#L197), 

https://github.com/diana-hep/pyhf/blob/d9a9ad13fbb6336a73ce9a6d53a6c044f8807a49/src/pyhf/utils.py#L197

which is _not_ what CLb is in the literature, at least not how it is defined when constructing CLs.

I think the implementation in `pvals_from_teststat` is taken from the discussion in the [asymptotics paper](https://arxiv.org/abs/1007.1727) from the discussion based on the Tevatron test statistic, which does not follow the same behavior as the profile likelihood ratio test statistic we use. But who knows, because I argue that Equation 76 abuses notation like nobody's business.

Let me know if I'm misinterpreting anything! I was just going through the [implementation](https://github.com/diana-hep/pyhf/blob/d9a9ad13fbb6336a73ce9a6d53a6c044f8807a49/src/pyhf/utils.py#L171) and the fact that the mathematical form for CLsb and CLb were the same

https://github.com/diana-hep/pyhf/blob/d9a9ad13fbb6336a73ce9a6d53a6c044f8807a49/src/pyhf/utils.py#L170-L171

 caught my eye.



Discussion:
I was trying to dig around for Eilam's statistics lecture series that he gave at CERN a few years ago, detailing all of this. On indico found a [set of slides](https://indico.cern.ch/event/687487/) that looked similar. If you look at Slide 74, you will see an expression for what he calls `p_{\mu}^{\prime}`, which should actually be called CL_s [1], is given by `(1 - CDF[ sqrt(q_mu) ] )/ ( CDF[ sqrt(q_muA) - sqrt(q_mu) ])`, which is inconsistent with the hypotest implementation in pyhf. I would check with the asymptotics people to confirm.

The chi-square asymptotic formulae do not go negative, and "less compatibility with the background only hypothesis" is for larger values of qmu, which is the opposite behavior motivating Equations 75 and 76 in the [asymptotics paper](https://arxiv.org/abs/1007.1727).

Slide 42 and beyond of [Lorenzo's lectures](https://indico.desy.de/indico/event/6083/contribution/1) are also helpful. Slide 42 itself shows a comparison of the Tevatron-like and profile-likelihood test statistic p-values.

[1] notation abuse abounds in the asymptotics literature
hi @dantrim 

i'll review the docstrings but I just wnated to give you a small notebook to see where this inversion of tail direction comes from. For q_mu and q_mu tilde test stats there is a definite "signal direction" in the test statistic (towards zero) which you can connect to the normal distributions of the best fitted mu (from wald's theorem)

so what is a right-tail p-value in mu space, becomes a left-tail p-value in test stat space and vice versa

<img width="536" alt="screenshot" src="https://user-images.githubusercontent.com/2318083/66188513-b3060380-e687-11e9-87d7-6b848af4c76b.png">

Hi @lukasheinrich , I think I would need to see a notebook and explanation. What you have there doesn't tell me much. As far as I understand, an increase in mu (incompatibility with null hypothesis) leads to increase in qmu?

Looking into the [RooStats implementation](https://root.cern.ch/doc/v608/classRooStats_1_1AsymptoticCalculator.html#a33f5b0c5ccd487cdfa482b3c9108351b), there are two versions of CLs depending on whether `BackgroundIsAlt`. By default `BackgroundIsAlt = False`, and the CLs values are computed as:

1.
```C++
CLs(BackgroundIsAlt = False, ROOSTATS) = CDF( sqrt(qmu_A) - sqrt(qmu) ) / (1 - CDF(sqrt(qmu))
```
2. 
```C++
CLs(BackgroundIsAlt = True, ROOSTATS) = (1 - CDF( sqrt(qmu) ) ) / ( CDF( sqrt(qmu_A) - sqrt(qmu)))
```

The CLs computation done in pyhf's [hypotest](https://github.com/diana-hep/pyhf/blob/d9a9ad13fbb6336a73ce9a6d53a6c044f8807a49/src/pyhf/utils.py#L131) is:

`CLs(pyhf) = ( 1 - CDF( sqrt(qmu)) ) / ( 1 - CDF( sqrt(qmu) - sqrt(qmu_A) ) )`

Just doing a blind comparison on the implementations, and assuming the notations are the same, the pyhf does not match with either of RooStats'.
> By default `BackgroundIsAlt = False`, and the CLs values are computed as:
> 
> 1.
> ```c++
> CLs(BackgroundIsAlt = False, ROOSTATS) = CDF( sqrt(qmu_A) - sqrt(qmu) ) / (1 - CDF(sqrt(qmu))
> ```
> 
> 2.
> ```c++
> CLs(BackgroundIsAlt = True, ROOSTATS) = (1 - CDF( sqrt(qmu) ) ) / ( CDF( sqrt(qmu_A) - sqrt(qmu)))
> ```
> 
> The CLs computation done in pyhf's [hypotest](https://github.com/diana-hep/pyhf/blob/d9a9ad13fbb6336a73ce9a6d53a6c044f8807a49/src/pyhf/utils.py#L131) is:
> 
> `CLs(pyhf) = ( 1 - CDF( sqrt(qmu)) ) / ( 1 - CDF( sqrt(qmu) - sqrt(qmu_A) ) )`
> 
> Just doing a blind comparison on the implementations, and assuming the notations are the same, the pyhf does not match with either of RooStats'.

The implementation is the same as the `CLs(BackgroundIsAlt = True, ROOSTATS)`, no? For the Normal (or any symmetrical distribution) distribution's CDF, `Phi`,

```
Phi(-x) = 1 - Phi(x)
```

so

```
Phi( sqrt(qmu_A) - sqrt(qmu)) = 1 - Phi( sqrt(qmu) - sqrt(qmu_A) )
```
Hi @matthewfeickert 
Maybe I am missing something, but I am not sure where the assumption on the distributions being symmetric is coming from? The sampling distributions of the qmu (i.e. f(qmu|mu')) are not symmetric...
> I am not sure where the assumption on the distributions being symmetric is coming from? The sampling distributions of the qmu (i.e. f(qmu|mu')) are not symmetric...

You're correct that `f(qmu|mu')` isn't symmetric, as it is  chi-square distributed ([arXiv:1007.1727](https://arxiv.org/abs/1007.1727) Eq. 55). However, for the CLs we're taking the ratio of probabilities _which are able to be efficiently calculated from the Normal distribution's CDF_ given the shift of the noncentrality parameter (q_μA = Λ) ([arXiv:1007.1727](https://arxiv.org/abs/1007.1727) related Eqs. 35, 50, 57, 59, 65, 67), which is what I was trying to get across in the [`pyhf.utils.pvals_from_teststat` docstring](https://diana-hep.org/pyhf/_generated/pyhf.utils.pvals_from_teststat.html) [1]:

![pvals_from_teststat_docstring](https://user-images.githubusercontent.com/5142394/66216064-0ba7c180-e6c5-11e9-9ba8-cf522fb39c51.png)

Our discussions over the last 2 days makes it clear to me though that in addition to writing a CLs notebook, I should also improve the docstrings.

[1] There is a known typo in the docstring (should be `\sqrt{q_{\mu}}` in the CDF). I haven't finished the docstrings PR that the fix is in, but it is fixed in [the corresponding branch's ReadTheDocs build](https://pyhf-staging.readthedocs.io/en/docs-add-docstrings-to-utils/_generated/pyhf.utils.pvals_from_teststat.html) (which is where the screenshot came from).

Resolving patch:
diff --git a/src/pyhf/infer/__init__.py b/src/pyhf/infer/__init__.py
--- a/src/pyhf/infer/__init__.py
+++ b/src/pyhf/infer/__init__.py
@@ -30,61 +30,90 @@ def hypotest(
 
     Args:
         poi_test (Number or Tensor): The value of the parameter of interest (POI)
-        data (Number or Tensor): The root of the calculated test statistic given the Asimov data, :math:`\sqrt{q_{\mu,A}}`
-        pdf (~pyhf.pdf.Model): The HistFactory statistical model
+        data (Number or Tensor): The data considered
+        pdf (~pyhf.pdf.Model): The statistical model adhering to the schema ``model.json``
         init_pars (Array or Tensor): The initial parameter values to be used for minimization
         par_bounds (Array or Tensor): The parameter value bounds to be used for minimization
-        qtilde (Bool): When ``True`` perform the calculation using the alternative test statistic, :math:`\tilde{q}`, as defined in Equation (62) of :xref:`arXiv:1007.1727`
+        qtilde (Bool): When ``True`` perform the calculation using the alternative
+         test statistic, :math:`\tilde{q}_{\mu}`, as defined under the Wald
+         approximation in Equation (62) of :xref:`arXiv:1007.1727`.
 
     Keyword Args:
-        return_tail_probs (bool): Bool for returning :math:`\textrm{CL}_{s+b}` and :math:`\textrm{CL}_{b}`
-        return_expected (bool): Bool for returning :math:`\textrm{CL}_{\textrm{exp}}`
-        return_expected_set (bool): Bool for returning the :math:`(-2,-1,0,1,2)\sigma` :math:`\textrm{CL}_{\textrm{exp}}` --- the "Brazil band"
+        return_tail_probs (bool): Bool for returning :math:`\mathrm{CL}_{s+b}` and :math:`\mathrm{CL}_{b}`
+        return_expected (bool): Bool for returning :math:`\mathrm{CL}_{\mathrm{exp}}`
+        return_expected_set (bool): Bool for returning the :math:`(-2,-1,0,1,2)\sigma` :math:`\mathrm{CL}_{\mathrm{exp}}` --- the "Brazil band"
 
     Returns:
         Tuple of Floats and lists of Floats:
 
-            - :math:`\textrm{CL}_{s}`: The :math:`p`-value compared to the given threshold :math:`\alpha`, typically taken to be :math:`0.05`, defined in :xref:`arXiv:1007.1727` as
+            - :math:`\mathrm{CL}_{s}`: The modified :math:`p`-value compared to
+              the given threshold :math:`\alpha`, typically taken to be :math:`0.05`,
+              defined in :xref:`arXiv:1007.1727` as
 
             .. math::
 
-                \textrm{CL}_{s} = \frac{\textrm{CL}_{s+b}}{\textrm{CL}_{b}} = \frac{p_{s+b}}{1-p_{b}}
+                \mathrm{CL}_{s} = \frac{\mathrm{CL}_{s+b}}{\mathrm{CL}_{b}} = \frac{p_{s+b}}{1-p_{b}}
 
-            to protect against excluding signal models in which there is little sensitivity. In the case that :math:`\textrm{CL}_{s} \leq \alpha` the given signal model is excluded.
+            to protect against excluding signal models in which there is little
+            sensitivity. In the case that :math:`\mathrm{CL}_{s} \leq \alpha`
+            the given signal model is excluded.
 
-            - :math:`\left[\textrm{CL}_{s+b}, \textrm{CL}_{b}\right]`: The signal + background :math:`p`-value and 1 minus the background only :math:`p`-value as defined in Equations (75) and (76) of :xref:`arXiv:1007.1727`
+            - :math:`\left[\mathrm{CL}_{s+b}, \mathrm{CL}_{b}\right]`: The
+              signal + background model hypothesis :math:`p`-value
 
             .. math::
 
-                \textrm{CL}_{s+b} = p_{s+b} = \int\limits_{q_{\textrm{obs}}}^{\infty} f\left(q\,\middle|s+b\right)\,dq = 1 - \Phi\left(\frac{q_{\textrm{obs}} + 1/\sigma_{s+b}^{2}}{2/\sigma_{s+b}}\right)
+                \mathrm{CL}_{s+b} = p_{s+b}
+                = p\left(q \geq q_{\mathrm{obs}}\middle|s+b\right)
+                = \int\limits_{q_{\mathrm{obs}}}^{\infty} f\left(q\,\middle|s+b\right)\,dq
+                = 1 - F\left(q_{\mathrm{obs}}(\mu)\,\middle|\mu'\right)
 
-            .. math::
-
-                \textrm{CL}_{b} = 1- p_{b} = 1 - \int\limits_{-\infty}^{q_{\textrm{obs}}} f\left(q\,\middle|b\right)\,dq = 1 - \Phi\left(\frac{q_{\textrm{obs}} - 1/\sigma_{b}^{2}}{2/\sigma_{b}}\right)
-
-            with Equations (73) and (74) for the mean
-
-            .. math::
-
-                E\left[q\right] = \frac{1 - 2\mu}{\sigma^{2}}
-
-            and variance
+            and 1 minus the background only model hypothesis :math:`p`-value
 
             .. math::
 
-                V\left[q\right] = \frac{4}{\sigma^{2}}
-
-            of the test statistic :math:`q` under the background only and and signal + background hypotheses. Only returned when ``return_tail_probs`` is ``True``.
-
-            - :math:`\textrm{CL}_{s,\textrm{exp}}`: The expected :math:`\textrm{CL}_{s}` value corresponding to the test statistic under the background only hypothesis :math:`\left(\mu=0\right)`. Only returned when ``return_expected`` is ``True``.
-
-            - :math:`\textrm{CL}_{s,\textrm{exp}}` band: The set of expected :math:`\textrm{CL}_{s}` values corresponding to the median significance of variations of the signal strength from the background only hypothesis :math:`\left(\mu=0\right)` at :math:`(-2,-1,0,1,2)\sigma`. That is, the :math:`p`-values that satisfy Equation (89) of :xref:`arXiv:1007.1727`
+                \mathrm{CL}_{b} = 1- p_{b}
+                = p\left(q \geq q_{\mathrm{obs}}\middle|b\right)
+                = 1 - \int\limits_{-\infty}^{q_{\mathrm{obs}}} f\left(q\,\middle|b\right)\,dq
+                = 1 - F\left(q_{\mathrm{obs}}(\mu)\,\middle|0\right)
+
+            for signal strength :math:`\mu` and model hypothesis signal strength
+            :math:`\mu'`, where the cumulative density functions
+            :math:`F\left(q(\mu)\,\middle|\mu'\right)` are given by Equations (57)
+            and (65) of :xref:`arXiv:1007.1727` for upper-limit-like test
+            statistic :math:`q \in \{q_{\mu}, \tilde{q}_{\mu}\}`.
+            Only returned when ``return_tail_probs`` is ``True``.
+
+            .. note::
+
+                The definitions of the :math:`\mathrm{CL}_{s+b}` and
+                :math:`\mathrm{CL}_{b}` used are based on profile likelihood
+                ratio test statistics.
+                This procedure is common in the LHC-era, but differs from
+                procedures used in the LEP and Tevatron eras, as briefly
+                discussed in :math:`\S` 3.8 of :xref:`arXiv:1007.1727`.
+
+            - :math:`\mathrm{CL}_{s,\mathrm{exp}}`: The expected :math:`\mathrm{CL}_{s}`
+              value corresponding to the test statistic under the background
+              only hypothesis :math:`\left(\mu=0\right)`.
+              Only returned when ``return_expected`` is ``True``.
+
+            - :math:`\mathrm{CL}_{s,\mathrm{exp}}` band: The set of expected
+              :math:`\mathrm{CL}_{s}` values corresponding to the median
+              significance of variations of the signal strength from the
+              background only hypothesis :math:`\left(\mu=0\right)` at
+              :math:`(-2,-1,0,1,2)\sigma`.
+              That is, the :math:`p`-values that satisfy Equation (89) of
+              :xref:`arXiv:1007.1727`
 
             .. math::
 
-                \textrm{band}_{N\sigma} = \mu' + \sigma\,\Phi^{-1}\left(1-\alpha\right) \pm N\sigma
+                \mathrm{band}_{N\sigma} = \mu' + \sigma\,\Phi^{-1}\left(1-\alpha\right) \pm N\sigma
 
-            for :math:`\mu'=0` and :math:`N \in \left\{-2, -1, 0, 1, 2\right\}`. These values define the boundaries of an uncertainty band sometimes referred to as the "Brazil band". Only returned when ``return_expected_set`` is ``True``.
+            for :math:`\mu'=0` and :math:`N \in \left\{-2, -1, 0, 1, 2\right\}`.
+            These values define the boundaries of an uncertainty band sometimes
+            referred to as the "Brazil band".
+            Only returned when ``return_expected_set`` is ``True``.
 
     """
     init_pars = init_pars or pdf.config.suggested_init()
diff --git a/src/pyhf/infer/calculators.py b/src/pyhf/infer/calculators.py
--- a/src/pyhf/infer/calculators.py
+++ b/src/pyhf/infer/calculators.py
@@ -74,8 +74,22 @@ def cdf(self, value):
         return tensorlib.normal_cdf((value - self.shift))
 
     def pvalue(self, value):
-        """
-        Compute the :math:`p`-value for a given value of the test statistic.
+        r"""
+        The :math:`p`-value for a given value of the test statistic corresponding
+        to signal strength :math:`\mu` and Asimov strength :math:`\mu'` as
+        defined in Equations (59) and (57) of :xref:`arXiv:1007.1727`
+
+        .. math::
+
+            p_{\mu} = 1-F\left(q_{\mu}\middle|\mu'\right) = 1- \Phi\left(\sqrt{q_{\mu}} - \frac{\left(\mu-\mu'\right)}{\sigma}\right)
+
+        with Equation (29)
+
+        .. math::
+
+            \frac{(\mu-\mu')}{\sigma} = \sqrt{\Lambda}= \sqrt{q_{\mu,A}}
+
+        given the observed test statistics :math:`q_{\mu}` and :math:`q_{\mu,A}`.
 
         Args:
             value (`float`): The test statistic value.
diff --git a/src/pyhf/infer/mle.py b/src/pyhf/infer/mle.py
--- a/src/pyhf/infer/mle.py
+++ b/src/pyhf/infer/mle.py
@@ -4,23 +4,61 @@
 
 
 def twice_nll(pars, data, pdf):
-    """
-    Twice the negative Log-Likelihood.
+    r"""
+    Two times the negative log-likelihood of the model parameters, :math:`\left(\mu, \boldsymbol{\theta}\right)`, given the observed data.
+    It is used in the calculation of the test statistic, :math:`t_{\mu}`, as defiend in Equation (8) in :xref:`arXiv:1007.1727`
+
+    .. math::
+
+       t_{\mu} = -2\ln\lambda\left(\mu\right)
+
+    where :math:`\lambda\left(\mu\right)` is the profile likelihood ratio as defined in Equation (7)
+
+    .. math::
+
+       \lambda\left(\mu\right) = \frac{L\left(\mu, \hat{\hat{\boldsymbol{\theta}}}\right)}{L\left(\hat{\mu}, \hat{\boldsymbol{\theta}}\right)}\,.
+
+    It serves as the objective function to minimize in :func:`~pyhf.infer.mle.fit`
+    and :func:`~pyhf.infer.mle.fixed_poi_fit`.
+
+    Example:
+        >>> import pyhf
+        >>> pyhf.set_backend("numpy")
+        >>> model = pyhf.simplemodels.hepdata_like(
+        ...     signal_data=[12.0, 11.0], bkg_data=[50.0, 52.0], bkg_uncerts=[3.0, 7.0]
+        ... )
+        >>> observations = [51, 48]
+        >>> data = pyhf.tensorlib.astensor(observations + model.config.auxdata)
+        >>> parameters = model.config.suggested_init()  # nominal parameters
+        >>> twice_nll = pyhf.infer.mle.twice_nll(parameters, data, model)
+        >>> twice_nll
+        array([30.77525435])
+        >>> -2 * model.logpdf(parameters, data) == twice_nll
+        array([ True])
 
     Args:
-        data (`tensor`): The data
+        pars (`tensor`): The parameters of the HistFactory model
+        data (`tensor`): The data to be considered
         pdf (~pyhf.pdf.Model): The statistical model adhering to the schema model.json
 
     Returns:
-        Twice the negative log likelihood.
-
+        Tensor: Two times the negative log-likelihood, :math:`-2\ln L\left(\mu, \boldsymbol{\theta}\right)`
     """
     return -2 * pdf.logpdf(pars, data)
 
 
 def fit(data, pdf, init_pars=None, par_bounds=None, **kwargs):
-    """
+    r"""
     Run a unconstrained maximum likelihood fit.
+    This is done by minimizing the objective function :func:`~pyhf.infer.mle.twice_nll`
+    of the model parameters given the observed data.
+    This is used to produce the maximal likelihood :math:`L\left(\hat{\mu}, \hat{\boldsymbol{\theta}}\right)`
+    in the profile likelihood ratio in Equation (7) in :xref:`arXiv:1007.1727`
+
+    .. math::
+
+       \lambda\left(\mu\right) = \frac{L\left(\mu, \hat{\hat{\boldsymbol{\theta}}}\right)}{L\left(\hat{\mu}, \hat{\boldsymbol{\theta}}\right)}
+
 
     .. note::
 
@@ -62,8 +100,17 @@ def fit(data, pdf, init_pars=None, par_bounds=None, **kwargs):
 
 
 def fixed_poi_fit(poi_val, data, pdf, init_pars=None, par_bounds=None, **kwargs):
-    """
+    r"""
     Run a maximum likelihood fit with the POI value fixed.
+    This is done by minimizing the objective function of :func:`~pyhf.infer.mle.twice_nll`
+    of the model parameters given the observed data, for a given fixed value of :math:`\mu`.
+    This is used to produce the constrained maximal likelihood for the given :math:`\mu`
+    :math:`L\left(\mu, \hat{\hat{\boldsymbol{\theta}}}\right)` in the profile
+    likelihood ratio in Equation (7) in :xref:`arXiv:1007.1727`
+
+    .. math::
+
+       \lambda\left(\mu\right) = \frac{L\left(\mu, \hat{\hat{\boldsymbol{\theta}}}\right)}{L\left(\hat{\mu}, \hat{\boldsymbol{\theta}}\right)}
 
     .. note::
 


Test patch:
diff --git a/src/pyhf/infer/test_statistics.py b/src/pyhf/infer/test_statistics.py
--- a/src/pyhf/infer/test_statistics.py
+++ b/src/pyhf/infer/test_statistics.py
@@ -52,7 +52,7 @@ def qmu(mu, data, pdf, init_pars, par_bounds):
     r"""
     The test statistic, :math:`q_{\mu}`, for establishing an upper
     limit on the strength parameter, :math:`\mu`, as defiend in
-    Equation (14) in :xref:`arXiv:1007.1727`.
+    Equation (14) in :xref:`arXiv:1007.1727`
 
     .. math::
        :nowrap:
@@ -64,6 +64,12 @@ def qmu(mu, data, pdf, init_pars, par_bounds):
           \end{array}\right.
         \end{equation}
 
+    where :math:`\lambda\left(\mu\right)` is the profile likelihood ratio as defined in Equation (7)
+
+    .. math::
+
+       \lambda\left(\mu\right) = \frac{L\left(\mu, \hat{\hat{\boldsymbol{\theta}}}\right)}{L\left(\hat{\mu}, \hat{\boldsymbol{\theta}}\right)}\,.
+
     Example:
         >>> import pyhf
         >>> pyhf.set_backend("numpy")
@@ -105,7 +111,29 @@ def qmu_tilde(mu, data, pdf, init_pars, par_bounds):
     r"""
     The test statistic, :math:`\tilde{q}_{\mu}`, for establishing an upper
     limit on the strength parameter, :math:`\mu`, for models with
-    bounded POI, as defiend in Equation (16) in :xref:`arXiv:1007.1727`.
+    bounded POI, as defiend in Equation (16) in :xref:`arXiv:1007.1727`
+
+    .. math::
+       :nowrap:
+
+       \begin{equation}
+          \tilde{q}_{\mu} = \left\{\begin{array}{ll}
+          -2\ln\tilde{\lambda}\left(\mu\right), &\hat{\mu} < \mu,\\
+          0, & \hat{\mu} > \mu
+          \end{array}\right.
+        \end{equation}
+
+    where :math:`\tilde{\lambda}\left(\mu\right)` is the constrained profile likelihood ratio as defined in Equation (10)
+
+    .. math::
+       :nowrap:
+
+       \begin{equation}
+          \tilde{\lambda}\left(\mu\right) = \left\{\begin{array}{ll}
+          \frac{L\left(\mu, \hat{\hat{\boldsymbol{\theta}}}(\mu)\right)}{L\left(\hat{\mu}, \hat{\hat{\boldsymbol{\theta}}}(0)\right)}, &\hat{\mu} < 0,\\
+          \frac{L\left(\mu, \hat{\hat{\boldsymbol{\theta}}}(\mu)\right)}{L\left(\hat{\mu}, \hat{\boldsymbol{\theta}}\right)}, &\hat{\mu} \geq 0.
+          \end{array}\right.
+        \end{equation}
 
     Example:
         >>> import pyhf
@@ -146,8 +174,18 @@ def qmu_tilde(mu, data, pdf, init_pars, par_bounds):
 def tmu(mu, data, pdf, init_pars, par_bounds):
     r"""
     The test statistic, :math:`t_{\mu}`, for establishing a two-sided
-    interval on the strength parameter, :math:`\mu`, as defiend in Equation (10)
-    in :xref:`arXiv:1007.1727`.
+    interval on the strength parameter, :math:`\mu`, as defiend in Equation (8)
+    in :xref:`arXiv:1007.1727`
+
+    .. math::
+
+       t_{\mu} = -2\ln\lambda\left(\mu\right)
+
+    where :math:`\lambda\left(\mu\right)` is the profile likelihood ratio as defined in Equation (7)
+
+    .. math::
+
+       \lambda\left(\mu\right) = \frac{L\left(\mu, \hat{\hat{\boldsymbol{\theta}}}\right)}{L\left(\hat{\mu}, \hat{\boldsymbol{\theta}}\right)}\,.
 
     Example:
         >>> import pyhf
@@ -188,9 +226,25 @@ def tmu(mu, data, pdf, init_pars, par_bounds):
 
 def tmu_tilde(mu, data, pdf, init_pars, par_bounds):
     r"""
-    The test statistic, :math:`t_{\mu}`, for establishing a two-sided
+    The test statistic, :math:`\tilde{t}_{\mu}`, for establishing a two-sided
     interval on the strength parameter, :math:`\mu`, for models with
-    bounded POI, as defiend in Equation (11) in :xref:`arXiv:1007.1727`.
+    bounded POI, as defiend in Equation (11) in :xref:`arXiv:1007.1727`
+
+    .. math::
+
+       \tilde{t}_{\mu} = -2\ln\tilde{\lambda}\left(\mu\right)
+
+    where :math:`\tilde{\lambda}\left(\mu\right)` is the constrained profile likelihood ratio as defined in Equation (10)
+
+    .. math::
+       :nowrap:
+
+       \begin{equation}
+          \tilde{\lambda}\left(\mu\right) = \left\{\begin{array}{ll}
+          \frac{L\left(\mu, \hat{\hat{\boldsymbol{\theta}}}(\mu)\right)}{L\left(\hat{\mu}, \hat{\hat{\boldsymbol{\theta}}}(0)\right)}, &\hat{\mu} < 0,\\
+          \frac{L\left(\mu, \hat{\hat{\boldsymbol{\theta}}}(\mu)\right)}{L\left(\hat{\mu}, \hat{\boldsymbol{\theta}}\right)}, &\hat{\mu} \geq 0.
+          \end{array}\right.
+        \end{equation}
 
     Example:
         >>> import pyhf