Repository: gammapy/gammapy

Issue:
Change NDDataArray default_interp_kwargs to extrapolate
@joleroi - while working on CTA IRFs, @facero and @leajouvin  just now had the problem that our IRFs aren't extrapolating. I think we discussed this last week and said we should change to extrapolate by default.

Currently we have this:
https://github.com/gammapy/gammapy/blame/0e3a282736308a67fdcd51cbfded123b339e4770/gammapy/utils/nddata.py#L40
```
default_interp_kwargs = dict(bounds_error=False)
```

According to the scipy docs, it should be this to extrapolate, no?
```
default_interp_kwargs = dict(bounds_error=False, fill_value=None)
```

For now, they can use this and continue with their work:
```python
from gammapy.utils.nddata import NDDataArray
NDDataArray.default_interp_kwargs = dict(bounds_error=False, fill_value=None)
```

@joleroi - If you agree, could you please make the change and add one test to make sure it's extrapolating?


Discussion:
> According to the scipy docs, it should be this to extrapolate, no?

I know that you don't remember, but you wanted to change this to fill nan values outside extrapolation bounds. You even wrote it down [here](http://docs.gammapy.org/en/latest/development/howto.html#interpolation-and-extrapolation)

> For now, they can use this and continue with their work:

@jjlk @JouvinLea - you don't have to do this. All irfs classes duplicated the ``default_interp_kwargs`` attribute (e.g. https://github.com/gammapy/gammapy/blob/master/gammapy/irf/energy_dispersion.py#L598). So it should work like this

```
from gammapy.irf import EnergyDispersion2D
EnergyDispersion2D.default_interp_kwargs = ...
```

If you want to change the default behaviour for a given IRF class, feel free to make a PR.

> @joleroi - If you agree, could you please make the change and add one test to make sure it's extrapolating?

I don't think we should add the default behaviour to extrapolate everywhere (see http://docs.gammapy.org/en/latest/development/howto.html#interpolation-and-extrapolation). Horrible things can happen. If you want to extrapolate and know what you're doing, it's not too hard to add the 1 liner I put above, no?
@joleroi - I thought we discussed this with @adonath in my office a week or two ago, and decided to change to extrapolate, no?

Yes, I have changed my mind on this, I'm proposing we change our policy here.

There's just no other good way. E.g. CTA IRFs and also some HESS IRFs have first offset bin node at something > zero, but still it's common for people to query them for offset = 0, close to the first node. This works fine and is a reasonable thing to do, e.g. CTA does diffuse MC and has the first offset bin from 0 to 1 deg, and the first node at 0.5 deg.

---

Apart from many users having run into the same issue, the solution you propose doesn't really work:
```
from gammapy.irf import EnergyDispersion2D
EnergyDispersion2D.default_interp_kwargs = ...
```
The problem is that most users will access IRFs from pipelines or high-level code without even knowing about underlying IRF classes. It's bad coupling between low-level IRF interpolation and what end users should do if you ask them to configure the IRF interpolators.

So @joleroi - unless you are strongly opposed, I suggest we change the policy and default to extrapolate for all IRFs in Gammapy.
(the same problem with first FOV offset bin is there for all CTA IRFs for example)

Please!?

> E.g. CTA IRFs and also some HESS IRFs have first offset bin node at something > zero, but still it's common for people to query them for offset = 0, close to the first node

I see that the current scheme lead to a problem if the offset axis does not go to 0.

However, e.g. for the EnergyDispersion the current behaviour is to fill zeros outside the valid migra range. 
```
https://github.com/gammapy/gammapy/blob/master/gammapy/irf/energy_dispersion.py#L45
```
This makes sense since the migration probability for extreme migra values is 0. I don't know what happens if you turn on interpolation here.

Other example, effective area: It's hard to see what happend if you extrapolate to low or high energies. For that case I've added the [evaluate_fill_nan](https://github.com/gammapy/gammapy/blob/master/gammapy/irf/effective_area.py#L215) which puts 0s at low energies and a constant value at high energies.

Bottom line: Just because you run into trouble for one use case it might not be a good idea to change the behaviour for all of them. So I'm still :-1: on changing the behaviour of the ``NDDataArray``.

What you could to is change the default interp_kwargs of the classes that include an offset axis (``EnergyDisperson2D``, ``EffectiveAreaTable2D``, PSF classes don't use NDData I think) to extrapolate there. This would fix your problem, no?

Resolving patch:
diff --git a/gammapy/irf/effective_area.py b/gammapy/irf/effective_area.py
--- a/gammapy/irf/effective_area.py
+++ b/gammapy/irf/effective_area.py
@@ -318,8 +318,14 @@ class EffectiveAreaTable2D(object):
     offset         : size =     4, min =  0.000 deg, max =  1.000 deg
     Data           : size =    40, min =  1.000 cm2, max =  1.000 cm2
     """
+    default_interp_kwargs = dict(bounds_error=False, fill_value=None)
+    """Default Interpolation kwargs for `~NDDataArray`. Extrapolate."""
 
-    def __init__(self, energy_lo, energy_hi, offset_lo, offset_hi, data, meta=None):
+    def __init__(self, energy_lo, energy_hi, offset_lo, offset_hi, data,
+                 meta=None, interp_kwargs=None):
+
+        if interp_kwargs is None:
+            interp_kwargs = self.default_interp_kwargs
         axes = [
             BinnedDataAxis(
                 energy_lo, energy_hi, 
@@ -328,7 +334,8 @@ def __init__(self, energy_lo, energy_hi, offset_lo, offset_hi, data, meta=None):
                 offset_lo, offset_hi,
                 interpolation_mode='linear', name='offset')
         ]
-        self.data = NDDataArray(axes=axes, data=data)
+        self.data = NDDataArray(axes=axes, data=data,
+                                interp_kwargs=interp_kwargs)
         if meta is not None:
             self.meta = Bunch(meta)
 
diff --git a/gammapy/irf/energy_dispersion.py b/gammapy/irf/energy_dispersion.py
--- a/gammapy/irf/energy_dispersion.py
+++ b/gammapy/irf/energy_dispersion.py
@@ -43,7 +43,7 @@ class EnergyDispersion(object):
         2-dim energy dispersion matrix (probability density).
     """
     default_interp_kwargs = dict(bounds_error=False, fill_value=0)
-    """Default Interpolation kwargs for `~NDDataArray`"""
+    """Default Interpolation kwargs for `~NDDataArray`. Fill zeros"""
 
     def __init__(self, e_true_lo, e_true_hi, e_reco_lo, e_reco_hi,  data,
                  interp_kwargs=None, meta=None):
@@ -595,8 +595,8 @@ class EnergyDispersion2D(object):
         plt.loglog()
 
     """
-    default_interp_kwargs = dict(bounds_error=False, fill_value=0)
-    """Default Interpolation kwargs for `~NDDataArray`"""
+    default_interp_kwargs = dict(bounds_error=False, fill_value=None)
+    """Default Interpolation kwargs for `~NDDataArray`. Extrapolate."""
 
     def __init__(self, e_true_lo, e_true_hi, migra_lo, migra_hi, offset_lo,
                  offset_hi, data, interp_kwargs=None):


Test patch:
diff --git a/gammapy/spectrum/tests/test_extract.py b/gammapy/spectrum/tests/test_extract.py
--- a/gammapy/spectrum/tests/test_extract.py
+++ b/gammapy/spectrum/tests/test_extract.py
@@ -67,12 +67,12 @@ class TestSpectrumExtraction:
         (dict(containment_correction=False), dict(n_on=172,
                                                   sigma=24.98,
                                                   aeff=549861.8 * u.m ** 2,
-                                                  edisp=0.22417647310823458
+                                                  edisp=0.2237980131973548
                                                  )),
         (dict(containment_correction=True), dict(n_on=172,
                                                  sigma=24.98,
                                                  aeff=412731.8043631101 * u.m ** 2,
-                                                 edisp=0.22417647310823458
+                                                 edisp=0.2237980131973548
                                                 ))
     ])
     def test_extract(self, pars, results, target, obs, bkg, tmpdir):
diff --git a/gammapy/spectrum/tests/test_fit.py b/gammapy/spectrum/tests/test_fit.py
--- a/gammapy/spectrum/tests/test_fit.py
+++ b/gammapy/spectrum/tests/test_fit.py
@@ -192,13 +192,14 @@ def test_basic_results(self):
         self.fit.fit()
         result = self.fit.result[0]
         assert self.fit.method == 'sherpa'
-        assert_allclose(result.statval, 32.689097201967954)
+        assert_allclose(result.statval, 32.65890, rtol=1e-5)
         pars = result.model.parameters
         assert_quantity_allclose(pars['index'].value,
-                                 2.2507746631523795) 
+                                 2.25103, rtol=1e-5)
         assert_quantity_allclose(pars['amplitude'].quantity,
-                                 2.0073443570129334e-7 * u.Unit('m-2 s-1 TeV-1'))
-        assert_allclose(result.npred_src[60], 0.5704948742222784)
+                                 2.00538e-7 * u.Unit('m-2 s-1 TeV-1'),
+                                 rtol=1e-5)
+        assert_allclose(result.npred_src[60], 0.568627, rtol=1e-5)
 
         with pytest.raises(ValueError):
             t = self.fit.result[0].to_table()
@@ -208,8 +209,8 @@ def test_basic_errors(self):
         self.fit.est_errors()
         result = self.fit.result[0]
         par_errors = result.model.parameters._ufloats
-        assert_allclose(par_errors['index'].s, 0.09824357532850064)
-        assert_allclose(par_errors['amplitude'].s, 2.2038551595980807e-12)
+        assert_allclose(par_errors['index'].s, 0.098231, rtol=1e-5)
+        assert_allclose(par_errors['amplitude'].s, 2.199406e-12, rtol=1e-5)
 
         t = self.fit.result[0].to_table()
 
@@ -264,15 +265,16 @@ def test_ecpl_fit(self):
         fit = SpectrumFit(self.obs_list[0], self.ecpl)
         fit.fit()
         assert_quantity_allclose(fit.result[0].model.parameters['lambda_'].quantity,
-                                 0.033130174674062804 / u.TeV)
+                                 0.0331969 / u.TeV, rtol=1e-5)
 
     def test_joint_fit(self):
         fit = SpectrumFit(self.obs_list, self.pwl)
         fit.fit()
         assert_quantity_allclose(fit.model.parameters['index'].quantity,
-                                 2.208835103098456)
+                                 2.20872, rtol=1e-5)
         assert_quantity_allclose(fit.model.parameters['amplitude'].quantity,
-                                 2.362014689033278e-11 * u.Unit('cm-2 s-1 TeV-1'))
+                                 2.35883e-11 * u.Unit('cm-2 s-1 TeV-1'),
+                                 rtol=1e-5)
 
         # Change energy binnig of one observation
         # TODO: Add Rebin method to SpectrumObservation
@@ -285,9 +287,10 @@ def test_joint_fit(self):
 
         # TODO: Check if such a large deviation makes sense
         assert_quantity_allclose(fit.model.parameters['index'].quantity,
-                                 2.165317253019541)
+                                 2.16517, rtol=1e-5)
         assert_quantity_allclose(fit.model.parameters['amplitude'].quantity,
-                                 3.2006027905020016e-11 * u.Unit('cm-2 s-1 TeV-1'))
+                                 3.19642e-11 * u.Unit('cm-2 s-1 TeV-1'),
+                                 rtol=1e-5)
 
 
     def test_stacked_fit(self):
@@ -296,9 +299,10 @@ def test_stacked_fit(self):
         fit = SpectrumFit(obs_list, self.pwl)
         fit.fit()
         pars = fit.model.parameters
-        assert_quantity_allclose(pars['index'].value, 2.24586418915541)
+        assert_quantity_allclose(pars['index'].value, 2.24588, rtol=1e-5)
         assert_quantity_allclose(pars['amplitude'].quantity,
-                                 2.4970910953885096e-11 * u.Unit('cm-2 s-1 TeV-1'))
+                                 2.49398e-11 * u.Unit('cm-2 s-1 TeV-1'),
+                                 rtol=1e-5)
 
     def test_run(self, tmpdir):
         fit = SpectrumFit(self.obs_list, self.pwl)
@@ -321,5 +325,5 @@ def test_sherpa_fit(tmpdir):
     model.gamma = 2
     sau.set_model(model * 1e-20)
     sau.fit()
-    assert_allclose(model.pars[0].val, 2.0021424984466254) 
-    assert_allclose(model.pars[2].val, 2.2977111184000267)
+    assert_allclose(model.pars[0].val, 2.00331, rtol=1e-5)
+    assert_allclose(model.pars[2].val, 2.29916, rtol=1e-5)
diff --git a/gammapy/spectrum/tests/test_flux_point.py b/gammapy/spectrum/tests/test_flux_point.py
--- a/gammapy/spectrum/tests/test_flux_point.py
+++ b/gammapy/spectrum/tests/test_flux_point.py
@@ -165,10 +165,10 @@ def get_test_cases():
                                 reference=Quantity(1, 'TeV')),
                  obs=obs(),
                  seg=seg(obs()),
-                 dnde = 2.7507365595357705e-11 * u.Unit('cm-2 s-1 TeV-1'),
-                 dnde_err = 4.7628658000907185e-12 * u.Unit('cm-2 s-1 TeV-1'),
-                 res = -0.11023361,
-                 res_err = 0.15406193,  
+                 dnde = 2.746487655049709e-11 * u.Unit('cm-2 s-1 TeV-1'),
+                 dnde_err = 4.755409742841381e-12 * u.Unit('cm-2 s-1 TeV-1'),
+                 res = -0.11081311544196776,
+                 res_err = 0.15395838267321502,  
                 )
         )
         return test_cases
diff --git a/gammapy/spectrum/tests/test_observation.py b/gammapy/spectrum/tests/test_observation.py
--- a/gammapy/spectrum/tests/test_observation.py
+++ b/gammapy/spectrum/tests/test_observation.py
@@ -34,7 +34,7 @@ def get_test_obs():
         obs=obs_1,
         total_on=172,
         livetime = 1581.73681640625 * u.second,
-        npred = 214.28006281297564,
+        npred = 214.28087642107414,
         excess = 167.36363636363637,
         excess_safe_range = 135)
     )
@@ -185,7 +185,7 @@ def test_stack_method(self):
         assert_quantity_allclose(stacked_obs.aeff.data.data[10],
                                  86443352.23037884 * u.cm ** 2)
         assert_quantity_allclose(stacked_obs.edisp.data.data[50, 52],
-                                 0.02803437535072469)
+                                 0.027995, rtol=1e-5)
 
     def test_write(self, tmpdir):
         self.obs_list.write(outdir=str(tmpdir), pha_typeII=False)