Repository: gammapy/gammapy

Issue:
Negative values in flux point errors 
I'm investigating what is causing the vertical bars when plotting flux points from my RXJ1713 analysis described here [(link)](https://github.com/gammasky/cta-analyses/blob/master/dc-1-checks/rx_j1713/RXJ_gammapy_data_analysis.ipynb). See spectrum shown below:
 
![image](https://user-images.githubusercontent.com/8652898/33183059-a2e2cc6e-d076-11e7-8290-0654dfcd296f.png)

The feature comes from the fact that in the asymetric errors, the dnde_errn column has negative values in it. This means that for this analysis, the lower limit obtained at this line:
https://github.com/gammapy/gammapy/blob/0d475fc42a165dd4853e31615eb4d08b83847ed7/gammapy/spectrum/flux_point.py#L811

has a higher value that the flux point. The lower limit is estimated here :
https://github.com/gammapy/gammapy/blob/0d475fc42a165dd4853e31615eb4d08b83847ed7/gammapy/spectrum/flux_point.py#L684

Has anyone experienced that behaviour in the past ? 


Discussion:
Thanks for reporting this issue! I've written this part of the code and have already used it successfully for the HGPS analysis.  I'll take a closer look at the notebook you've linked and come back to you...
I also looked at the flux point class once and did something, but didn't take the time to improve it much. Here's a reminder issue also mentioning plot: #1034 
I think the state is still the same: `plot` does quite a lot of computations and thus is hard to test, so refactoring that code is at least the long-term solution.
@facero I checked out the notebook from the repo and tried to execute it, but unfortunately it fails with following error: 
```
FileNotFoundError: [Errno 2] No such file or directory: 'detected_sources.fits'
```
This file is file is only written later in another cell. Could you commit a version of the notebook that runs linearly from top to bottom (or maybe commit the `detected_sources.fits` or the exclusion mask)? This would help a lot to reproduce the issue.

From what you have described in your initial post, that the following line becomes negative:
https://github.com/gammapy/gammapy/blob/0d475fc42a165dd4853e31615eb4d08b83847ed7/gammapy/spectrum/flux_point.py#L811
This can be a hint, that either the fit didn't succeed or maybe the search interval for the `delta_TS = 1` points for the error estimation is too large. The only way to debug this properly is to compute the likelihood profile for every point, which is currently not possible. But we've been wanting to implement this since a long time...
@facero Looking at the notebook I see artefacts in the difference images between the images computed with `IACTBasicImageEstimator` and `make_cube.py`. I looks like there is a one pixel shift in the FoV positions, do you plan to debug this?
@adonath I've added the missing file detected_sources.fits.

Concerning the notebook, the ring-like artifacts in the ratio of counts map made me think also of a difference in center of FoV. It also could be a difference in Radial cut. The value was the same but in one case it could angle< 3.5 deg or <= .35 deg.
I will double check that as well.

@facero Yes, indeed it looks like the difference is in the FoV cut, because the pixel row is symmetric in all directions and not a systematic shift in one direction. I just talked to @cdeil in person and he mentioned that your are looking into this together. Thanks!

The notebooks is working, I'll start to look into the issue now. 
@facero I was able to track the bug...it is very subtle and related to the use of the `ExponentialCutoffPowerLaw` model in the flux point computation. This explains why it did not appear in the HGPS analysis, because we only use  `PowerLaw` models. As it's a bit complicated to explain, but easy to fix I'll do it myself on Monday and open a PR.  
 

Resolving patch:
diff --git a/gammapy/spectrum/flux_point.py b/gammapy/spectrum/flux_point.py
--- a/gammapy/spectrum/flux_point.py
+++ b/gammapy/spectrum/flux_point.py
@@ -741,7 +741,8 @@ def ts_diff(x):
         try:
             result = brentq(ts_diff, amplitude_min, amplitude_max,
                             maxiter=100, rtol=1e-2)
-            return 1E-12 * result * u.Unit(fit.model.parameters['amplitude'].unit)
+            fit.model.parameters['amplitude'].value = result * 1E-12
+            return fit.model(fit.model.parameters['reference'].quantity)
         except (RuntimeError, ValueError):
             # Where the root finding fails NaN is set as amplitude
             log.debug('Flux point upper limit computation failed.')


Test patch:
diff --git a/gammapy/spectrum/tests/test_flux_point.py b/gammapy/spectrum/tests/test_flux_point.py
--- a/gammapy/spectrum/tests/test_flux_point.py
+++ b/gammapy/spectrum/tests/test_flux_point.py
@@ -11,7 +11,7 @@
 from ...utils.testing import requires_dependency, requires_data
 from ...utils.modeling import ParameterList
 from ...spectrum import SpectrumResult, SpectrumFit
-from ...spectrum.models import PowerLaw, SpectralModel
+from ...spectrum.models import PowerLaw, SpectralModel, ExponentialCutoffPowerLaw
 from ..flux_point import FluxPoints
 from ..flux_point import FluxPointProfiles
 from ..flux_point import FluxPointFitter
@@ -154,7 +154,7 @@ def test_compute_flux_points_dnde_exp(method):
 @requires_dependency('sherpa')
 @requires_dependency('matplotlib')
 @requires_dependency('scipy')
-@pytest.mark.parametrize('config', ['pl'])
+@pytest.mark.parametrize('config', ['pl', 'ecpl'])
 def test_flux_points(config):
     if config == 'pl':
         config = dict(
@@ -173,6 +173,24 @@ def test_flux_points(config):
             res=-0.1126,
             res_err=0.1536,
         )
+    elif config == 'ecpl':
+        config = dict(
+            model=ExponentialCutoffPowerLaw(
+                index=Quantity(2, ''),
+                amplitude=Quantity(1e-11, 'm-2 s-1 TeV-1'),
+                reference=Quantity(1, 'TeV'),
+                lambda_= Quantity(0.1, 'TeV-1')
+            ),
+            obs=obs(),
+            seg=seg(obs()),
+            dnde=2.7465e-11 * u.Unit('cm-2 s-1 TeV-1'),
+            dnde_err=4.7555e-12 * u.Unit('cm-2 s-1 TeV-1'),
+            dnde_errn=4.5333e-12 * u.Unit('cm-2 s-1 TeV-1'),
+            dnde_errp=5.0050e-12 * u.Unit('cm-2 s-1 TeV-1'),
+            dnde_ul=3.7998e-11 * u.Unit('cm-2 s-1 TeV-1'),
+            res=-0.057955,
+            res_err=0.1634,
+        )
 
     tester = FluxPointTester(config)
     tester.test_all()