Repository: gammapy/gammapy

Issue:
fit.minos_contour parameter handling
While accesing the fit.minos_contour with my Spectral model with parameters(amplitude, reference, alpha and beta which corresponds to strings par_000_amplitude, par_001_alpha, par_002_beta) but with the minos_contour when I am passing the parameters it is taking strings(par_000_amplitude, par_001_reference, par_002_alpha, par_003_beta) which is giivng me the key error of parameter not found.
fit.minos_contour parameter handling
While accesing the fit.minos_contour with my Spectral model with parameters(amplitude, reference, alpha and beta which corresponds to strings par_000_amplitude, par_001_alpha, par_002_beta) but with the minos_contour when I am passing the parameters it is taking strings(par_000_amplitude, par_001_reference, par_002_alpha, par_003_beta) which is giivng me the key error of parameter not found.


Discussion:
Thanks for opening this issue @Rishank2610. Note you can directly pass the parameter objects, or parameter names like so:

```python3

from gammapy.spectrum import SpectrumDatasetOnOff
from gammapy.utils.fitting import Fit
from gammapy.spectrum.models import PowerLaw
import matplotlib.pyplot as plt

filename = '$GAMMAPY_DATA/joint-crab/spectra/hess/pha_obs23523.fits'
dataset = SpectrumDatasetOnOff.from_ogip_files(filename)

model = PowerLaw(
    index=2,
    amplitude='1e-12 cm-2 s-1 TeV-1',
    reference='1 TeV',
)

dataset.model = model

fit = Fit([dataset])
result = fit.run()

# by parameter name
contour = fit.minos_contour("amplitude", "index", numpoints=50)

# by parameter object
contour = fit.minos_contour(model.amplitude, model.index, numpoints=50)

plt.fill(contour["x"], contour["y"], fill=False)
plt.xlabel("Amplitude (cm-2 s-1 TeV-1)")
plt.ylabel("Index");
```

Let me know if this works for you...
I am getting the same error.
Which version of Gammapy are you using (`python -c "import gammapy; print(gammapy.__version__)"`)? The example I posted above works correctly with Gammapy 0.12.
I am using Gammapy-0.12, for more details I am attaching the code I am running for my analysis.
```python3
from gammapy.spectrum import SpectrumDatasetOnOff, SpectrumDatasetOnOffStacker
from gammapy.utils.fitting import Fit, Model, Parameter
from gammapy.spectrum.models import PowerLaw
from astropy import units as u
from gammapy.spectrum.models import SpectralModel, LogParabola
from gammapy.utils.fitting import Parameter, Parameters
import numpy as np

obs_ids = [5029747, 5029748]
spectrum_observations = []
for obs in obs_ids:
    filename = "results/spectra/magic/pha_obs{}.fits"
    ds = SpectrumDatasetOnOff.from_ogip_files(filename.format(obs))
    spectrum_observations.append(ds)
    
model = LogParabola(
        amplitude="3.8e-11 cm-2 s-1 TeV-1", reference="1 TeV", alpha=2.47, beta=0.24
    )

for obs in spectrum_observations:
    obs.model = model

fit = Fit(spectrum_observations)
result = fit.run()

numpoints=20
sigma=np.sqrt(2.3)

contour = fit.minos_contour("amplitude", "alpha", numpoints=numpoints, sigma=sigma)
```

and I am getting error 

```
KeyError                                  Traceback (most recent call last)
<ipython-input-4-1b55b5dcc81e> in <module>
      3 contours = {}
      4 
----> 5 contour = fit.minos_contour("amplitude", "alpha", numpoints=numpoints, sigma=sigma)

~/anaconda2/envs/gammapy-0.12/lib/python3.6/site-packages/gammapy/utils/fitting/fit.py in minos_contour(self, x, y, numpoints, sigma)
    405 
    406         with parameters.restore_values:
--> 407             result = mncontour(self.minuit, parameters, x, y, numpoints, sigma)
    408 
    409         x = result["x"] * x.scale

~/anaconda2/envs/gammapy-0.12/lib/python3.6/site-packages/gammapy/utils/fitting/iminuit.py in mncontour(minuit, parameters, x, y, numpoints, sigma)
    106     y = _make_parname(idx, parameters[idx])
    107 
--> 108     x_info, y_info, contour = minuit.mncontour(x, y, numpoints, sigma)
    109     contour = np.array(contour)
    110 

iminuit/_libiminuit.pyx in iminuit._libiminuit.Minuit.mncontour()

KeyError: 'par_002_alpha'
```
Thanks @Rishank2610, now I can reproduce the error and I can also confirm this is a bug in Gammapy.
In the `mncontour` helper function in `utils/fitting/iminuit.py`, the parameter name is created from the full parameter list, while it should be created from the free parameters only. I think I can fix this soon.

Meanwhile you can use the following work-around, because the `Minuit` object is available on the Gammapy `Fit` object:

```python3
from gammapy.spectrum import SpectrumDatasetOnOff, SpectrumDatasetOnOffStacker
from gammapy.utils.fitting import Fit, Model, Parameter
from gammapy.spectrum.models import PowerLaw
from astropy import units as u
from gammapy.spectrum.models import SpectralModel, LogParabola
from gammapy.utils.fitting import Parameter, Parameters
import numpy as np

obs_ids = [23523, 23526]
spectrum_observations = []

for obs in obs_ids:
    filename = '$GAMMAPY_DATA/joint-crab/spectra/hess/pha_obs{}.fits'.format(obs)
    ds = SpectrumDatasetOnOff.from_ogip_files(filename.format(obs))
    spectrum_observations.append(ds)
    
model = LogParabola(
        amplitude="3.8e-11 cm-2 s-1 TeV-1", reference="1 TeV", alpha=2.47, beta=0.24
    )

for obs in spectrum_observations:
    obs.model = model

fit = Fit(spectrum_observations)
fit.datasets.parameters.apply_autoscale = False
result = fit.run()

numpoints=20
sigma=np.sqrt(2.3)

print(fit.minuit.parameters)
fit.minuit.draw_contour("par_000_amplitude", "par_001_alpha")

# or 
contour = fit.minuit.mncontour("par_000_amplitude", "par_001_alpha")
```
it is working fine for one analysis but then it is  giving me error in other analysis
`
from gammapy.spectrum import SpectrumDatasetOnOff, SpectrumDatasetOnOffStacker
from gammapy.utils.fitting import Fit, Model, Parameter
from gammapy.spectrum.models import PowerLaw
from astropy import units as u
from gammapy.spectrum.models import SpectralModel
from gammapy.utils.fitting import Parameter, Parameters
import numpy as np


class Log10ParabolaEnergyScale(SpectralModel):

    def __init__(
        self,
        amplitude=1e-12 * u.Unit("cm-2 s-1 TeV-1"),
        reference=1 * u.TeV,
        alpha=3,
        beta=1,
        z=0.12,
    ):

        self.parameters = Parameters(
            [
                Parameter("amplitude", amplitude),
                Parameter("reference", reference, frozen=True),
                Parameter("alpha", alpha),
                Parameter("beta", beta),
                Parameter("z", z),
            ]
        )

    @staticmethod
    def evaluate(energy, amplitude, reference, alpha, beta, z):
        """Evaluate the model (static function)."""
        scale_factor = 1 / (1 + z)
        scaled_energy = energy * scale_factor
        try:
            xx = scaled_energy / reference
            exponent = -alpha - beta * np.log10(xx)
        except AttributeError:
            from uncertainties.unumpy import log10

            xx = scaled_energy / reference
            exponent = -alpha - beta * log10(xx)
        return amplitude * np.power(xx, exponent) * scale_factor

class SystematicsSpetrumModel(SpectralModel):
    def __init__(self):
        self.parameters = Parameters(
            [
                Parameter("amplitude", 3e-12, unit="cm-2 s-1 TeV-1"),
                Parameter("reference", 1, unit="TeV", frozen=True),
                Parameter("alpha", 2.4),
                Parameter("beta", 0.2),
                Parameter("z_magic", 0),
            ]
        )

class SystematicsSpectrumFit():
    delta = {"magic": 0.15}

    def __init__(self):
        self._model = SystematicsSpetrumModel()
        self.fits = {name: self.make_fit(name) for name in ["magic"]}

    def make_fit(self, name):
        obs_ids = [5029747, 5029748]
        observations = []
        for obs in obs_ids:
            filename = "results/spectra/magic/pha_obs{}.fits"
            ds = SpectrumDatasetOnOff.from_ogip_files(filename.format(obs))
            observations.append(ds)
        model = Log10ParabolaEnergyScale(amplitude="3e-12 cm-2 s-1 TeV-1", reference="1 TeV", alpha=2.4, beta=0.2, z=0)
        #fit_range = dataset.energy_range
        for obs in observations:
            obs.model = model
        return Fit(observations)

    def total_stat(self, parameters):
        stat = 0

        for name, fit in self.fits.items():
            # Parameter update
            pars = fit._model.parameters
            pars["amplitude"].value = parameters["amplitude"].value
            pars["alpha"].value = parameters["alpha"].value
            pars["beta"].value = parameters["beta"].value
            pars["z"].value = parameters["z_" + name].value

            # Poisson likelihood
            stat += fit.total_stat(pars)

            # Gaussian likelihood (energy scale systematics)
            stat += (pars["z"].value / self.delta[name]) ** 2

        return stat

fit = SystematicsSpectrumFit().make_fit("magic")
fit.datasets.parameters.apply_autoscale = False
result = fit.run()
print(result)

numpoints=20
sigma=np.sqrt(2.3)

print(fit.minuit.parameters)
fit.minuit.draw_contour("par_000_amplitude", "par_001_alpha")

contour = fit.minuit.mncontour("par_000_amplitude", "par_001_alpha", numpoints=numpoints, sigma=sigma)
`
This is code example is impossible for me to debug. Maybe you can try to find a small self-contained example, that just uses data from `$GAMMAPY_DATA`. In generaI I don't see a reason, why it should fail. Please check the parameter names with `print(fit.minuit.parameters)`, before you pass them to `mncontour()`. This should always work.

 Otherwise I'm afraid you'll have to wait for the fix...
Thank you for your help. I guess I have to wait for the fix.
Thanks for opening this issue @Rishank2610. Note you can directly pass the parameter objects, or parameter names like so:

```python3

from gammapy.spectrum import SpectrumDatasetOnOff
from gammapy.utils.fitting import Fit
from gammapy.spectrum.models import PowerLaw
import matplotlib.pyplot as plt

filename = '$GAMMAPY_DATA/joint-crab/spectra/hess/pha_obs23523.fits'
dataset = SpectrumDatasetOnOff.from_ogip_files(filename)

model = PowerLaw(
    index=2,
    amplitude='1e-12 cm-2 s-1 TeV-1',
    reference='1 TeV',
)

dataset.model = model

fit = Fit([dataset])
result = fit.run()

# by parameter name
contour = fit.minos_contour("amplitude", "index", numpoints=50)

# by parameter object
contour = fit.minos_contour(model.amplitude, model.index, numpoints=50)

plt.fill(contour["x"], contour["y"], fill=False)
plt.xlabel("Amplitude (cm-2 s-1 TeV-1)")
plt.ylabel("Index");
```

Let me know if this works for you...
I am getting the same error.
Which version of Gammapy are you using (`python -c "import gammapy; print(gammapy.__version__)"`)? The example I posted above works correctly with Gammapy 0.12.
I am using Gammapy-0.12, for more details I am attaching the code I am running for my analysis.
```python3
from gammapy.spectrum import SpectrumDatasetOnOff, SpectrumDatasetOnOffStacker
from gammapy.utils.fitting import Fit, Model, Parameter
from gammapy.spectrum.models import PowerLaw
from astropy import units as u
from gammapy.spectrum.models import SpectralModel, LogParabola
from gammapy.utils.fitting import Parameter, Parameters
import numpy as np

obs_ids = [5029747, 5029748]
spectrum_observations = []
for obs in obs_ids:
    filename = "results/spectra/magic/pha_obs{}.fits"
    ds = SpectrumDatasetOnOff.from_ogip_files(filename.format(obs))
    spectrum_observations.append(ds)
    
model = LogParabola(
        amplitude="3.8e-11 cm-2 s-1 TeV-1", reference="1 TeV", alpha=2.47, beta=0.24
    )

for obs in spectrum_observations:
    obs.model = model

fit = Fit(spectrum_observations)
result = fit.run()

numpoints=20
sigma=np.sqrt(2.3)

contour = fit.minos_contour("amplitude", "alpha", numpoints=numpoints, sigma=sigma)
```

and I am getting error 

```
KeyError                                  Traceback (most recent call last)
<ipython-input-4-1b55b5dcc81e> in <module>
      3 contours = {}
      4 
----> 5 contour = fit.minos_contour("amplitude", "alpha", numpoints=numpoints, sigma=sigma)

~/anaconda2/envs/gammapy-0.12/lib/python3.6/site-packages/gammapy/utils/fitting/fit.py in minos_contour(self, x, y, numpoints, sigma)
    405 
    406         with parameters.restore_values:
--> 407             result = mncontour(self.minuit, parameters, x, y, numpoints, sigma)
    408 
    409         x = result["x"] * x.scale

~/anaconda2/envs/gammapy-0.12/lib/python3.6/site-packages/gammapy/utils/fitting/iminuit.py in mncontour(minuit, parameters, x, y, numpoints, sigma)
    106     y = _make_parname(idx, parameters[idx])
    107 
--> 108     x_info, y_info, contour = minuit.mncontour(x, y, numpoints, sigma)
    109     contour = np.array(contour)
    110 

iminuit/_libiminuit.pyx in iminuit._libiminuit.Minuit.mncontour()

KeyError: 'par_002_alpha'
```
Thanks @Rishank2610, now I can reproduce the error and I can also confirm this is a bug in Gammapy.
In the `mncontour` helper function in `utils/fitting/iminuit.py`, the parameter name is created from the full parameter list, while it should be created from the free parameters only. I think I can fix this soon.

Meanwhile you can use the following work-around, because the `Minuit` object is available on the Gammapy `Fit` object:

```python3
from gammapy.spectrum import SpectrumDatasetOnOff, SpectrumDatasetOnOffStacker
from gammapy.utils.fitting import Fit, Model, Parameter
from gammapy.spectrum.models import PowerLaw
from astropy import units as u
from gammapy.spectrum.models import SpectralModel, LogParabola
from gammapy.utils.fitting import Parameter, Parameters
import numpy as np

obs_ids = [23523, 23526]
spectrum_observations = []

for obs in obs_ids:
    filename = '$GAMMAPY_DATA/joint-crab/spectra/hess/pha_obs{}.fits'.format(obs)
    ds = SpectrumDatasetOnOff.from_ogip_files(filename.format(obs))
    spectrum_observations.append(ds)
    
model = LogParabola(
        amplitude="3.8e-11 cm-2 s-1 TeV-1", reference="1 TeV", alpha=2.47, beta=0.24
    )

for obs in spectrum_observations:
    obs.model = model

fit = Fit(spectrum_observations)
fit.datasets.parameters.apply_autoscale = False
result = fit.run()

numpoints=20
sigma=np.sqrt(2.3)

print(fit.minuit.parameters)
fit.minuit.draw_contour("par_000_amplitude", "par_001_alpha")

# or 
contour = fit.minuit.mncontour("par_000_amplitude", "par_001_alpha")
```
it is working fine for one analysis but then it is  giving me error in other analysis
`
from gammapy.spectrum import SpectrumDatasetOnOff, SpectrumDatasetOnOffStacker
from gammapy.utils.fitting import Fit, Model, Parameter
from gammapy.spectrum.models import PowerLaw
from astropy import units as u
from gammapy.spectrum.models import SpectralModel
from gammapy.utils.fitting import Parameter, Parameters
import numpy as np


class Log10ParabolaEnergyScale(SpectralModel):

    def __init__(
        self,
        amplitude=1e-12 * u.Unit("cm-2 s-1 TeV-1"),
        reference=1 * u.TeV,
        alpha=3,
        beta=1,
        z=0.12,
    ):

        self.parameters = Parameters(
            [
                Parameter("amplitude", amplitude),
                Parameter("reference", reference, frozen=True),
                Parameter("alpha", alpha),
                Parameter("beta", beta),
                Parameter("z", z),
            ]
        )

    @staticmethod
    def evaluate(energy, amplitude, reference, alpha, beta, z):
        """Evaluate the model (static function)."""
        scale_factor = 1 / (1 + z)
        scaled_energy = energy * scale_factor
        try:
            xx = scaled_energy / reference
            exponent = -alpha - beta * np.log10(xx)
        except AttributeError:
            from uncertainties.unumpy import log10

            xx = scaled_energy / reference
            exponent = -alpha - beta * log10(xx)
        return amplitude * np.power(xx, exponent) * scale_factor

class SystematicsSpetrumModel(SpectralModel):
    def __init__(self):
        self.parameters = Parameters(
            [
                Parameter("amplitude", 3e-12, unit="cm-2 s-1 TeV-1"),
                Parameter("reference", 1, unit="TeV", frozen=True),
                Parameter("alpha", 2.4),
                Parameter("beta", 0.2),
                Parameter("z_magic", 0),
            ]
        )

class SystematicsSpectrumFit():
    delta = {"magic": 0.15}

    def __init__(self):
        self._model = SystematicsSpetrumModel()
        self.fits = {name: self.make_fit(name) for name in ["magic"]}

    def make_fit(self, name):
        obs_ids = [5029747, 5029748]
        observations = []
        for obs in obs_ids:
            filename = "results/spectra/magic/pha_obs{}.fits"
            ds = SpectrumDatasetOnOff.from_ogip_files(filename.format(obs))
            observations.append(ds)
        model = Log10ParabolaEnergyScale(amplitude="3e-12 cm-2 s-1 TeV-1", reference="1 TeV", alpha=2.4, beta=0.2, z=0)
        #fit_range = dataset.energy_range
        for obs in observations:
            obs.model = model
        return Fit(observations)

    def total_stat(self, parameters):
        stat = 0

        for name, fit in self.fits.items():
            # Parameter update
            pars = fit._model.parameters
            pars["amplitude"].value = parameters["amplitude"].value
            pars["alpha"].value = parameters["alpha"].value
            pars["beta"].value = parameters["beta"].value
            pars["z"].value = parameters["z_" + name].value

            # Poisson likelihood
            stat += fit.total_stat(pars)

            # Gaussian likelihood (energy scale systematics)
            stat += (pars["z"].value / self.delta[name]) ** 2

        return stat

fit = SystematicsSpectrumFit().make_fit("magic")
fit.datasets.parameters.apply_autoscale = False
result = fit.run()
print(result)

numpoints=20
sigma=np.sqrt(2.3)

print(fit.minuit.parameters)
fit.minuit.draw_contour("par_000_amplitude", "par_001_alpha")

contour = fit.minuit.mncontour("par_000_amplitude", "par_001_alpha", numpoints=numpoints, sigma=sigma)
`
This is code example is impossible for me to debug. Maybe you can try to find a small self-contained example, that just uses data from `$GAMMAPY_DATA`. In generaI I don't see a reason, why it should fail. Please check the parameter names with `print(fit.minuit.parameters)`, before you pass them to `mncontour()`. This should always work.

 Otherwise I'm afraid you'll have to wait for the fix...
Thank you for your help. I guess I have to wait for the fix.

Resolving patch:
diff --git a/gammapy/utils/fitting/iminuit.py b/gammapy/utils/fitting/iminuit.py
--- a/gammapy/utils/fitting/iminuit.py
+++ b/gammapy/utils/fitting/iminuit.py
@@ -99,11 +99,13 @@ def confidence_iminuit(minuit, parameters, parameter, sigma, maxcall=0):
 
 
 def mncontour(minuit, parameters, x, y, numpoints, sigma):
-    idx = parameters._get_idx(x)
-    x = _make_parname(idx, parameters[idx])
+    par_x = parameters[x]
+    idx_x = parameters.free_parameters.index(par_x)
+    x = _make_parname(idx_x, par_x)
 
-    idx = parameters._get_idx(y)
-    y = _make_parname(idx, parameters[idx])
+    par_y = parameters[y]
+    idx_y = parameters.free_parameters.index(par_y)
+    y = _make_parname(idx_y, par_y)
 
     x_info, y_info, contour = minuit.mncontour(x, y, numpoints, sigma)
     contour = np.array(contour)


Test patch:
diff --git a/gammapy/utils/fitting/tests/test_fit.py b/gammapy/utils/fitting/tests/test_fit.py
--- a/gammapy/utils/fitting/tests/test_fit.py
+++ b/gammapy/utils/fitting/tests/test_fit.py
@@ -133,20 +133,21 @@ def test_likelihood_profile_reoptimize():
 
 def test_minos_contour():
     dataset = MyDataset()
+    dataset.parameters["x"].frozen = True
     fit = Fit(dataset)
     fit.optimize(backend="minuit")
-    result = fit.minos_contour("x", "y")
+    result = fit.minos_contour("y", "z")
 
     assert result["success"] is True
 
     x = result["x"]
     assert_allclose(len(x), 10)
-    assert_allclose(x[0], 1, rtol=1e-5)
-    assert_allclose(x[-1], 1.499963, rtol=1e-5)
+    assert_allclose(x[0], 299, rtol=1e-5)
+    assert_allclose(x[-1], 299.133975, rtol=1e-5)
     y = result["y"]
     assert_allclose(len(y), 10)
-    assert_allclose(y[0], 300, rtol=1e-5)
-    assert_allclose(y[-1], 300.866004, rtol=1e-5)
+    assert_allclose(y[0], 0.04, rtol=1e-5)
+    assert_allclose(y[-1], 0.54, rtol=1e-5)
 
     # Check that original value state wasn't changed
-    assert_allclose(dataset.parameters["x"].value, 2)
+    assert_allclose(dataset.parameters["y"].value, 300)