Repository: scikit-hep/pyhf

Issue:
Expose minuit instance in fit
# Description

A fit via `pyhf.infer.mle.fit()` with minuit backend currently does not offer access to the minuit instance to for further operations, e.g. access to the correlation matrix. It would be convenient to optionally return it.

## Is your feature request related to a problem? Please describe.
The provided `pyhf.infer.mle.fit()` can only be used for studies with limited scope, as  important information such as parameter correlations are inaccessible.

### Describe the solution you'd like
A keyword argument to optionally return the minuit instance, similar to `return_uncertainties`. Alternatively, the correlation matrix might be returned directly, though the full minuit instance might be more convenient (e.g. for Minos errors).

### Describe alternatives you've considered
A user wanting more detailed control over the fit could implement it separately and not go through `infer` to bypass this limitation. This might be a good solution to prevent supporting every last use case (and will offer more detailed control over minuit, which might be desired).

# Relevant Issues and Pull Requests
#881

# Additional context
none



Discussion:
You can't get the optimizer used via `_, opt = pyhf.get_backend()` ?
That returns a `pyhf.optimize.opt_minuit.minuit_optimizer`, and the minuit instance is built within `_make_minuit()` as `mm`, but not saved as a member variable after `minimize()`. I think I cannot get the right object this way.
I thin i'd rather provide APIs for the common information (eg `return_correlation` or `return_hessian`) instead of directly exposing the underlying implementation / minuit instance. 

the ctor of the minuit optimizer should be able to take on fine-grained global options 

pyhf.optimize.opt_minuit.minuit_optimizer(option1 = , option2 = )

what do you think @alexander-held ?


An API to get the available information, like `return_correlation`, would be very convenient. This prevents users from having to worry about writing backend-agnostic implementations (to the extent that this is possible, since not all backends provide all the information).

> the ctor of the minuit optimizer should be able to take on fine-grained global options

Is this already possible now, or a goal? I have not managed so far to pass information to the Minuit instance through `pyhf`. I am in particular thinking of customizing things like strategy / tolerance.
As of `v0.5.0`, the `minimize()` API provides a `return_result_obj` which contains a `minuit` attribute for the underlying minuit instance.
The updates in #951 and #988 cover most of what I can think of. The last related thing that would be convenient is support for propagating `strategy` / `tol` through to `Minuit`, see https://github.com/scikit-hep/iminuit/issues/466. Thanks a lot for these updates!
Given the outcome of https://github.com/scikit-hep/iminuit/issues/466, I am not sure whether support for `strategy` / `tol` through `pyhf.infer` is possible. I ended up re-implementing the relevant functionality to get access to these settings.

Resolving patch:
diff --git a/src/pyhf/optimize/opt_minuit.py b/src/pyhf/optimize/opt_minuit.py
--- a/src/pyhf/optimize/opt_minuit.py
+++ b/src/pyhf/optimize/opt_minuit.py
@@ -10,7 +10,7 @@ class minuit_optimizer(OptimizerMixin):
     Optimizer that uses iminuit.Minuit.migrad.
     """
 
-    __slots__ = ['name', 'errordef', 'steps', 'strategy']
+    __slots__ = ['name', 'errordef', 'steps', 'strategy', 'tolerance']
 
     def __init__(self, *args, **kwargs):
         """
@@ -28,11 +28,13 @@ def __init__(self, *args, **kwargs):
             errordef (:obj:`float`): See minuit docs. Default is 1.0.
             steps (:obj:`int`): Number of steps for the bounds. Default is 1000.
             strategy (:obj:`int`): See :attr:`iminuit.Minuit.strategy`. Default is None.
+            tolerance (:obj:`float`): tolerance for termination. See specific optimizer for detailed meaning. Default is 0.1.
         """
         self.name = 'minuit'
         self.errordef = kwargs.pop('errordef', 1)
         self.steps = kwargs.pop('steps', 1000)
         self.strategy = kwargs.pop('strategy', None)
+        self.tolerance = kwargs.pop('tolerance', 0.1)
         super().__init__(*args, **kwargs)
 
     def _get_minimizer(
@@ -101,12 +103,14 @@ def _minimize(
         strategy = options.pop(
             'strategy', self.strategy if self.strategy else not do_grad
         )
+        tolerance = options.pop('tolerance', self.tolerance)
         if options:
             raise exceptions.Unsupported(
                 f"Unsupported options were passed in: {list(options.keys())}."
             )
 
         minimizer.strategy = strategy
+        minimizer.tol = tolerance
         minimizer.migrad(ncall=maxiter)
         # Following lines below come from:
         # https://github.com/scikit-hep/iminuit/blob/22f6ed7146c1d1f3274309656d8c04461dde5ba3/src/iminuit/_minimize.py#L106-L125
diff --git a/src/pyhf/optimize/opt_scipy.py b/src/pyhf/optimize/opt_scipy.py
--- a/src/pyhf/optimize/opt_scipy.py
+++ b/src/pyhf/optimize/opt_scipy.py
@@ -9,15 +9,19 @@ class scipy_optimizer(OptimizerMixin):
     Optimizer that uses :func:`scipy.optimize.minimize`.
     """
 
-    __slots__ = ['name']
+    __slots__ = ['name', 'tolerance']
 
     def __init__(self, *args, **kwargs):
         """
         Initialize the scipy_optimizer.
 
-        See :class:`pyhf.optimize.mixins.OptimizerMixin` for configuration options.
+        See :class:`pyhf.optimize.mixins.OptimizerMixin` for other configuration options.
+
+        Args:
+            tolerance (:obj:`float`): tolerance for termination. See specific optimizer for detailed meaning. Default is None.
         """
         self.name = 'scipy'
+        self.tolerance = kwargs.pop('tolerance', None)
         super().__init__(*args, **kwargs)
 
     def _get_minimizer(
@@ -40,9 +44,10 @@ def _minimize(
         Same signature as :func:`scipy.optimize.minimize`.
 
         Minimizer Options:
-            maxiter (`int`): maximum number of iterations. Default is 100000.
-            verbose (`bool`): print verbose output during minimization. Default is off.
-            method (`str`): minimization routine. Default is 'SLSQP'.
+            maxiter (:obj:`int`): maximum number of iterations. Default is 100000.
+            verbose (:obj:`bool`): print verbose output during minimization. Default is off.
+            method (:obj:`str`): minimization routine. Default is 'SLSQP'.
+            tolerance (:obj:`float`): tolerance for termination. See specific optimizer for detailed meaning. Default is None.
 
         Returns:
             fitresult (scipy.optimize.OptimizeResult): the fit result
@@ -50,6 +55,7 @@ def _minimize(
         maxiter = options.pop('maxiter', self.maxiter)
         verbose = options.pop('verbose', self.verbose)
         method = options.pop('method', 'SLSQP')
+        tolerance = options.pop('tolerance', self.tolerance)
         if options:
             raise exceptions.Unsupported(
                 f"Unsupported options were passed in: {list(options.keys())}."
@@ -73,5 +79,6 @@ def _minimize(
             jac=do_grad,
             bounds=bounds,
             constraints=constraints,
+            tol=tolerance,
             options=dict(maxiter=maxiter, disp=bool(verbose)),
         )


Test patch:
diff --git a/tests/test_optim.py b/tests/test_optim.py
--- a/tests/test_optim.py
+++ b/tests/test_optim.py
@@ -225,6 +225,19 @@ def test_minuit_strategy_global(mocker, backend, strategy):
     assert spy.spy_return.minuit.strategy == 1
 
 
+def test_set_tolerance(backend):
+    m = pyhf.simplemodels.hepdata_like([50.0], [100.0], [10.0])
+    data = pyhf.tensorlib.astensor([125.0] + m.config.auxdata)
+
+    assert pyhf.infer.mle.fit(data, m, tolerance=0.01) is not None
+
+    pyhf.set_backend(pyhf.tensorlib, pyhf.optimize.scipy_optimizer(tolerance=0.01))
+    assert pyhf.infer.mle.fit(data, m) is not None
+
+    pyhf.set_backend(pyhf.tensorlib, pyhf.optimize.minuit_optimizer(tolerance=0.01))
+    assert pyhf.infer.mle.fit(data, m) is not None
+
+
 @pytest.mark.parametrize(
     'optimizer',
     [pyhf.optimize.scipy_optimizer, pyhf.optimize.minuit_optimizer],