Repository: optimagic-dev/optimagic (language: python)

Issue:
Better result object for optimization
### Current situation

- `maximize` and `minimize` return a dictionary with information
- The dictionary keys could be improved (e.g. `solution_params` should be just `params` if the whole dictionary the solution of the optimization problem
- When printed the result is not very readable
- The history of criterion evaluations is not part of the result

### Describe the solution you'd like

`maximize` and `minimize` should return a custom object (`OptimizationResult`). The main advantages will be:
- By implementing a `__repr__` we can make terminal/notebook output more informative. In particular, the success and message entries of the current result dictionary should be displayed very prominently. 
- All the current functionality stays available as attributes, but we can design the names more carefully than in the current dict.
- We can add a few methods to provide additional functionality:
    - `.convergence_plot()`
    - `.to_dict()`
    - `.from_dict()`
    - ...

### Backwards compatibility

We could make the results object fully backwards compatible by implementing `__getitem__` to make it behave like a dict. However, I think the userbase is currently small enough that we could simply make a breaking change. 

### Question

Do we need an enhancement proposal for this or can I make a PR? Main reason why an enhancement proposal would be cumbersome is that I think the design will evolve during implementation. 

Improve Benchmarking 
### Problems:

1. The optimization histories for benchmarks are from a log database. The intention behind this was to get thread safety for optimizers that parallelize. However, when running benchmarks it is almost always a good idea not to parallelize on the level of function evaluations but on the benchmark problem level. 
2. The processing of optimization histories for the benchmark assumes that parameters are a DataFrame. It also does all calculations in pandas which might not be ideal. It would currently not run with general pytree parameters.

### Solution

1. Add an option to `maximize` and `minimize` that allows to collect the optimization histories (i.e. function evaluations and parameters) in memory. The mechanism can be similar to how the cache works, i.e. a list is partialled into `internal_criterion_and_derivative_template` and in each iteration, the parameters and function value are appended to that list. In contrast to the cache, this behavior is switched on by the user and not determined from the `algo_info`. If it is on, the histories are added to the optimization result. The following things should be available:
    - the scalar criterion value 
    - the raw criterion output 
    - the external params
2. Rewrite the processing of the optimization histories for general pytrees. Add a fast path in case that parameters are numpy arrays. Use mainly numpy, not pandas for the calculations.
3. Adjust convergence_plot 
4. Adjust profile_plot



Discussion / hints:

Fix patch:
diff --git a/src/estimagic/batch_evaluators.py b/src/estimagic/batch_evaluators.py
index 202b4b028..6068affe1 100644
--- a/src/estimagic/batch_evaluators.py
+++ b/src/estimagic/batch_evaluators.py
@@ -124,7 +124,10 @@ def joblib_batch_evaluator(
     def internal_func(*args, **kwargs):
         return func(*args, **kwargs)
 
-    res = Parallel(n_jobs=n_cores)(delayed(internal_func)(arg) for arg in arguments)
+    if n_cores == 1:
+        res = [internal_func(arg) for arg in arguments]
+    else:
+        res = Parallel(n_jobs=n_cores)(delayed(internal_func)(arg) for arg in arguments)
 
     return res
 
diff --git a/src/estimagic/decorators.py b/src/estimagic/decorators.py
index 06e764eae..c37f776d0 100644
--- a/src/estimagic/decorators.py
+++ b/src/estimagic/decorators.py
@@ -11,6 +11,7 @@
 
 """
 import functools
+import inspect
 import warnings
 from typing import NamedTuple
 
@@ -143,9 +144,10 @@ class AlgoInfo(NamedTuple):
     primary_criterion_entry: str
     name: str
     parallelizes: bool
-    disable_cache: bool
     needs_scaling: bool
     is_available: bool
+    arguments: list
+    is_global: bool = False
 
 
 def mark_minimizer(
@@ -153,10 +155,9 @@ def mark_minimizer(
     *,
     primary_criterion_entry="value",
     name=None,
-    parallelizes=False,
-    disable_cache=False,
     needs_scaling=False,
     is_available=True,
+    is_global=False,
 ):
     """Decorator to mark a function as internal estimagic minimizer and add information.
 
@@ -169,8 +170,6 @@ def mark_minimizer(
         name (str): The name of the internal algorithm.
         parallelizes (bool): Must be True if an algorithm evaluates the criterion,
             derivative or criterion_and_derivative in parallel.
-        disable_cache (bool): If True, no caching for the criterion function
-            or its derivatives are used.
         needs_scaling (bool): Must be True if the algorithm is not reasonable
             independent of the scaling of the parameters.
         is_available (bool): Whether the algorithm is available. This is needed for
@@ -191,28 +190,31 @@ def mark_minimizer(
             f"{primary_criterion_entry}."
         )
 
-    if not isinstance(parallelizes, bool):
-        raise TypeError("parallelizes must be a bool.")
-
-    if not isinstance(disable_cache, bool):
-        raise TypeError("disable_cache must be a bool.")
-
     if not isinstance(needs_scaling, bool):
         raise TypeError("needs_scaling must be a bool.")
 
     if not isinstance(is_available, bool):
         raise TypeError("is_available must be a bool.")
 
-    algo_info = AlgoInfo(
-        primary_criterion_entry=primary_criterion_entry,
-        name=name,
-        parallelizes=parallelizes,
-        disable_cache=disable_cache,
-        needs_scaling=needs_scaling,
-        is_available=is_available,
-    )
-
     def decorator_mark_minimizer(func):
+        arguments = list(inspect.signature(func).parameters)
+
+        if isinstance(func, functools.partial):
+            partialed_in = set(func.keywords)
+            arguments = [a for a in arguments if a not in partialed_in]
+
+        parallelizes = "n_cores" in arguments
+
+        algo_info = AlgoInfo(
+            primary_criterion_entry=primary_criterion_entry,
+            name=name,
+            parallelizes=parallelizes,
+            needs_scaling=needs_scaling,
+            is_available=is_available,
+            arguments=arguments,
+            is_global=is_global,
+        )
+
         @functools.wraps(func)
         def wrapper_mark_minimizer(*args, **kwargs):
             return func(*args, **kwargs)
diff --git a/src/estimagic/differentiation/generate_steps.py b/src/estimagic/differentiation/generate_steps.py
index 137136127..d7e38388a 100644
--- a/src/estimagic/differentiation/generate_steps.py
+++ b/src/estimagic/differentiation/generate_steps.py
@@ -1,7 +1,12 @@
 import warnings
+from typing import NamedTuple
 
 import numpy as np
-from estimagic.utilities import namedtuple_from_kwargs
+
+
+class Steps(NamedTuple):
+    pos: np.ndarray
+    neg: np.ndarray
 
 
 def generate_steps(
@@ -110,7 +115,7 @@ def generate_steps(
         pos[pos > upper_step_bounds.reshape(-1, 1)] = np.nan
         neg[neg < lower_step_bounds.reshape(-1, 1)] = np.nan
 
-    steps = namedtuple_from_kwargs(pos=pos.T, neg=neg.T)
+    steps = Steps(pos=pos.T, neg=neg.T)
 
     return steps
 
diff --git a/src/estimagic/optimization/__init__.py b/src/estimagic/optimization/__init__.py
index 7b386c29d..969ac67c3 100644
--- a/src/estimagic/optimization/__init__.py
+++ b/src/estimagic/optimization/__init__.py
@@ -36,11 +36,7 @@
                 AVAILABLE_ALGORITHMS[name] = func
 
 
-GLOBAL_ALGORITHMS = [
-    "nlopt_direct",
-    "nlopt_esch",
-    "nlopt_isres",
-    "nlopt_crs2_lm",
-]
-
-GLOBAL_ALGORITHMS += [name for name in AVAILABLE_ALGORITHMS if name.startswith("pygmo")]
+GLOBAL_ALGORITHMS = []
+for name, func in ALL_ALGORITHMS.items():
+    if func._algorithm_info.is_global:
+        GLOBAL_ALGORITHMS.append(name)
diff --git a/src/estimagic/optimization/bhhh.py b/src/estimagic/optimization/bhhh.py
index b1e0f7da9..6e09412bd 100644
--- a/src/estimagic/optimization/bhhh.py
+++ b/src/estimagic/optimization/bhhh.py
@@ -6,9 +6,7 @@
 @mark_minimizer(
     name="bhhh",
     primary_criterion_entry="contributions",
-    parallelizes=False,
     needs_scaling=False,
-    disable_cache=False,
     is_available=True,
 )
 def bhhh(
diff --git a/src/estimagic/optimization/cyipopt_optimizers.py b/src/estimagic/optimization/cyipopt_optimizers.py
index 9bb5e7a02..125df0217 100644
--- a/src/estimagic/optimization/cyipopt_optimizers.py
+++ b/src/estimagic/optimization/cyipopt_optimizers.py
@@ -14,9 +14,7 @@
 @mark_minimizer(
     name="ipopt",
     primary_criterion_entry="value",
-    parallelizes=False,
     needs_scaling=False,
-    disable_cache=False,
     is_available=IS_CYIPOPT_INSTALLED,
 )
 def ipopt(
diff --git a/src/estimagic/optimization/error_penalty.py b/src/estimagic/optimization/error_penalty.py
new file mode 100644
index 000000000..7663eeb32
--- /dev/null
+++ b/src/estimagic/optimization/error_penalty.py
@@ -0,0 +1,118 @@
+from functools import partial
+
+import numpy as np
+from estimagic.config import CRITERION_PENALTY_CONSTANT
+from estimagic.config import CRITERION_PENALTY_SLOPE
+from estimagic.parameters.conversion import aggregate_func_output_to_value
+
+
+def _penalty_value(x, constant, slope, x0, dim_out=None):  # noqa: U100
+    return constant + slope * np.linalg.norm(x - x0)
+
+
+def _penalty_contributions(x, constant, slope, x0, dim_out):
+    contrib = (constant + slope * np.linalg.norm(x - x0)) / dim_out
+    return np.ones(dim_out) * contrib
+
+
+def _penalty_root_contributions(x, constant, slope, x0, dim_out):
+    contrib = np.sqrt((constant + slope * np.linalg.norm(x - x0)) / dim_out)
+    return np.ones(dim_out) * contrib
+
+
+def _penalty_value_derivative(x, constant, slope, x0, dim_out=None):  # noqa: U100
+    return slope * (x - x0) / np.linalg.norm(x - x0)
+
+
+def _penalty_contributions_derivative(x, constant, slope, x0, dim_out):  # noqa: U100
+    row = slope * (x - x0) / (dim_out * np.linalg.norm(x - x0))
+    return np.full((dim_out, len(x)), row)
+
+
+def _penalty_root_contributions_derivative(x, constant, slope, x0, dim_out):
+    inner_deriv = slope * (x - x0) / np.linalg.norm(x - x0)
+    outer_deriv = 0.5 / np.sqrt(_penalty_value(x, constant, slope, x0) * dim_out)
+    row = outer_deriv * inner_deriv
+    return np.full((dim_out, len(x)), row)
+
+
+def get_error_penalty_function(
+    error_handling,
+    start_x,
+    start_criterion,
+    error_penalty,
+    primary_key,
+    direction,
+):
+    if error_handling == "raise":
+        return None
+    elif error_handling != "continue":
+        raise ValueError("Error handling must be 'raise' or 'continue'")
+
+    error_penalty = {} if error_penalty is None else error_penalty
+
+    first_value = aggregate_func_output_to_value(
+        f_eval=start_criterion, primary_key=primary_key
+    )
+
+    constant, slope = _process_error_penalty(
+        error_penalty=error_penalty,
+        first_value=first_value,
+        direction=direction,
+    )
+
+    dim_out = 1 if primary_key == "value" else len(start_criterion)
+
+    kwargs = {"constant": constant, "slope": slope, "x0": start_x, "dim_out": dim_out}
+
+    if primary_key == "value":
+        _penalty = partial(_penalty_value, **kwargs)
+        _derivative = partial(_penalty_value_derivative, **kwargs)
+    elif primary_key == "contributions":
+        _penalty = partial(_penalty_contributions, **kwargs)
+        _derivative = partial(
+            _penalty_contributions_derivative,
+            **kwargs,
+        )
+    elif primary_key == "root_contributions":
+        _penalty = partial(
+            _penalty_root_contributions,
+            **kwargs,
+        )
+        _derivative = partial(_penalty_root_contributions_derivative, **kwargs)
+
+    else:
+        raise ValueError()
+
+    def penalty(x, task):
+        if task == "criterion":
+            out = _penalty(x)
+        elif task == "derivative":
+            out = _derivative(x)
+        elif task == "criterion_and_derivative":
+            out = (_penalty(x), _derivative(x))
+
+        return out
+
+    return penalty
+
+
+def _process_error_penalty(error_penalty, first_value, direction):
+    """Add default options to error_penalty options."""
+    error_penalty = error_penalty.copy()
+
+    if direction == "minimize":
+        default_constant = (
+            first_value + np.abs(first_value) + CRITERION_PENALTY_CONSTANT
+        )
+        default_slope = CRITERION_PENALTY_SLOPE
+    else:
+        default_constant = (
+            first_value - np.abs(first_value) - CRITERION_PENALTY_CONSTANT
+        )
+        default_slope = -CRITERION_PENALTY_SLOPE
+
+    constant = error_penalty.get("constant", default_constant)
+    slope = error_penalty.get("slope", default_slope)
+
+    return constant, slope
diff --git a/src/estimagic/optimization/fides_optimizers.py b/src/estimagic/optimization/fides_optimizers.py
index 6f3c1f6f1..2647fcc33 100644
--- a/src/estimagic/optimization/fides_optimizers.py
+++ b/src/estimagic/optimization/fides_optimizers.py
@@ -20,9 +20,7 @@
 @mark_minimizer(
     name="fides",
     primary_criterion_entry="value",
-    disable_cache=False,
     needs_scaling=False,
-    parallelizes=False,
     is_available=IS_FIDES_INSTALLED,
 )
 def fides(
diff --git a/src/estimagic/optimization/get_algorithm.py b/src/estimagic/optimization/get_algorithm.py
index b487a5ccd..c3ed90d08 100644
--- a/src/estimagic/optimization/get_algorithm.py
+++ b/src/estimagic/optimization/get_algorithm.py
@@ -1,5 +1,4 @@
 import functools
-import inspect
 import warnings
 from functools import partial
 
@@ -39,13 +38,7 @@ def process_user_algorithm(algorithm):
 
     algo_info = algorithm._algorithm_info
 
-    arguments = set(inspect.signature(algorithm).parameters)
-
-    if isinstance(algorithm, partial):
-        partialed_in = set(algorithm.args).union(set(algorithm.keywords))
-        arguments = arguments.difference(partialed_in)
-
-    return algorithm, algo_info, arguments
+    return algorithm, algo_info
 
 
 def get_final_algorithm(
@@ -57,6 +50,7 @@ def get_final_algorithm(
     algo_options,
     logging,
     db_kwargs,
+    collect_history,
 ):
     """Get algorithm-function with partialled options.
 
@@ -88,18 +82,29 @@ def get_final_algorithm(
         valid_kwargs=valid_kwargs,
     )
 
-    partialled_algorithm = partial(raw_algorithm, **internal_options)
+    algorithm = partial(raw_algorithm, **internal_options)
 
-    raw_algorithm = _add_logging_to_algorithm(
-        partialled_algorithm,
+    algorithm = _add_logging(
+        algorithm,
         logging=logging,
         db_kwargs=db_kwargs,
     )
 
-    return raw_algorithm
+    can_collect = internal_options.get("n_cores") in (None, 1)
+
+    if not can_collect and collect_history is True:
+        raise ValueError(
+            "Histories can only be collected with optimizers that do not evaluate "
+            "The criterion function or its derivatives in parallel."
+        )
+
+    if can_collect and collect_history is not False:
+        algorithm = _add_history_collection(algorithm)
+
+    return algorithm
 
 
-def _add_logging_to_algorithm(algorithm=None, *, logging=None, db_kwargs=None):
+def _add_logging(algorithm=None, *, logging=None, db_kwargs=None):
     """Add logging of status to the algorithm."""
 
     def decorator_add_logging_to_algorithm(algorithm):
@@ -143,6 +148,27 @@ def wrapper_add_logging_algorithm(**kwargs):
         return decorator_add_logging_to_algorithm
 
 
+def _add_history_collection(algorithm):
+    """Partial a history container into all functions that define the optimization."""
+
+    @functools.wraps(algorithm)
+    def wrapper_add_history_collection(**kwargs):
+        func_names = {"criterion", "derivative", "criterion_and_derivative"}
+
+        container = []
+
+        _kwargs = kwargs.copy()
+        for name in func_names:
+            if name in kwargs:
+                _kwargs[name] = partial(kwargs[name], history_container=container)
+
+        out = algorithm(**_kwargs)
+        out["history"] = container
+        return out
+
+    return wrapper_add_history_collection
+
+
 def _adjust_options_to_algorithm(
     algo_options,
     lower_bounds,
diff --git a/src/estimagic/optimization/internal_criterion_template.py b/src/estimagic/optimization/internal_criterion_template.py
index aa86abf66..2bf64065b 100644
--- a/src/estimagic/optimization/internal_criterion_template.py
+++ b/src/estimagic/optimization/internal_criterion_template.py
@@ -1,13 +1,11 @@
 import datetime
 import warnings
 
-import numpy as np
 from estimagic.differentiation.derivatives import first_derivative
 from estimagic.exceptions import get_traceback
 from estimagic.exceptions import UserFunctionRuntimeError
 from estimagic.logging.database_utilities import append_row
 from estimagic.parameters.conversion import aggregate_func_output_to_value
-from estimagic.utilities import hash_array
 
 
 def internal_criterion_and_derivative_template(
@@ -24,11 +22,9 @@ def internal_criterion_and_derivative_template(
     logging,
     db_kwargs,
     error_handling,
-    error_penalty,
-    first_criterion_evaluation,
-    cache,
-    cache_size,
+    error_penalty_func,
     fixed_log_data,
+    history_container=None,
 ):
     """Template for the internal criterion and derivative function.
 
@@ -55,7 +51,6 @@ def internal_criterion_and_derivative_template(
             - primary_criterion_entry
             - name
             - parallelizes
-            - disable_cache
             - needs_scaling
             - is_available
         derivative (callable, optional): (partialed) user provided function that
@@ -77,21 +72,14 @@ def internal_criterion_and_derivative_template(
         error_handling (str): Either "raise" or "continue". Note that "continue" does
             not absolutely guarantee that no error is raised but we try to handle as
             many errors as possible in that case without aborting the optimization.
-        error_penalty (dict): Dict with the entries "constant" (float) and "slope"
-            (float). If the criterion or derivative raise an error and error_handling is
-            "continue", return ``constant + slope * norm(params - start_params)`` where
-            ``norm`` is the euclidean distance as criterion value and adjust the
-            derivative accordingly. This is meant to guide the optimizer back into a
-            valid region of parameter space (in direction of the start parameters).
-            Note that the constant has to be high enough to ensure that the penalty is
-            actually a bad function value. The default constant is 2 times the criterion
-            value at the start parameters. The default slope is 0.1.
-        first_criterion_evaluation (dict): Dictionary with entries "internal_params",
-            "external_params", "output".
-        cache (dict): Dictionary used as cache for criterion and derivative evaluations.
-        cache_size (int): Number of evaluations that are kept in cache. Default 10.
+        error_penalty_func (callable): Function that takes ``x`` and ``task`` and
+            returns a penalized criterion function, its derivative or both (depending)
+            on task.
         fixed_log_data (dict): Dictionary with fixed data to be saved in the database.
             Has the entries "stage" (str) and "substage" (int).
+        history_container (list or None): List to which parameter, criterion and
+            derivative histories are appended. Should be set to None if an algorithm
+            parallelizes over criterion or derivative evaluations.
 
     Returns:
         float, np.ndarray or tuple: If task=="criterion" it returns the output of
@@ -100,10 +88,7 @@ def internal_criterion_and_derivative_template(
             If task=="criterion_and_derivative" it returns both as a tuple.
 
     """
-    x_hash = hash_array(x)
-    cache_entry = cache.get(x_hash, {})
-
-    to_dos = _determine_to_dos(task, cache_entry, derivative, criterion_and_derivative)
+    to_dos = _determine_to_dos(task, derivative, criterion_and_derivative)
 
     caught_exceptions = []
     new_criterion, new_external_criterion = None, None
@@ -124,9 +109,6 @@ def func(x):
             return out
 
         options = numdiff_options.copy()
-        f0 = cache_entry.get("criterion", None)
-        if f0 is not None:
-            options["f0"] = {"relevant": f0, "full": None}
         options["key"] = "relevant"
         options["return_func_value"] = True
 
@@ -140,7 +122,7 @@ def func(x):
         except Exception as e:
             tb = get_traceback()
             caught_exceptions.append(tb)
-            if "criterion" in cache_entry or error_handling == "raise":
+            if error_handling == "raise":
                 msg = (
                     "An error occurred when evaluating criterion to calculate a "
                     "numerical derivative during optimization."
@@ -164,7 +146,7 @@ def func(x):
         except Exception as e:
             tb = get_traceback()
             caught_exceptions.append(tb)
-            if "criterion" in cache_entry or error_handling == "raise":
+            if error_handling == "raise":
                 msg = (
                     "An error ocurred when evaluating criterion_and_derivative "
                     "during optimization."
@@ -187,7 +169,7 @@ def func(x):
             except Exception as e:
                 tb = get_traceback()
                 caught_exceptions.append(tb)
-                if "derivative" in cache_entry or error_handling == "raise":
+                if error_handling == "raise":
                     msg = (
                         "An error ocurred when evaluating criterion during "
                         "optimization."
@@ -209,7 +191,7 @@ def func(x):
             except Exception as e:
                 tb = get_traceback()
                 caught_exceptions.append(tb)
-                if "criterion" in cache_entry or error_handling == "raise":
+                if error_handling == "raise":
                     msg = (
                         "An error ocurred when evaluating derivative during "
                         "optimization"
@@ -230,21 +212,19 @@ def func(x):
         new_derivative = converter.derivative_to_internal(new_external_derivative, x)
 
     if caught_exceptions:
-        new_criterion, new_derivative = _penalty_and_derivative(
-            x, first_criterion_evaluation, error_penalty, algo_info
+        new_criterion, new_derivative = error_penalty_func(
+            x, task="criterion_and_derivative"
         )
 
-    if not (algo_info.parallelizes or algo_info.disable_cache) and cache_size >= 1:
-        _cache_new_evaluations(new_criterion, new_derivative, x_hash, cache, cache_size)
+    if new_criterion is not None:
+        scalar_critval = aggregate_func_output_to_value(
+            f_eval=new_criterion,
+            primary_key=algo_info.primary_criterion_entry,
+        )
+    else:
+        scalar_critval = None
 
     if (new_criterion is not None or new_derivative is not None) and logging:
-        if new_criterion is not None:
-            scalar_critval = aggregate_func_output_to_value(
-                f_eval=new_criterion,
-                primary_key=algo_info.primary_criterion_entry,
-            )
-        else:
-            scalar_critval = None
 
         _log_new_evaluations(
             new_criterion=new_external_criterion,
@@ -261,13 +241,19 @@ def func(x):
         new_derivative=new_derivative,
         task=task,
         direction=direction,
-        cache=cache,
-        x_hash=x_hash,
     )
+
+    if history_container is not None:
+        hist_entry = {
+            "params": current_params,
+            "criterion": new_criterion,
+            "scalar_criterion": scalar_critval,
+        }
+        history_container.append(hist_entry)
     return res
 
 
-def _determine_to_dos(task, cache_entry, derivative, criterion_and_derivative):
+def _determine_to_dos(task, derivative, criterion_and_derivative):
     """Determine which functions have to be evaluated at the new parameters.
 
     Args:
@@ -288,8 +274,8 @@ def _determine_to_dos(task, cache_entry, derivative, criterion_and_derivative):
             - ["derivative"]
 
     """
-    criterion_needed = "criterion" in task and "criterion" not in cache_entry
-    derivative_needed = "derivative" in task and "derivative" not in cache_entry
+    criterion_needed = "criterion" in task
+    derivative_needed = "derivative" in task
 
     to_dos = []
     if criterion_and_derivative is not None and criterion_needed and derivative_needed:
@@ -310,75 +296,6 @@ def _determine_to_dos(task, cache_entry, derivative, criterion_and_derivative):
     return to_dos
 
 
-def _penalty_and_derivative(x, first_eval, error_penalty, algo_info):
-    constant = error_penalty["constant"]
-    slope = error_penalty["slope"]
-    x0 = first_eval["internal_params"]
-
-    primary = algo_info.primary_criterion_entry
-
-    if primary == "value":
-        penalty = _penalty_value(x, constant, slope, x0)
-        derivative = _penalty_value_derivative(x, constant, slope, x0)
-    elif primary == "contributions":
-        dim_out = len(first_eval["output"][primary])
-        penalty = _penalty_contributions(x, constant, slope, x0, dim_out)
-        derivative = _penalty_contributions_derivative(x, constant, slope, x0, dim_out)
-    elif primary == "root_contributions":
-        dim_out = len(first_eval["output"][primary])
-        penalty = _penalty_root_contributions(x, constant, slope, x0, dim_out)
-        derivative = _penalty_root_contributions_derivative(
-            x, constant, slope, x0, dim_out
-        )
-    else:
-        raise ValueError()
-
-    return penalty, derivative
-
-
-def _penalty_value(x, constant, slope, x0, dim_out=None):
-    return constant + slope * np.linalg.norm(x - x0)
-
-
-def _penalty_contributions(x, constant, slope, x0, dim_out):
-    contrib = (constant + slope * np.linalg.norm(x - x0)) / dim_out
-    return np.ones(dim_out) * contrib
-
-
-def _penalty_root_contributions(x, constant, slope, x0, dim_out):
-    contrib = np.sqrt((constant + slope * np.linalg.norm(x - x0)) / dim_out)
-    return np.ones(dim_out) * contrib
-
-
-def _penalty_value_derivative(x, constant, slope, x0, dim_out=None):
-    return slope * (x - x0) / np.linalg.norm(x - x0)
-
-
-def _penalty_contributions_derivative(x, constant, slope, x0, dim_out):
-    row = slope * (x - x0) / (dim_out * np.linalg.norm(x - x0))
-    return np.full((dim_out, len(x)), row)
-
-
-def _penalty_root_contributions_derivative(x, constant, slope, x0, dim_out):
-    inner_deriv = slope * (x - x0) / np.linalg.norm(x - x0)
-    outer_deriv = 0.5 / np.sqrt(_penalty_value(x, constant, slope, x0) * dim_out)
-    row = outer_deriv * inner_deriv
-    return np.full((dim_out, len(x)), row)
-
-
-def _cache_new_evaluations(new_criterion, new_derivative, x_hash, cache, cache_size):
-    cache_entry = cache.get(x_hash, {}).copy()
-    if len(cache) >= cache_size:
-        # list(dict) returns keys in insertion order: https://tinyurl.com/o464nrz
-        oldest_entry = list(cache)[0]
-        del cache[oldest_entry]
-    if new_criterion is not None:
-        cache_entry["criterion"] = new_criterion
-    if new_derivative is not None:
-        cache_entry["derivative"] = new_derivative
-    cache[x_hash] = cache_entry
-
-
 def _log_new_evaluations(
     new_criterion,
     new_derivative,
@@ -422,16 +339,11 @@ def _get_output_for_optimizer(
     new_derivative,
     task,
     direction,
-    cache,
-    x_hash,
 ):
-    if "criterion" in task and new_criterion is None:
-        new_criterion = cache[x_hash]["criterion"]
+
     if "criterion" in task and direction == "maximize":
         new_criterion = -new_criterion
 
-    if "derivative" in task and new_derivative is None:
-        new_derivative = cache[x_hash]["derivative"]
     if "derivative" in task and direction == "maximize":
         new_derivative = -new_derivative
 
diff --git a/src/estimagic/optimization/nag_optimizers.py b/src/estimagic/optimization/nag_optimizers.py
index f2030cf95..a8265af02 100644
--- a/src/estimagic/optimization/nag_optimizers.py
+++ b/src/estimagic/optimization/nag_optimizers.py
@@ -59,7 +59,6 @@
 @mark_minimizer(
     name="nag_dfols",
     primary_criterion_entry="root_contributions",
-    disable_cache=True,
     needs_scaling=True,
     is_available=IS_DFOLS_INSTALLED,
 )
@@ -252,7 +251,6 @@ def nag_dfols(
 
 @mark_minimizer(
     name="nag_dfols",
-    disable_cache=True,
     needs_scaling=True,
     is_available=IS_PYBOBYQA_INSTALLED,
 )
diff --git a/src/estimagic/optimization/neldermead.py b/src/estimagic/optimization/neldermead.py
index 93986be58..de7fe465b 100644
--- a/src/estimagic/optimization/neldermead.py
+++ b/src/estimagic/optimization/neldermead.py
@@ -16,8 +16,6 @@
 @mark_minimizer(
     name="parallel_neldermead",
     primary_criterion_entry="value",
-    disable_cache=False,
-    parallelizes=True,
     needs_scaling=True,
     is_available=True,
 )
diff --git a/src/estimagic/optimization/nlopt_optimizers.py b/src/estimagic/optimization/nlopt_optimizers.py
index 8d44fee8d..73443fc8a 100644
--- a/src/estimagic/optimization/nlopt_optimizers.py
+++ b/src/estimagic/optimization/nlopt_optimizers.py
@@ -22,9 +22,7 @@
 @mark_minimizer(
     name="nlopt_bobyqa",
     primary_criterion_entry="value",
-    parallelizes=False,
     needs_scaling=False,
-    disable_cache=False,
     is_available=IS_NLOPT_INSTALLED,
 )
 def nlopt_bobyqa(
@@ -65,9 +63,7 @@ def nlopt_bobyqa(
 @mark_minimizer(
     name="nlopt_neldermead",
     primary_criterion_entry="value",
-    parallelizes=False,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_NLOPT_INSTALLED,
 )
 def nlopt_neldermead(
@@ -108,9 +104,7 @@ def nlopt_neldermead(
 @mark_minimizer(
     name="nlopt_praxis",
     primary_criterion_entry="value",
-    parallelizes=False,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_NLOPT_INSTALLED,
 )
 def nlopt_praxis(
@@ -148,9 +142,7 @@ def nlopt_praxis(
 @mark_minimizer(
     name="nlopt_cobyla",
     primary_criterion_entry="value",
-    parallelizes=False,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_NLOPT_INSTALLED,
 )
 def nlopt_cobyla(
@@ -191,9 +183,7 @@ def nlopt_cobyla(
 @mark_minimizer(
     name="nlopt_sbplx",
     primary_criterion_entry="value",
-    parallelizes=False,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_NLOPT_INSTALLED,
 )
 def nlopt_sbplx(
@@ -233,9 +223,7 @@ def nlopt_sbplx(
 @mark_minimizer(
     name="nlopt_newuoa",
     primary_criterion_entry="value",
-    parallelizes=False,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_NLOPT_INSTALLED,
 )
 def nlopt_newuoa(
@@ -279,9 +267,7 @@ def nlopt_newuoa(
 @mark_minimizer(
     name="nlopt_tnewton",
     primary_criterion_entry="value",
-    parallelizes=False,
     needs_scaling=False,
-    disable_cache=False,
     is_available=IS_NLOPT_INSTALLED,
 )
 def nlopt_tnewton(
@@ -323,9 +309,7 @@ def nlopt_tnewton(
 @mark_minimizer(
     name="nlopt_lbfgsb",
     primary_criterion_entry="value",
-    parallelizes=False,
     needs_scaling=False,
-    disable_cache=False,
     is_available=IS_NLOPT_INSTALLED,
 )
 def nlopt_lbfgs(
@@ -367,9 +351,7 @@ def nlopt_lbfgs(
 @mark_minimizer(
     name="nlopt_ccsaq",
     primary_criterion_entry="value",
-    parallelizes=False,
     needs_scaling=False,
-    disable_cache=False,
     is_available=IS_NLOPT_INSTALLED,
 )
 def nlopt_ccsaq(
@@ -411,9 +393,7 @@ def nlopt_ccsaq(
 @mark_minimizer(
     name="nlopt_mma",
     primary_criterion_entry="value",
-    parallelizes=False,
     needs_scaling=False,
-    disable_cache=False,
     is_available=IS_NLOPT_INSTALLED,
 )
 def nlopt_mma(
@@ -455,9 +435,7 @@ def nlopt_mma(
 @mark_minimizer(
     name="nlopt_var",
     primary_criterion_entry="value",
-    parallelizes=False,
     needs_scaling=False,
-    disable_cache=False,
     is_available=IS_NLOPT_INSTALLED,
 )
 def nlopt_var(
@@ -504,9 +482,7 @@ def nlopt_var(
 @mark_minimizer(
     name="nlopt_slsqp",
     primary_criterion_entry="value",
-    parallelizes=False,
     needs_scaling=False,
-    disable_cache=False,
     is_available=IS_NLOPT_INSTALLED,
 )
 def nlopt_slsqp(
@@ -546,10 +522,9 @@ def nlopt_slsqp(
 @mark_minimizer(
     name="nlopt_direct",
     primary_criterion_entry="value",
-    parallelizes=False,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_NLOPT_INSTALLED,
+    is_global=True,
 )
 def nlopt_direct(
     criterion,
@@ -604,9 +579,8 @@ def nlopt_direct(
 @mark_minimizer(
     name="nlopt_esch",
     primary_criterion_entry="value",
-    parallelizes=False,
+    is_global=True,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_NLOPT_INSTALLED,
 )
 def nlopt_esch(
@@ -647,9 +621,8 @@ def nlopt_esch(
 @mark_minimizer(
     name="nlopt_isres",
     primary_criterion_entry="value",
-    parallelizes=False,
+    is_global=True,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_NLOPT_INSTALLED,
 )
 def nlopt_isres(
@@ -690,9 +663,8 @@ def nlopt_isres(
 @mark_minimizer(
     name="nlopt_crs2_lm",
     primary_criterion_entry="value",
-    parallelizes=False,
+    is_global=True,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_NLOPT_INSTALLED,
 )
 def nlopt_crs2_lm(
diff --git a/src/estimagic/optimization/optimize.py b/src/estimagic/optimization/optimize.py
index 874854ab2..08402d9f4 100644
--- a/src/estimagic/optimization/optimize.py
+++ b/src/estimagic/optimization/optimize.py
@@ -2,10 +2,7 @@
 import warnings
 from pathlib import Path
 
-import numpy as np
 from estimagic.batch_evaluators import process_batch_evaluator
-from estimagic.config import CRITERION_PENALTY_CONSTANT
-from estimagic.config import CRITERION_PENALTY_SLOPE
 from estimagic.exceptions import InvalidFunctionError
 from estimagic.logging.database_utilities import append_row
 from estimagic.logging.database_utilities import load_database
@@ -13,6 +10,7 @@
 from estimagic.logging.database_utilities import make_optimization_problem_table
 from estimagic.logging.database_utilities import make_steps_table
 from estimagic.optimization.check_arguments import check_optimize_kwargs
+from estimagic.optimization.error_penalty import get_error_penalty_function
 from estimagic.optimization.get_algorithm import get_final_algorithm
 from estimagic.optimization.get_algorithm import process_user_algorithm
 from estimagic.optimization.internal_criterion_template import (
@@ -23,17 +21,9 @@
 from estimagic.optimization.process_results import process_internal_optimizer_result
 from estimagic.optimization.tiktak import run_multistart_optimization
 from estimagic.optimization.tiktak import WEIGHT_FUNCTIONS
-from estimagic.parameters.conversion import aggregate_func_output_to_value
 from estimagic.parameters.conversion import get_converter
 from estimagic.parameters.parameter_groups import get_params_groups
 from estimagic.process_user_function import process_func_of_params
-from estimagic.utilities import hash_array
-
-
-BASE_DOCSTRING = """
-
-
-"""
 
 
 def maximize(
@@ -57,11 +47,12 @@ def maximize(
     log_options=None,
     error_handling="raise",
     error_penalty=None,
-    cache_size=100,
     scaling=False,
     scaling_options=None,
     multistart=False,
     multistart_options=None,
+    collect_history=None,
+    skip_checks=False,
 ):
     """Maximize criterion using algorithm subject to constraints.
 
@@ -141,8 +132,6 @@ def maximize(
             actually a bad function value. The default constant is f0 + abs(f0) + 100
             for minimizations and f0 - abs(f0) - 100 for maximizations, where
             f0 is the criterion value at start parameters. The default slope is 0.1.
-        cache_size (int): Number of criterion and derivative evaluations that are cached
-            in memory in case they are needed.
         scaling (bool): If True, the parameter vector is rescaled internally for
             better performance with scale sensitive optimizers.
         scaling_options (dict or None): Options to configure the internal scaling ot
@@ -198,6 +187,13 @@ def maximize(
             discarded from the sample.
             - optimization_error_handling (str): One of "raise" or "continue". Default
             is continue, which means that failed optimizations are simply discarded.
+        collect_history (bool): Whether the history of parameters and criterion values
+            should be collected and returned as part of the result. Default None,
+            which means that the history is collected as long as optimizers do not
+            parallelize over criterion evaluations.
+        skip_checks (bool): Whether checks on the inputs are skipped. This makes the
+            optimization faster, especially for very fast criterion functions. Default
+            False.
 
     """
     return _optimize(
@@ -221,11 +217,12 @@ def maximize(
         log_options=log_options,
         error_handling=error_handling,
         error_penalty=error_penalty,
-        cache_size=cache_size,
         scaling=scaling,
         scaling_options=scaling_options,
         multistart=multistart,
         multistart_options=multistart_options,
+        collect_history=collect_history,
+        skip_checks=skip_checks,
     )
 
 
@@ -250,11 +247,12 @@ def minimize(
     log_options=None,
     error_handling="raise",
     error_penalty=None,
-    cache_size=100,
     scaling=False,
     scaling_options=None,
     multistart=False,
     multistart_options=None,
+    collect_history=None,
+    skip_checks=False,
 ):
     """Minimize criterion using algorithm subject to constraints.
 
@@ -334,8 +332,6 @@ def minimize(
             actually a bad function value. The default constant is f0 + abs(f0) + 100
             for minimizations and f0 - abs(f0) - 100 for maximizations, where
             f0 is the criterion value at start parameters. The default slope is 0.1.
-        cache_size (int): Number of criterion and derivative evaluations that are cached
-            in memory in case they are needed.
         scaling (bool): If True, the parameter vector is rescaled internally for
             better performance with scale sensitive optimizers.
         scaling_options (dict or None): Options to configure the internal scaling ot
@@ -391,6 +387,13 @@ def minimize(
             discarded from the sample.
             - optimization_error_handling (str): One of "raise" or "continue". Default
             is continue, which means that failed optimizations are simply discarded.
+        collect_history (bool): Whether the history of parameters and criterion values
+            should be collected and returned as part of the result. Default None,
+            which means that the history is collected as long as optimizers do not
+            parallelize over criterion evaluations.
+        skip_checks (bool): Whether checks on the inputs are skipped. This makes the
+            optimization faster, especially for very fast criterion functions. Default
+            False.
 
     """
     return _optimize(
@@ -414,11 +417,12 @@ def minimize(
         log_options=log_options,
         error_handling=error_handling,
         error_penalty=error_penalty,
-        cache_size=cache_size,
         scaling=scaling,
         scaling_options=scaling_options,
         multistart=multistart,
         multistart_options=multistart_options,
+        collect_history=collect_history,
+        skip_checks=skip_checks,
     )
 
 
@@ -444,11 +448,12 @@ def _optimize(
     log_options,
     error_handling,
     error_penalty,
-    cache_size,
     scaling,
     scaling_options,
     multistart,
     multistart_options,
+    collect_history,
+    skip_checks,
 ):
     """Minimize or maximize criterion using algorithm subject to constraints.
 
@@ -474,33 +479,35 @@ def _optimize(
     if logging:
         logging = Path(logging)
 
-    check_optimize_kwargs(
-        direction=direction,
-        criterion=criterion,
-        criterion_kwargs=criterion_kwargs,
-        params=params,
-        algorithm=algorithm,
-        constraints=constraints,
-        algo_options=algo_options,
-        derivative=derivative,
-        derivative_kwargs=derivative_kwargs,
-        criterion_and_derivative=criterion_and_derivative,
-        criterion_and_derivative_kwargs=criterion_and_derivative_kwargs,
-        numdiff_options=numdiff_options,
-        logging=logging,
-        log_options=log_options,
-        error_handling=error_handling,
-        error_penalty=error_penalty,
-        cache_size=cache_size,
-        scaling=scaling,
-        scaling_options=scaling_options,
-        multistart=multistart,
-        multistart_options=multistart_options,
-    )
+    if not skip_checks:
+        check_optimize_kwargs(
+            direction=direction,
+            criterion=criterion,
+            criterion_kwargs=criterion_kwargs,
+            params=params,
+            algorithm=algorithm,
+            constraints=constraints,
+            algo_options=algo_options,
+            derivative=derivative,
+            derivative_kwargs=derivative_kwargs,
+            criterion_and_derivative=criterion_and_derivative,
+            criterion_and_derivative_kwargs=criterion_and_derivative_kwargs,
+            numdiff_options=numdiff_options,
+            logging=logging,
+            log_options=log_options,
+            error_handling=error_handling,
+            error_penalty=error_penalty,
+            scaling=scaling,
+            scaling_options=scaling_options,
+            multistart=multistart,
+            multistart_options=multistart_options,
+        )
     # ==================================================================================
     # Get the algorithm info
     # ==================================================================================
-    raw_algo, algo_info, algo_kwargs = process_user_algorithm(algorithm)
+    raw_algo, algo_info = process_user_algorithm(algorithm)
+
+    algo_kwargs = set(algo_info.arguments)
 
     if algo_info.primary_criterion_entry == "root_contributions":
         if direction == "maximize":
@@ -539,12 +546,16 @@ def _optimize(
         func=criterion,
         kwargs=criterion_kwargs,
         name="criterion",
+        skip_checks=skip_checks,
     )
     if isinstance(derivative, dict):
         derivative = derivative.get(algo_info.primary_criterion_entry)
     if derivative is not None:
         derivative = process_func_of_params(
-            func=derivative, kwargs=derivative_kwargs, name="derivative"
+            func=derivative,
+            kwargs=derivative_kwargs,
+            name="derivative",
+            skip_checks=skip_checks,
         )
     if isinstance(criterion_and_derivative, dict):
         criterion_and_derivative = criterion_and_derivative.get(
@@ -556,6 +567,7 @@ def _optimize(
             func=criterion_and_derivative,
             kwargs=criterion_and_derivative_kwargs,
             name="criterion_and_derivative",
+            skip_checks=skip_checks,
         )
 
     # ==================================================================================
@@ -613,11 +625,6 @@ def _optimize(
         soft_upper_bounds=soft_upper_bounds,
         add_soft_bounds=multistart,
     )
-
-    first_crit_eval_scalar = aggregate_func_output_to_value(
-        converter.func_to_internal(first_crit_eval),
-        algo_info.primary_criterion_entry,
-    )
     # ==================================================================================
     # initialize the log database
     # ==================================================================================
@@ -644,26 +651,23 @@ def _optimize(
             "constraints."
         )
 
-    first_eval = {
-        "internal_params": internal_params.values,
-        "external_params": params,
-        "output": first_crit_eval,
-    }
-
     numdiff_options = _fill_numdiff_options_with_defaults(
         numdiff_options=numdiff_options,
         lower_bounds=internal_params.lower_bounds,
         upper_bounds=internal_params.upper_bounds,
     )
 
-    # set default error penalty
-    error_penalty = _fill_error_penalty_with_defaults(
-        error_penalty, first_crit_eval_scalar, direction
+    # get error penalty function
+    error_penalty_func = get_error_penalty_function(
+        error_handling=error_handling,
+        start_x=internal_params.values,
+        start_criterion=converter.func_to_internal(first_crit_eval),
+        error_penalty=error_penalty,
+        primary_key=algo_info.primary_criterion_entry,
+        direction=direction,
     )
-    # create cache
+
     x = internal_params.values
-    x_hash = hash_array(x)
-    cache = {x_hash: {"criterion": converter.func_to_internal(first_eval["output"])}}
     # ==================================================================================
     # get the internal algorithm
     # ==================================================================================
@@ -676,6 +680,7 @@ def _optimize(
         algo_options=algo_options,
         logging=logging,
         db_kwargs=db_kwargs,
+        collect_history=collect_history,
     )
     # ==================================================================================
     # partial arguments into the internal_criterion_and_derivative_template
@@ -689,14 +694,10 @@ def _optimize(
         "numdiff_options": numdiff_options,
         "logging": logging,
         "db_kwargs": db_kwargs,
-        "first_criterion_evaluation": first_eval,
-        "cache": cache,
-        "cache_size": cache_size,
         "algo_info": algo_info,
+        "error_handling": error_handling,
+        "error_penalty_func": error_penalty_func,
     }
-    if not multistart:
-        to_partial["error_handling"] = error_handling
-        to_partial["error_penalty"] = error_penalty
 
     internal_criterion_and_derivative = functools.partial(
         internal_criterion_and_derivative_template,
@@ -745,7 +746,6 @@ def _optimize(
             logging=logging,
             db_kwargs=db_kwargs,
             error_handling=error_handling,
-            error_penalty=error_penalty,
         )
 
     # ==================================================================================
@@ -761,27 +761,6 @@ def _optimize(
     return res
 
 
-def _fill_error_penalty_with_defaults(error_penalty, first_value, direction):
-    """Add default options to error_penalty options."""
-    error_penalty = error_penalty.copy()
-
-    if direction == "minimize":
-        default_constant = (
-            first_value + np.abs(first_value) + CRITERION_PENALTY_CONSTANT
-        )
-        default_slope = CRITERION_PENALTY_SLOPE
-    else:
-        default_constant = (
-            first_value - np.abs(first_value) - CRITERION_PENALTY_CONSTANT
-        )
-        default_slope = -CRITERION_PENALTY_SLOPE
-
-    error_penalty["constant"] = error_penalty.get("constant", default_constant)
-    error_penalty["slope"] = error_penalty.get("slope", default_slope)
-
-    return error_penalty
-
-
 def _create_and_initialize_database(logging, log_options, problem_data):
     """Create and initialize to sqlite database for logging."""
     path = Path(logging)
diff --git a/src/estimagic/optimization/pounders.py b/src/estimagic/optimization/pounders.py
index 8da17a0b3..df0ec9738 100644
--- a/src/estimagic/optimization/pounders.py
+++ b/src/estimagic/optimization/pounders.py
@@ -38,8 +38,6 @@
     name="pounders",
     primary_criterion_entry="root_contributions",
     needs_scaling=True,
-    parallelizes=True,
-    disable_cache=False,
     is_available=True,
 )
 def pounders(
diff --git a/src/estimagic/optimization/pygmo_optimizers.py b/src/estimagic/optimization/pygmo_optimizers.py
index e6be95fb2..df3152258 100644
--- a/src/estimagic/optimization/pygmo_optimizers.py
+++ b/src/estimagic/optimization/pygmo_optimizers.py
@@ -22,9 +22,8 @@
 @mark_minimizer(
     name="pygmo_gaco",
     primary_criterion_entry="value",
-    parallelizes=True,
+    is_global=True,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_PYGMO_INSTALLED,
 )
 def pygmo_gaco(
@@ -105,9 +104,8 @@ def pygmo_gaco(
 @mark_minimizer(
     name="pygmo_bee_colony",
     primary_criterion_entry="value",
-    parallelizes=True,
+    is_global=True,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_PYGMO_INSTALLED,
 )
 def pygmo_bee_colony(
@@ -158,9 +156,8 @@ def pygmo_bee_colony(
 @mark_minimizer(
     name="pygmo_de",
     primary_criterion_entry="value",
-    parallelizes=True,
+    is_global=True,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_PYGMO_INSTALLED,
 )
 def pygmo_de(
@@ -238,9 +235,8 @@ def pygmo_de(
 @mark_minimizer(
     name="pygmo_sea",
     primary_criterion_entry="value",
-    parallelizes=True,
+    is_global=True,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_PYGMO_INSTALLED,
 )
 def pygmo_sea(
@@ -289,9 +285,8 @@ def pygmo_sea(
 @mark_minimizer(
     name="pygmo_sga",
     primary_criterion_entry="value",
-    parallelizes=True,
+    is_global=True,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_PYGMO_INSTALLED,
 )
 def pygmo_sga(
@@ -410,9 +405,8 @@ def pygmo_sga(
 @mark_minimizer(
     name="pygmo_sade",
     primary_criterion_entry="value",
-    parallelizes=True,
+    is_global=True,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_PYGMO_INSTALLED,
 )
 def pygmo_sade(
@@ -498,9 +492,8 @@ def pygmo_sade(
 @mark_minimizer(
     name="pygmo_cmaes",
     primary_criterion_entry="value",
-    parallelizes=True,
+    is_global=True,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_PYGMO_INSTALLED,
 )
 def pygmo_cmaes(
@@ -571,9 +564,8 @@ def pygmo_cmaes(
 @mark_minimizer(
     name="pygmo_simulated_annealing",
     primary_criterion_entry="value",
-    parallelizes=True,
+    is_global=True,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_PYGMO_INSTALLED,
 )
 def pygmo_simulated_annealing(
@@ -637,9 +629,8 @@ def pygmo_simulated_annealing(
 @mark_minimizer(
     name="pygmo_pso",
     primary_criterion_entry="value",
-    parallelizes=True,
+    is_global=True,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_PYGMO_INSTALLED,
 )
 def pygmo_pso(
@@ -733,9 +724,8 @@ def pygmo_pso(
 @mark_minimizer(
     name="pygmo_pso_gen",
     primary_criterion_entry="value",
-    parallelizes=True,
+    is_global=True,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_PYGMO_INSTALLED,
 )
 def pygmo_pso_gen(
@@ -828,9 +818,8 @@ def pygmo_pso_gen(
 @mark_minimizer(
     name="pygmo_mbh",
     primary_criterion_entry="value",
-    parallelizes=True,
+    is_global=True,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_PYGMO_INSTALLED,
 )
 def pygmo_mbh(
@@ -889,9 +878,8 @@ def pygmo_mbh(
 @mark_minimizer(
     name="pygmo_xnes",
     primary_criterion_entry="value",
-    parallelizes=True,
+    is_global=True,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_PYGMO_INSTALLED,
 )
 def pygmo_xnes(
@@ -960,9 +948,8 @@ def pygmo_xnes(
 @mark_minimizer(
     name="pygmo_gwo",
     primary_criterion_entry="value",
-    parallelizes=True,
+    is_global=True,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_PYGMO_INSTALLED,
 )
 def pygmo_gwo(
@@ -1011,9 +998,8 @@ def pygmo_gwo(
 @mark_minimizer(
     name="pygmo_compass_search",
     primary_criterion_entry="value",
-    parallelizes=True,
+    is_global=True,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_PYGMO_INSTALLED,
 )
 def pygmo_compass_search(
@@ -1077,9 +1063,8 @@ def pygmo_compass_search(
 @mark_minimizer(
     name="pygmo_ihs",
     primary_criterion_entry="value",
-    parallelizes=True,
+    is_global=True,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_PYGMO_INSTALLED,
 )
 def pygmo_ihs(
@@ -1146,9 +1131,8 @@ def pygmo_ihs(
 @mark_minimizer(
     name="pygmo_de1220",
     primary_criterion_entry="value",
-    parallelizes=True,
+    is_global=True,
     needs_scaling=True,
-    disable_cache=False,
     is_available=IS_PYGMO_INSTALLED,
 )
 def pygmo_de1220(
diff --git a/src/estimagic/optimization/tao_optimizers.py b/src/estimagic/optimization/tao_optimizers.py
index 5ababb4fe..3e2e257bf 100644
--- a/src/estimagic/optimization/tao_optimizers.py
+++ b/src/estimagic/optimization/tao_optimizers.py
@@ -47,12 +47,11 @@ def tao_pounders(
             "Windows. Windows users can use estimagics 'pounders' algorithm instead."
         )
 
-    x = _initialise_petsc_array(x)
+    first_eval = criterion(x)
+    n_errors = len(first_eval)
+    _x = _initialise_petsc_array(x)
     # We need to know the number of contributions of the criterion value to allocate the
     # array.
-    n_errors = len(
-        criterion.keywords["first_criterion_evaluation"]["output"]["root_contributions"]
-    )
     residuals_out = _initialise_petsc_array(n_errors)
 
     # Create the solver object.
@@ -81,7 +80,7 @@ def func_tao(tao, x, resid_out):
     tao.setResidual(func_tao, residuals_out)
 
     if trustregion_initial_radius is None:
-        trustregion_initial_radius = calculate_trustregion_initial_radius(x)
+        trustregion_initial_radius = calculate_trustregion_initial_radius(_x)
     elif trustregion_initial_radius <= 0:
         raise ValueError("The initial trust region radius must be > 0.")
     tao.setInitialTrustRegionRadius(trustregion_initial_radius)
@@ -92,7 +91,7 @@ def func_tao(tao, x, resid_out):
     tao.setVariableBounds(lower_bounds, upper_bounds)
 
     # Put the starting values into the container and pass them to the optimizer.
-    tao.setInitial(x)
+    tao.setInitial(_x)
 
     # Obtain tolerances for the convergence criteria. Since we can not create
     # scaled_gradient_tolerance manually we manually set absolute_gradient_tolerance and
@@ -153,7 +152,7 @@ def func_tao(tao, x, resid_out):
     results = _process_pounders_results(residuals_out, tao)
 
     # Destroy petsc objects for memory reasons.
-    for obj in [tao, x, residuals_out, lower_bounds, upper_bounds]:
+    for obj in [tao, _x, residuals_out, lower_bounds, upper_bounds]:
         obj.destroy()
 
     return results
diff --git a/src/estimagic/optimization/tiktak.py b/src/estimagic/optimization/tiktak.py
index 82afb150e..63019e0b3 100644
--- a/src/estimagic/optimization/tiktak.py
+++ b/src/estimagic/optimization/tiktak.py
@@ -32,7 +32,6 @@ def run_multistart_optimization(
     logging,
     db_kwargs,
     error_handling,
-    error_penalty,
 ):
     steps = determine_steps(options["n_samples"], options["n_optimizations"])
 
@@ -131,7 +130,7 @@ def run_multistart_optimization(
     }
 
     problem_functions = {
-        name: partial(func, error_handling=error_handling, error_penalty=error_penalty)
+        name: partial(func, error_handling=error_handling)
         for name, func in problem_functions.items()
     }
 
@@ -307,8 +306,7 @@ def run_explorations(
     Args:
         func (callable): An already partialled version of
             ``internal_criterion_and_derivative_template`` where the following arguments
-            are still free: ``x``, ``task``, ``algo_info``, ``error_handling``,
-            ``error_penalty``, ``fixed_log_data``.
+            are still free: ``x``, ``task``, ``error_handling``, ``fixed_log_data``.
         primary_key: The primary criterion entry of the local optimizer. Needed to
             interpret the output of the internal criterion function.
         sample (numpy.ndarray): 2d numpy array where each row is a sampled internal
@@ -335,8 +333,8 @@ def run_explorations(
         parallelizes=True,
         needs_scaling=False,
         name="tiktak_explorer",
-        disable_cache=True,
         is_available=True,
+        arguments=[],
     )
 
     _func = partial(
@@ -344,7 +342,6 @@ def run_explorations(
         task="criterion",
         algo_info=algo_info,
         error_handling=error_handling,
-        error_penalty={"constant": np.nan, "slope": np.nan},
     )
 
     arguments = []
diff --git a/src/estimagic/parameters/conversion.py b/src/estimagic/parameters/conversion.py
index 239def3cc..a6150bf81 100644
--- a/src/estimagic/parameters/conversion.py
+++ b/src/estimagic/parameters/conversion.py
@@ -234,12 +234,16 @@ def _get_fast_path_converter(params, lower_bounds, upper_bounds, primary_key):
 
     if lower_bounds is None:
         lower_bounds = np.full(len(params), -np.inf)
+    else:
+        lower_bounds = lower_bounds.astype(float)
 
     if upper_bounds is None:
         upper_bounds = np.full(len(params), np.inf)
+    else:
+        upper_bounds = upper_bounds.astype(float)
 
     flat_params = FlatParams(
-        values=params,
+        values=params.astype(float),
         lower_bounds=lower_bounds,
         upper_bounds=upper_bounds,
         free_mask=np.full(len(params), True),
@@ -288,6 +292,10 @@ def _is_fast_func_eval(f, key):
 
 
 def _is_fast_deriv_eval(d, key):
+    # this is the case if no or closed form derivatives are used
+    if d is None:
+        return True
+
     if key == "value":
         if not (_is_1d_arr(d) or (_is_dict_with(d, key) and _is_1d_arr(d[key]))):
             return False
diff --git a/src/estimagic/process_user_function.py b/src/estimagic/process_user_function.py
index 9a7e8798e..87c375021 100644
--- a/src/estimagic/process_user_function.py
+++ b/src/estimagic/process_user_function.py
@@ -7,7 +7,11 @@
 from estimagic.utilities import propose_alternatives
 
 
-def process_func_of_params(func, kwargs, name="your function"):
+def process_func_of_params(func, kwargs, name="your function", skip_checks=False):
+    # fast path
+    if skip_checks and kwargs in (None, {}):
+        return func
+
     kept, ignored = filter_kwargs(func, kwargs)
 
     if ignored:
@@ -25,28 +29,31 @@ def process_func_of_params(func, kwargs, name="your function"):
 
     out = partial(func, **kept)
 
-    unpartialled_args = get_unpartialled_arguments(out)
-    no_default_args = get_arguments_without_default(out)
+    if not skip_checks:
 
-    no_free_argument_left = len(unpartialled_args) < 1
+        unpartialled_args = get_unpartialled_arguments(out)
+        no_default_args = get_arguments_without_default(out)
 
-    if no_free_argument_left and kept:
-        raise InvalidKwargsError(
-            f"Too many keyword arguments for {name}. After applying all keyword "
-            "arguments there must be at least one free argument (the params) left."
-        )
-    elif no_free_argument_left:
-        raise InvalidFunctionError(f"{name} must have at least one free argument.")
+        no_free_argument_left = len(unpartialled_args) < 1
 
-    required_args = unpartialled_args.intersection(no_default_args)
-    too_many_required_arguments = len(required_args) > 1
+        if no_free_argument_left and kept:
+            raise InvalidKwargsError(
+                f"Too many keyword arguments for {name}. After applying all keyword "
+                "arguments there must be at least one free argument (the params) left."
+            )
+        elif no_free_argument_left:
+            raise InvalidFunctionError(f"{name} must have at least one free argument.")
 
-    if too_many_required_arguments:
-        raise InvalidKwargsError(
-            f"Too few keyword arguments for {name}. After applying all keyword "
-            "arguments at most one required argument (the params) should remain. "
-            f"in your case the following required arguments remain: {required_args}."
-        )
+        required_args = unpartialled_args.intersection(no_default_args)
+        too_many_required_arguments = len(required_args) > 1
+
+        if too_many_required_arguments:
+            raise InvalidKwargsError(
+                f"Too few keyword arguments for {name}. After applying all keyword "
+                "arguments at most one required argument (the params) should remain. "
+                "in your case the following required arguments remain: "
+                f"{required_args}."
+            )
 
     return out
 


Test patch:
diff --git a/tests/optimization/test_error_penalty.py b/tests/optimization/test_error_penalty.py
new file mode 100644
index 000000000..6f962b78f
--- /dev/null
+++ b/tests/optimization/test_error_penalty.py
@@ -0,0 +1,101 @@
+import functools
+
+import numpy as np
+import pytest
+from estimagic.differentiation.derivatives import first_derivative
+from estimagic.optimization.error_penalty import _penalty_contributions
+from estimagic.optimization.error_penalty import _penalty_contributions_derivative
+from estimagic.optimization.error_penalty import (
+    _penalty_root_contributions,
+)
+from estimagic.optimization.error_penalty import _penalty_root_contributions_derivative
+from estimagic.optimization.error_penalty import _penalty_value
+from estimagic.optimization.error_penalty import _penalty_value_derivative
+from estimagic.optimization.error_penalty import get_error_penalty_function
+from numpy.testing import assert_array_almost_equal as aaae
+
+
+@pytest.mark.parametrize("seed", range(10))
+def test_penalty_aggregations(seed):
+    np.random.seed(seed)
+    x = np.random.uniform(size=5)
+    x0 = np.random.uniform(size=5)
+    slope = 0.3
+    constant = 3
+    dim_out = 10
+
+    scalar = _penalty_value(x, constant, slope, x0)
+    contribs = _penalty_contributions(x, constant, slope, x0, dim_out)
+    root_contribs = _penalty_root_contributions(x, constant, slope, x0, dim_out)
+
+    assert np.isclose(scalar, contribs.sum())
+    assert np.isclose(scalar, (root_contribs**2).sum())
+
+
+pairs = [
+    (_penalty_value, _penalty_value_derivative),
+    (_penalty_contributions, _penalty_contributions_derivative),
+    (_penalty_root_contributions, _penalty_root_contributions_derivative),
+]
+
+
+@pytest.mark.parametrize("func, deriv", pairs)
+def test_penalty_derivatives(func, deriv):
+    np.random.seed(1234)
+    x = np.random.uniform(size=5)
+    x0 = np.random.uniform(size=5)
+    slope = 0.3
+    constant = 3
+    dim_out = 8
+
+    calculated = deriv(x, constant, slope, x0, dim_out)
+
+    partialed = functools.partial(
+        func, constant=constant, slope=slope, x0=x0, dim_out=dim_out
+    )
+    expected = first_derivative(partialed, x)
+
+    aaae(calculated, expected["derivative"])
+
+
+@pytest.mark.parametrize("seed", range(10))
+def test_penalty_aggregations_via_get_error_penalty(seed):
+    np.random.seed(seed)
+    x = np.random.uniform(size=5)
+    x0 = np.random.uniform(size=5)
+    slope = 0.3
+    constant = 3
+
+    scalar_func = get_error_penalty_function(
+        error_handling="continue",
+        start_x=x0,
+        start_criterion=3,
+        error_penalty={"slope": slope, "constant": constant},
+        primary_key="value",
+        direction="minimize",
+    )
+
+    contribs_func = get_error_penalty_function(
+        error_handling="continue",
+        start_x=x0,
+        start_criterion=np.ones(10),
+        error_penalty={"slope": slope, "constant": constant},
+        primary_key="contributions",
+        direction="minimize",
+    )
+
+    root_contribs_func = get_error_penalty_function(
+        error_handling="continue",
+        start_x=x0,
+        start_criterion=np.ones(10),
+        error_penalty={"slope": slope, "constant": constant},
+        primary_key="root_contributions",
+        direction="minimize",
+    )
+
+    scalar = scalar_func(x, task="criterion")
+    contribs = contribs_func(x, task="criterion")
+    root_contribs = root_contribs_func(x, task="criterion")
+
+    assert np.isclose(scalar, contribs.sum())
+    assert np.isclose(scalar, (root_contribs**2).sum())
diff --git a/tests/optimization/test_internal_criterion_and_derivative_template.py b/tests/optimization/test_internal_criterion_and_derivative_template.py
index cfbdd1dd3..fccfe67be 100644
--- a/tests/optimization/test_internal_criterion_and_derivative_template.py
+++ b/tests/optimization/test_internal_criterion_and_derivative_template.py
@@ -1,54 +1,22 @@
-import functools
 import itertools
 
 import numpy as np
 import pandas as pd
 import pytest
 from estimagic.decorators import AlgoInfo
-from estimagic.differentiation.derivatives import first_derivative
 from estimagic.examples.criterion_functions import sos_criterion_and_gradient
 from estimagic.examples.criterion_functions import sos_dict_criterion
 from estimagic.examples.criterion_functions import sos_dict_criterion_with_pd_objects
 from estimagic.examples.criterion_functions import sos_gradient
 from estimagic.examples.criterion_functions import sos_pandas_gradient
 from estimagic.examples.criterion_functions import sos_scalar_criterion
-from estimagic.optimization.internal_criterion_template import _penalty_contributions
-from estimagic.optimization.internal_criterion_template import (
-    _penalty_contributions_derivative,
-)
-from estimagic.optimization.internal_criterion_template import (
-    _penalty_root_contributions,
-)
-from estimagic.optimization.internal_criterion_template import (
-    _penalty_root_contributions_derivative,
-)
-from estimagic.optimization.internal_criterion_template import _penalty_value
-from estimagic.optimization.internal_criterion_template import _penalty_value_derivative
 from estimagic.optimization.internal_criterion_template import (
     internal_criterion_and_derivative_template,
 )
-from estimagic.optimization.optimize import _fill_error_penalty_with_defaults
 from estimagic.parameters.conversion import get_converter
 from numpy.testing import assert_array_almost_equal as aaae
 
 
-def no_second_call(func):
-    counter = np.array([0])
-
-    if func is None:
-        wrapper_func = None
-    else:
-
-        def wrapper_func(params, counter=counter, *args, **kwargs):
-            res = func(params, *args, **kwargs)
-            if counter[0] >= 1:
-                raise AssertionError("This was called twice.")
-            counter += 1
-            return res
-
-    return wrapper_func
-
-
 def reparametrize_from_internal(x):
     res = pd.DataFrame()
     res["value"] = x
@@ -69,19 +37,16 @@ def base_inputs():
         "algo_info": AlgoInfo(
             name="my_algorithm",
             primary_criterion_entry="value",
-            parallelizes=False,
             needs_scaling=False,
-            disable_cache=False,
             is_available=True,
+            parallelizes=False,
+            arguments=[],
         ),
+        "error_handling": "raise",
         "numdiff_options": {},
         "logging": False,
         "db_kwargs": {"database": False, "fast_logging": False, "path": "logging.db"},
-        "error_handling": "raise",
-        "error_penalty": None,
-        "first_criterion_evaluation": {"internal_params": x, "external_params": params},
-        "cache": {},
-        "cache_size": 10,
+        "error_penalty_func": None,
         "fixed_log_data": {"stage": "optimization", "substage": 0},
     }
     return inputs
@@ -114,12 +79,11 @@ def test_criterion_and_derivative_template(
     inputs = {k: v for k, v in base_inputs.items() if k != "params"}
     inputs["converter"] = converter
 
-    inputs["first_criterion_evaluation"]["output"] = crit(base_inputs["params"])
-    crit = crit if (deriv, crit_and_deriv) == (None, None) else no_second_call(crit)
+    crit = crit if (deriv, crit_and_deriv) == (None, None) else crit
 
     inputs["criterion"] = crit
-    inputs["derivative"] = no_second_call(deriv)
-    inputs["criterion_and_derivative"] = no_second_call(crit_and_deriv)
+    inputs["derivative"] = deriv
+    inputs["criterion_and_derivative"] = crit_and_deriv
     inputs["direction"] = direction
 
     calc_criterion, calc_derivative = internal_criterion_and_derivative_template(
@@ -165,100 +129,30 @@ def test_internal_criterion_with_penalty(base_inputs, direction):
     inputs = {k: v for k, v in base_inputs.items() if k != "params"}
 
     inputs["converter"] = converter
-    scaling = 1 if direction == "minimize" else -1
-    inputs["first_criterion_evaluation"]["output"] = scaling * 30
 
     def raising_crit_and_deriv(x):
         raise ValueError()
 
+    inputs["error_handling"] = "continue"
     inputs["x"] = inputs["x"] + 10
     inputs["criterion"] = sos_scalar_criterion
     inputs["derivative"] = sos_gradient
     inputs["criterion_and_derivative"] = raising_crit_and_deriv
     inputs["direction"] = direction
-    inputs["error_handling"] = "continue"
-    inputs["error_penalty"] = _fill_error_penalty_with_defaults(
-        error_penalty={},
-        first_value=scaling * 30,
-        direction=direction,
-    )
+    inputs["error_penalty_func"] = lambda x, task: (42, 52)
 
     with pytest.warns():
         calc_criterion, calc_derivative = internal_criterion_and_derivative_template(
             task="criterion_and_derivative", **inputs
         )
 
-    calc_criterion2 = internal_criterion_and_derivative_template(
-        task="criterion", **inputs
-    )
-
-    calc_derivative2 = internal_criterion_and_derivative_template(
-        task="derivative", **inputs
-    )
-
-    norm = np.linalg.norm(np.ones(5) * 10)
-    slope = 0.1 if direction == "minimize" else -0.1
-    constant = 160 if direction == "minimize" else -160
-
-    x = inputs["x"]
-    x0 = np.arange(5)
-
-    expected_crit = constant + slope * norm
-    expected_grad = slope * 10 / np.linalg.norm(x - x0)
+    expected_crit = 42
+    expected_grad = 52
 
     if direction == "minimize":
-        for c in calc_criterion, calc_criterion2:
-            assert c == expected_crit
-
-        for d in calc_derivative, calc_derivative2:
-            aaae(d, expected_grad)
+        assert calc_criterion == expected_crit
+        aaae(calc_derivative, expected_grad)
 
     else:
-        for c in calc_criterion, calc_criterion2:
-            assert c == -expected_crit
-
-        for d in calc_derivative, calc_derivative2:
-            aaae(d, -expected_grad)
-
-
-@pytest.mark.parametrize("seed", range(10))
-def test_penalty_aggregations(seed):
-    np.random.seed(seed)
-    x = np.random.uniform(size=5)
-    x0 = np.random.uniform(size=5)
-    slope = 0.3
-    constant = 3
-    dim_out = 10
-
-    scalar = _penalty_value(x, constant, slope, x0)
-    contribs = _penalty_contributions(x, constant, slope, x0, dim_out)
-    root_contribs = _penalty_root_contributions(x, constant, slope, x0, dim_out)
-
-    assert np.isclose(scalar, contribs.sum())
-    assert np.isclose(scalar, (root_contribs**2).sum())
-
-
-pairs = [
-    (_penalty_value, _penalty_value_derivative),
-    (_penalty_contributions, _penalty_contributions_derivative),
-    (_penalty_root_contributions, _penalty_root_contributions_derivative),
-]
-
-
-@pytest.mark.parametrize("func, deriv", pairs)
-def test_penalty_derivatives(func, deriv):
-    np.random.seed(1234)
-    x = np.random.uniform(size=5)
-    x0 = np.random.uniform(size=5)
-    slope = 0.3
-    constant = 3
-    dim_out = 8
-
-    calculated = deriv(x, constant, slope, x0, dim_out)
-
-    partialed = functools.partial(
-        func, constant=constant, slope=slope, x0=x0, dim_out=dim_out
-    )
-    expected = first_derivative(partialed, x)
-
-    aaae(calculated, expected["derivative"])
+        assert calc_criterion == -expected_crit
+        aaae(calc_derivative, -expected_grad)
diff --git a/tests/optimization/test_many_algorithms.py b/tests/optimization/test_many_algorithms.py
index 7124f9143..cf4eead35 100644
--- a/tests/optimization/test_many_algorithms.py
+++ b/tests/optimization/test_many_algorithms.py
@@ -46,6 +46,8 @@ def test_algorithm_on_sum_of_squares(algorithm):
         criterion=sos,
         params=np.arange(3),
         algorithm=algorithm,
+        collect_history=False,
+        skip_checks=True,
     )
 
     assert res["success"] in [True, None]
@@ -65,6 +67,8 @@ def test_algorithm_on_sum_of_squares_with_binding_bounds(algorithm):
         lower_bounds=np.array([1, -np.inf, -np.inf]),
         upper_bounds=np.array([np.inf, np.inf, -1]),
         algorithm=algorithm,
+        collect_history=False,
+        skip_checks=True,
     )
 
     assert res["success"] in [True, None]
@@ -90,6 +94,8 @@ def test_global_algorithms_on_sum_of_squares(algorithm):
         lower_bounds=np.array([0.2, -0.5]),
         upper_bounds=np.array([1, 0.5]),
         algorithm=algorithm,
+        collect_history=False,
+        skip_checks=True,
     )
     assert res["success"] in [True, None]
     aaae(res["solution_params"], np.array([0.2, 0]), decimal=1)
diff --git a/tests/optimization/test_tiktak.py b/tests/optimization/test_tiktak.py
index ad6ca13b9..d1bcad7af 100644
--- a/tests/optimization/test_tiktak.py
+++ b/tests/optimization/test_tiktak.py
@@ -62,7 +62,6 @@ def _dummy(x, **kwargs):
             "task",
             "algo_info",
             "error_handling",
-            "error_penalty",
             "fixed_log_data",
         }
         if x.sum() == 5:
diff --git a/tests/parameters/test_conversion.py b/tests/parameters/test_conversion.py
index 66a7e8f10..10ac9b987 100644
--- a/tests/parameters/test_conversion.py
+++ b/tests/parameters/test_conversion.py
@@ -187,6 +187,9 @@ def test_is_fast_func_eval_true(key, f):
     ("contributions", {"contributions": helper}),
     ("root_contributions", helper),
     ("root_contributions", {"root_contributions": helper}),
+    ("value", None),
+    ("contributions", None),
+    ("root_contributions", None),
 ]
 
 
@@ -213,9 +216,6 @@ def test_is_fast_func_eval_false(key, f):
     ("contributions", {"contributions": np.arange(8).reshape(2, 2, 2)}),
     ("root_contributions", np.arange(8).reshape(2, 2, 2)),
     ("root_contributions", {"root_contributions": np.arange(8).reshape(2, 2, 2)}),
-    ("value", None),
-    ("contributions", None),
-    ("root_contributions", None),
 ]
 
 
diff --git a/tests/test_decorators.py b/tests/test_decorators.py
index ac303f8cf..3b253d079 100644
--- a/tests/test_decorators.py
+++ b/tests/test_decorators.py
@@ -76,8 +76,6 @@ def minimize_stupid():
 INVALID_TYPES = [
     {"name": None},
     {"name": [1, 2, 3]},
-    {"parallelizes": 15},
-    {"disable_cache": 20},
     {"needs_scaling": 25},
     {"is_available": 30},
 ]