Repository: projectmesa/mesa

Issue:
Batchrunner does not handle exotic input parameters?
My model class takes inputs such as below:

class ExampleModel(Model):
    def __init__(self,
                 seed: int = 0,
                 input_data: Type[pd.DataFrame] = pd.read_csv('InputData.csv'),
                 scenario: str = 'baseline',
                 social_network_parameters: dict = {
                     "average_degree": 10, "rewiring_probability": 0.1},
                 year_period: int = 30,
                 etc. ) -> None:

It would be great if we could have a way to iterate through a list of strings for instance. In my case, the scenario parameter will select part of the Data Frame input based on the scenario string that is given. Ideally, I would like to give batch_run a list such as: ['baseline', 'scenario1', 'scenario2'].

It seems that because the batch_run function wants to launch batches based on iterables it takes the name of my first Data Frame column as an input for the input_data parameter or the first letter of my string (i.e., 'b' in 'baseline' for the example above) for the scenario parameter.

Although it was already not possible to do too many exotic things with input parameters to iterate through in the old BatchRunner classes, at least we could have strings, dictionaries, and so on as input parameters because of the "fixed_parameters" input of the BatchRunner classes. Now it seems batch_run will base its batch of simulations based on any iterable. I am not sure how that could be fixed but I think it could improve Mesa's functionalities. 

PS: I do love the simplicity of the new batch_run function, so I am not advocating going back to the old BatchRunner classes but just improving batch_run.

Discussion:
codecov[bot]: # [Codecov](https://codecov.io/gh/projectmesa/mesa/pull/1413?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=projectmesa) Report
Base: **91.03**% // Head: **90.99**% // Decreases project coverage by **`-0.03%`** :warning:
> Coverage data is based on head [(`fd99bab`)](https://codecov.io/gh/projectmesa/mesa/pull/1413?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=projectmesa) compared to base [(`5c68906`)](https://codecov.io/gh/projectmesa/mesa/commit/5c68906b7f7f1d96c2467695b3ee0eb18d9c01ed?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=projectmesa).
> Patch coverage: 94.11% of modified lines in pull request are covered.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main    #1413      +/-   ##
==========================================
- Coverage   91.03%   90.99%   -0.04%     
==========================================
  Files          18       18              
  Lines        1361     1355       -6     
  Branches      262      263       +1     
==========================================
- Hits         1239     1233       -6     
  Misses         86       86              
  Partials       36       36              
```


| [Impacted Files](https://codecov.io/gh/projectmesa/mesa/pull/1413?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=projectmesa) | Coverage Δ | |
|---|---|---|
| [mesa/batchrunner.py](https://codecov.io/gh/projectmesa/mesa/pull/1413/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=projectmesa#diff-bWVzYS9iYXRjaHJ1bm5lci5weQ==) | `92.95% <94.11%> (-0.19%)` | :arrow_down: |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=projectmesa). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=projectmesa)

</details>

[:umbrella: View full report at Codecov](https://codecov.io/gh/projectmesa/mesa/pull/1413?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=projectmesa).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=projectmesa).

rht: The implementation logic is simpler too.

> (which additionally makes it deterministic which is nice)

Though, I think the current implementation is already deterministic.

My only concern before merging this is that since each param is repeated `iterations` time first, before going to the next param. The ordering is different, and hence the random numbers are used differently. This may cause a problem with people who seed their simulation, only to find their simulation giving a different result mysteriously when they update Mesa.

Maybe we can get away with issuing a warning about this in the release note.
joedight: 
> My only concern before merging this is that since each param is repeated `iterations` time first, before going to the next param. The ordering is different, and hence the random numbers are used differently. This may cause a problem with people who seed their simulation, only to find their simulation giving a different result mysteriously when they update Mesa.

I've changed that now, the behaviour should match what it was before.

> Though, I think the current implementation is already deterministic.

Sorry, I meant determinism in assigning iteration and run numbers given imap_unordered can return out-of-order (I think).


Patch:
diff --git a/.codespellignore b/.codespellignore
index 4faefa4c..94a0147d 100644
--- a/.codespellignore
+++ b/.codespellignore
@@ -5,3 +5,4 @@ ist
 inactivate
 ue
 fpr
+falsy
diff --git a/mesa/batchrunner.py b/mesa/batchrunner.py
index a66b13d8..76d6a317 100644
--- a/mesa/batchrunner.py
+++ b/mesa/batchrunner.py
@@ -15,7 +15,6 @@ from multiprocessing import Pool, cpu_count
 from warnings import warn
 from typing import (
     Any,
-    Counter,
     Dict,
     Iterable,
     List,
@@ -67,7 +66,13 @@ def batch_run(
         [description]
     """
 
-    kwargs_list = _make_model_kwargs(parameters) * iterations
+    runs_list = []
+    run_id = 0
+    for iteration in range(iterations):
+        for kwargs in _make_model_kwargs(parameters):
+            runs_list.append((run_id, iteration, kwargs))
+            run_id += 1
+
     process_func = partial(
         _model_run_func,
         model_cls,
@@ -75,34 +80,19 @@ def batch_run(
         data_collection_period=data_collection_period,
     )
 
-    total_iterations = len(kwargs_list)
-    run_counter = count()
-
     results: List[Dict[str, Any]] = []
 
-    with tqdm(total=total_iterations, disable=not display_progress) as pbar:
-        iteration_counter: Counter[Tuple[Any, ...]] = Counter()
-
-        def _fn(paramValues, rawdata):
-            iteration_counter[paramValues] += 1
-            iteration = iteration_counter[paramValues]
-            run_id = next(run_counter)
-            data = []
-            for run_data in rawdata:
-                out = {"RunId": run_id, "iteration": iteration - 1}
-                out.update(run_data)
-                data.append(out)
-            results.extend(data)
-            pbar.update()
-
+    with tqdm(total=len(runs_list), disable=not display_progress) as pbar:
         if number_processes == 1:
-            for kwargs in kwargs_list:
-                paramValues, rawdata = process_func(kwargs)
-                _fn(paramValues, rawdata)
+            for run in runs_list:
+                data = process_func(run)
+                results.extend(data)
+                pbar.update()
         else:
             with Pool(number_processes) as p:
-                for paramValues, rawdata in p.imap_unordered(process_func, kwargs_list):
-                    _fn(paramValues, rawdata)
+                for data in p.imap_unordered(process_func, runs_list):
+                    results.extend(data)
+                    pbar.update()
 
     return results
 
@@ -140,18 +130,18 @@ def _make_model_kwargs(
 
 def _model_run_func(
     model_cls: Type[Model],
-    kwargs: Dict[str, Any],
+    run: Tuple[int, int, Dict[str, Any]],
     max_steps: int,
     data_collection_period: int,
-) -> Tuple[Tuple[Any, ...], List[Dict[str, Any]]]:
+) -> List[Dict[str, Any]]:
     """Run a single model run and collect model and agent data.
 
     Parameters
     ----------
     model_cls : Type[Model]
         The model class to batch-run
-    kwargs : Dict[str, Any]
-        model kwargs used for this run
+    run: Tuple[int, int, Dict[str, Any]]
+        The run id, iteration number, and kwargs for this run
     max_steps : int
         Maximum number of model steps after which the model halts, by default 1000
     data_collection_period : int
@@ -159,9 +149,10 @@ def _model_run_func(
 
     Returns
     -------
-    Tuple[Tuple[Any, ...], List[Dict[str, Any]]]
+    List[Dict[str, Any]]
         Return model_data, agent_data from the reporters
     """
+    run_id, iteration, kwargs = run
     model = model_cls(**kwargs)
     while model.running and model.schedule.steps <= max_steps:
         model.step()
@@ -178,15 +169,30 @@ def _model_run_func(
         # If there are agent_reporters, then create an entry for each agent
         if all_agents_data:
             stepdata = [
-                {**{"Step": step}, **kwargs, **model_data, **agent_data}
+                {
+                    "RunId": run_id,
+                    "iteration": iteration,
+                    "Step": step,
+                    **kwargs,
+                    **model_data,
+                    **agent_data,
+                }
                 for agent_data in all_agents_data
             ]
         # If there is only model data, then create a single entry for the step
         else:
-            stepdata = [{**{"Step": step}, **kwargs, **model_data}]
+            stepdata = [
+                {
+                    "RunId": run_id,
+                    "iteration": iteration,
+                    "Step": step,
+                    **kwargs,
+                    **model_data,
+                }
+            ]
         data.extend(stepdata)
 
-    return tuple(kwargs.values()), data
+    return data
 
 
 def _collect_data(


Test patch:
diff --git a/tests/test_batch_run.py b/tests/test_batch_run.py
index 4f387173..fffd422d 100644
--- a/tests/test_batch_run.py
+++ b/tests/test_batch_run.py
@@ -50,6 +50,7 @@ class MockModel(Model):
         fixed_model_param=None,
         schedule=None,
         enable_agent_reporters=True,
+        n_agents=3,
         **kwargs
     ):
         super().__init__()
@@ -57,7 +58,7 @@ class MockModel(Model):
         self.variable_model_param = variable_model_param
         self.variable_agent_param = variable_agent_param
         self.fixed_model_param = fixed_model_param
-        self.n_agents = 3
+        self.n_agents = n_agents
         if enable_agent_reporters:
             agent_reporters = {"agent_id": "unique_id", "agent_local": "local"}
         else:
@@ -145,3 +146,52 @@ def test_batch_run_no_agent_reporters():
 
 def test_batch_run_single_core():
     batch_run(MockModel, {}, number_processes=1, iterations=10)
+
+
+def test_batch_run_unhashable_param():
+    result = batch_run(
+        MockModel,
+        {
+            "n_agents": 2,
+            "variable_model_params": [{"key": "value"}],
+        },
+        iterations=2,
+    )
+    template = {
+        "Step": 1000,
+        "reported_model_param": 42,
+        "agent_local": 250.0,
+        "n_agents": 2,
+        "variable_model_params": {"key": "value"},
+    }
+
+    assert result == [
+        {
+            "RunId": 0,
+            "iteration": 0,
+            "AgentID": 0,
+            "agent_id": 0,
+            **template,
+        },
+        {
+            "RunId": 0,
+            "iteration": 0,
+            "AgentID": 1,
+            "agent_id": 1,
+            **template,
+        },
+        {
+            "RunId": 1,
+            "iteration": 1,
+            "AgentID": 0,
+            "agent_id": 0,
+            **template,
+        },
+        {
+            "RunId": 1,
+            "iteration": 1,
+            "AgentID": 1,
+            "agent_id": 1,
+            **template,
+        },
+    ]