Repository: projectmesa/mesa (python)

Issue:
BatchRunner swaps content of variable_params columns, only in random proportion of executions
**Describe the bug**
When using BatchRunner, and calling get_model_vars_dataframe(), the contents of the columns in the output  of two variable parameters are swapped: the column heads stay the same, but matches with the parameter values of another parameter. I cannot tell yet whether it is just the columns of the DataFrame that is wrong, or also the batch run with the wrong values.

The remarkable thing is: when I execute my script several times, with the same code, it only happens in a number of executions. Maybe the behaviour of the bug is dependent on a random value?

**Expected behavior**
I expect the BatchRunner model vars dataframe to give back the variable parameter values that correspond with a certain parameter. Instead, the parameter values of one parameter are swapped with the parameter values of another parameter.

**To Reproduce**
I do not have a minimum working example. I have some code which depends on my specific model:

```
from mesa.batchrunner import BatchRunner
from constants import HEIGHT, WIDTH, PROPORTION_L2, SUFFIX_PROB, CAPACITY_L1, CAPACITY_L2, \
                        DROP_SUBJECT_PROB, DROP_OBJECT_PROB, MIN_BOUNDARY_FEATURE_DIST
from Model import Model

fixed_params = {
    "height": HEIGHT,
    "width": WIDTH,
    "proportion_l2": PROPORTION_L2,
    "suffix_prob": SUFFIX_PROB,
    "capacity_l1": CAPACITY_L1,
    "capacity_l2": CAPACITY_L2,
    "drop_subject_prob": DROP_SUBJECT_PROB,
    "drop_object_prob": DROP_OBJECT_PROB,
    "min_boundary_feature_dist": MIN_BOUNDARY_FEATURE_DIST
}

variable_params = {"proportion_l2":[0.0, 0.2, 0.5], "capacity_l1": [0.0,1.0,2.0,5.0]}

for k in variable_params:
    del fixed_params[k]

stats = {"global_filled_prefix_l1": lambda m: m.global_filled_prefix_l1,
         "global_filled_suffix_l1": lambda m: m.global_filled_suffix_l1,
         "global_filled_prefix_l2": lambda m: m.global_filled_prefix_l2,
         "global_filled_suffix_l2": lambda m: m.global_filled_suffix_l2}

batch_run = BatchRunner(
    Model,
    variable_params,
    fixed_params,
    iterations=1,
    max_steps=1,
    model_reporters=stats
)

batch_run.run_all()
cols = list(variable_params.keys()) + list(stats.keys())
run_data = batch_run.get_model_vars_dataframe()
run_data = run_data[cols]
print(run_data)
run_data.to_csv("evaluation.tsv", sep="\t")
```

It gives the following output:
```
12it [00:02,  5.13it/s]
    proportion_l2  capacity_l1  global_filled_prefix_l1  global_filled_suffix_l1  global_filled_prefix_l2  global_filled_suffix_l2
0             0.0          0.0                      0.0                      0.0                      0.0                      0.0
1             1.0          0.0                      0.0                      0.0                      0.0                      0.0
2             2.0          0.0                      0.0                      0.0                      0.0                      0.0
3             5.0          0.0                      0.0                      0.0                      0.0                      0.0
4             0.0          0.2                      0.0                      0.0                      0.0                      0.0
5             1.0          0.2                      0.0                      0.0                      0.0                      0.0
6             2.0          0.2                      0.0                      0.0                      0.0                      0.0
7             5.0          0.2                      0.0                      0.0                      0.0                      0.0
8             0.0          0.5                      0.0                      0.0                      0.0                      0.0
9             1.0          0.5                      0.0                      0.0                      0.0                      0.0
10            2.0          0.5                      0.0                      0.0                      0.0                      0.0
11            5.0          0.5                      0.0                      0.0                      0.0                      0.0
```

In the output, the values of ther parameters `capacity_l1` and `proportion_l2` have been swapped, if you compare them how these have been initialized.

**Additional context**
The bug occurs on two different Ubuntu machines, with mesa 0.8.7 or 0.8.8. The bug doesn't occur in mesa 0.8.6.


Pull request:
Fix order of variable_params in model and agent vars data frames
Demonstrates and ~(hopefully soon)~ fixes #978.

As mentioned in https://github.com/projectmesa/mesa/issues/978#issuecomment-765809899, there's an issue where `set()` is used, which results in a non-deterministic order of the columns of data frames created by `FixedBatchRunner._prepare_report_table`. Not using `set` should be the solution, here.

Patch:
diff --git a/mesa/batchrunner.py b/mesa/batchrunner.py
index 0c43c584..4178cb23 100644
--- a/mesa/batchrunner.py
+++ b/mesa/batchrunner.py
@@ -100,6 +100,11 @@ class FixedBatchRunner:
         self.iterations = iterations
         self.max_steps = max_steps
 
+        for params in self.parameters_list:
+            if list(params) != list(self.parameters_list[0]):
+                msg = "parameter names in parameters_list are not equal across the list"
+                raise ValueError(msg)
+
         self.model_reporters = model_reporters
         self.agent_reporters = agent_reporters
 
@@ -257,10 +262,10 @@ class FixedBatchRunner:
         column as a key.
         """
         extra_cols = ["Run"] + (extra_cols or [])
-        index_cols = set()
-        for params in self.parameters_list:
-            index_cols |= params.keys()
-        index_cols = list(index_cols) + extra_cols
+        index_cols = []
+        if self.parameters_list:
+            index_cols = list(self.parameters_list[0].keys())
+        index_cols += extra_cols
 
         records = []
         for param_key, values in vars_dict.items():


Test patch:
diff --git a/tests/test_batchrunner.py b/tests/test_batchrunner.py
index 86714911..564b4e36 100644
--- a/tests/test_batchrunner.py
+++ b/tests/test_batchrunner.py
@@ -8,7 +8,7 @@ import unittest
 from mesa import Agent, Model
 from mesa.time import BaseScheduler
 from mesa.datacollection import DataCollector
-from mesa.batchrunner import BatchRunner, ParameterProduct, ParameterSampler
+from mesa.batchrunner import BatchRunner, FixedBatchRunner, ParameterProduct, ParameterSampler
 
 
 NUM_AGENTS = 7
@@ -120,7 +120,7 @@ class TestBatchRunner(unittest.TestCase):
 
     def launch_batch_processing_fixed(self):
         # Adding second batchrun to test fixed params increase coverage
-        batch2 = BatchRunner(
+        batch = BatchRunner(
             self.mock_model,
             fixed_parameters={"fixed": "happy"},
             iterations=4,
@@ -129,15 +129,31 @@ class TestBatchRunner(unittest.TestCase):
             agent_reporters=None,
         )
 
-        batch2.run_all()
-        return batch2
+        batch.run_all()
+        return batch
+
+    def launch_batch_processing_fixed_list(self):
+        batch = FixedBatchRunner(
+            self.mock_model,
+            parameters_list=self.variable_params,
+            fixed_parameters=self.fixed_params,
+            iterations=self.iterations,
+            max_steps=self.max_steps,
+            model_reporters=self.model_reporters,
+            agent_reporters=self.agent_reporters,
+        )
+        batch.run_all()
+        return batch
 
     @property
     def model_runs(self):
         """
         Returns total number of batch runner's iterations.
         """
-        return reduce(mul, map(len, self.variable_params.values())) * self.iterations
+        if isinstance(self.variable_params, list):
+            return len(self.variable_params) * self.iterations
+        else:
+            return reduce(mul, map(len, self.variable_params.values())) * self.iterations
 
     def test_model_level_vars(self):
         """
@@ -151,6 +167,11 @@ class TestBatchRunner(unittest.TestCase):
         )  # extra column with run index
         self.assertEqual(model_vars.shape, (self.model_runs, expected_cols))
         self.assertEqual(len(model_collector.keys()), self.model_runs)
+        for var, values in self.variable_params.items():
+            self.assertEqual(set(model_vars[var].unique()), set(values))
+        if self.fixed_params:
+            for var, values in self.fixed_params.items():
+                self.assertEqual(set(model_vars[var].unique()), set(values))
 
     def test_agent_level_vars(self):
         """
@@ -169,6 +190,8 @@ class TestBatchRunner(unittest.TestCase):
         assert "agent_id" in list(agent_collector[(0, 1, 1)].columns)
         assert "Step" in list(agent_collector[(0, 1, 5)].index.names)
         assert "nose" not in list(agent_collector[(0, 1, 1)].columns)
+        for var, values in self.variable_params.items():
+            self.assertEqual(set(agent_vars[var].unique()), set(values))
 
         self.assertEqual(
             agent_collector[(0, 1, 0)].shape, (NUM_AGENTS * self.max_steps, 2)
@@ -224,6 +247,39 @@ class TestBatchRunner(unittest.TestCase):
             model_vars["reported_fixed_param"].iloc[0], self.fixed_params["fixed_name"]
         )
 
+    def test_model_with_variable_kwargs_list(self):
+        self.variable_params = [
+            {"variable_model_param": 1, "variable_agent_param": 1},
+            {"variable_model_param": 2, "variable_agent_param": 1},
+            {"variable_model_param": 2, "variable_agent_param": 8},
+            {"variable_model_param": 3, "variable_agent_param": 8},
+        ]
+        n_params = len(self.variable_params[0])
+        batch = self.launch_batch_processing_fixed_list()
+
+        model_vars = batch.get_model_vars_dataframe()
+        expected_cols = n_params + len(self.model_reporters) + 1
+        self.assertEqual(model_vars.shape, (self.model_runs, expected_cols))
+
+        agent_vars = batch.get_agent_vars_dataframe()
+        expected_cols = n_params + len(self.agent_reporters) + 2
+        self.assertEqual(
+            agent_vars.shape, (self.model_runs * NUM_AGENTS, expected_cols)
+        )
+
+    def test_model_with_variable_kwargs_list_mixed_length(self):
+        self.variable_params = [
+            {"variable_model_param": 1},
+            {"variable_model_param": 2},
+            {"variable_model_param": 2, "variable_agent_param": 8},
+            {"variable_model_param": 3, "variable_agent_param": 8},
+            {"variable_agent_param": 1},
+        ]
+        # This is currently not supported. Check that it raises the correct error.
+        msg = "parameter names in parameters_list are not equal across the list"
+        with self.assertRaises(ValueError, msg=msg):
+            self.launch_batch_processing_fixed_list()
+
 
 class TestParameters(unittest.TestCase):
     def test_product(self):