Repository: projectmesa/mesa

Issue:
Enhance DataCollector to Validate model_reporters Functions
**Describe the bug**
Currently, the DataCollector's model_reporters validation has limitations when handling string attributes. The issue arises because:

1. Model reporters can accept four types of inputs:
   ```python
   model_reporters = {
       "Count": lambda m: len(m.agents),              # Lambda function
       "Status": "market_status",                     # Attribute name (string)
       "Metric": model.calculate_metric,              # Method reference
       "Custom": [calculate_wealth, [param1, param2]] # Function with parameters
   }
   ```

2. When using string attributes (e.g., "market_status"), there's no validation to check if these attributes actually exist in the model. This leads to:
   - Silent failures when collecting undefined attributes
   - Confusion when attributes are misspelled or not initialized
   - Delayed error detection until runtime



Adding a validation mechanism to ensure model_reporters keys mapped to callable functions or valid strings would greatly improve error detection and provide better debugging experiences for users.

**Expected behavior**
When using DataCollector, if an invalid value is provided in model_reporters (e.g., a non-callable object or undefined function):

Raise a ValueError with a clear error message, such as:
"Invalid reporter for 'Gini': must be a callable function/method or a valid attribute name as a string."
This prompt will help users:

1. Understand their mistake.
2. Quickly identify whether the provided function (e.g., compute_gini) is missing, incorrectly implemented, or improperly referenced.

**To Reproduce**

Valid Example:
```python
`from mesa.datacollection import DataCollector  

def compute_gini(model):     
	return 0.5  

collector = DataCollector(     
	model_reporters={"Gini": compute_gini},     
	agent_reporters={"Wealth": "wealth"} 
)`
```

Invalid Example (Missing Function):

```python
`from mesa.datacollection import DataCollector  

collector = DataCollector(     
	model_reporters={"Gini": "compute_gini"},  # Error: compute_gini is not callable     
	agent_reporters={"Wealth": "wealth"} 
)`
```
**Expected Output:**

```
`ValueError: Invalid reporter for 'Gini': must be a callable function/method or a valid attribut`
```

Proposed Solution
Modify the DataCollector class to include validation for model_reporters, similar to the following:

```python
def _validate_model_reporter(self, name, reporter): 
	# Type 1: Lambda function
	if callable(reporter): 
		return 
	# Type 2: Class attributes (string)
	elif isinstance(reporter, str): 
		return 
	# Type xxx: other types mechanism to warn
	else: 
		raise ValueError(
		 f"Invalid reporter for '{name}': must be a callable function/method or a valid attribute name as a string." 
		)
```


**Additional context**

- Affected Class: DataCollector
- Documentation: [Introductory Tutorial](https://mesa.readthedocs.io/stable/tutorials/intro_tutorial.html#collecting-data)
- Location in tutorial: the section of "Collecting Data"
- api documentation: [Data collection](https://mesa.readthedocs.io/stable/apis/datacollection.html#)

Discussion:
@peter-kinger: I have created a PR to address this issue. Please refer to: https://github.com/projectmesa/mesa/pull/2605@quaquel: I like the idea of this issue and the PR. I hope to find time to look at it more closely soon. Its a part of the code base I am not particularly familiar with, however. @peter-kinger: > I like the idea of this issue and the PR. I hope to find time to look at it more closely soon. Its a part of the code base I am not particularly familiar with, however.


Thanks for the attention to issues. During the progress of the PR, there were some problems. Let me try to fix them.


Solution patch:
diff --git a/mesa/datacollection.py b/mesa/datacollection.py
index 309d56d8e27..63c0e7f0150 100644
--- a/mesa/datacollection.py
+++ b/mesa/datacollection.py
@@ -117,6 +117,9 @@ def __init__(
         self._agenttype_records = {}
         self.tables = {}
 
+        # add the signal of the validation of model reporter
+        self._validated = False
+
         if model_reporters is not None:
             for name, reporter in model_reporters.items():
                 self._new_model_reporter(name, reporter)
@@ -134,13 +137,66 @@ def __init__(
             for name, columns in tables.items():
                 self._new_table(name, columns)
 
+    def _validate_model_reporter(self, name, reporter, model):
+        """Validate model reporter and handle validation results appropriately.
+
+        Args:
+            name: Name of the reporter
+            reporter: Reporter definition (lambda/method/attribute/function list)
+            model: Model instance
+
+        Raises:
+            ValueError: If reporter is None or has invalid format
+            AttributeError: If model attribute doesn't exist
+            TypeError: If reporter type is not supported
+            RuntimeError: If reporter execution fails
+        """
+        self._validated = True  # put the change of signal firstly avoid losing efficacy
+
+        # Type 1: Lambda function
+        if isinstance(reporter, types.LambdaType):
+            try:
+                reporter(model)
+            except Exception as e:
+                raise RuntimeError(
+                    f"Lambda reporter '{name}' failed validation: {e!s}\n"
+                    f"Example: lambda m: len(m.agents)"
+                ) from e
+
+        # Type 2: Method of class/instance
+        if not callable(reporter) and not isinstance(reporter, types.LambdaType):
+            pass
+
+        # Type 3: Model attribute (string)
+        if isinstance(reporter, str):
+            try:
+                if not hasattr(model, reporter):
+                    raise AttributeError(
+                        f"Model reporter '{name}' references non-existent attribute '{reporter}'\n"
+                    )
+                getattr(model, reporter)  # 验证属性是否可访问
+            except AttributeError as e:
+                raise AttributeError(
+                    f"Model reporter '{name}' attribute validation failed: {e!s}\n"
+                ) from e
+
+        # Type 4: Function with parameters in list
+        if isinstance(reporter, list) and (not reporter or not callable(reporter[0])):
+            raise ValueError(
+                f"Invalid function list format for reporter '{name}'\n"
+                f"Expected: [function, [param1, param2]], got: {reporter}"
+            )
+
     def _new_model_reporter(self, name, reporter):
         """Add a new model-level reporter to collect.
 
         Args:
             name: Name of the model-level variable to collect.
-            reporter: Attribute string, or function object that returns the
-                      variable when given a model instance.
+            reporter: Can be one of four types:
+                1. Attribute name (str): "attribute_name"
+                2. Lambda function: lambda m: len(m.agents)
+                3. Method: model.get_count or Model.get_count
+                4. List of [function, [parameters]]
         """
         self.model_reporters[name] = reporter
         self.model_vars[name] = []
@@ -262,6 +318,10 @@ def get_reports(agent):
     def collect(self, model):
         """Collect all the data for the given model object."""
         if self.model_reporters:
+            if not self._validated:
+                for name, reporter in self.model_reporters.items():
+                    self._validate_model_reporter(name, reporter, model)
+
             for var, reporter in self.model_reporters.items():
                 # Check if lambda or partial function
                 if isinstance(reporter, types.LambdaType | partial):


Test patch:
diff --git a/tests/test_datacollector.py b/tests/test_datacollector.py
index 0b5153d3c2c..7f7de853f6a 100644
--- a/tests/test_datacollector.py
+++ b/tests/test_datacollector.py
@@ -352,5 +352,67 @@ def test_agenttype_superclass_reporter(self):
         self.assertTrue(super_data.equals(agent_data))
 
 
+class MockModelForErrors(Model):
+    """Test model for error handling."""
+
+    def __init__(self):
+        """Initialize the test model for error handling."""
+        super().__init__()
+        self.num_agents = 10
+        self.valid_attribute = "test"
+
+    def valid_method(self):
+        """Valid method for testing."""
+        return self.num_agents
+
+
+def helper_function(model, param1):
+    """Test function with parameters."""
+    return model.num_agents * param1
+
+
+class TestDataCollectorErrorHandling(unittest.TestCase):
+    """Test error handling in DataCollector."""
+
+    def setUp(self):
+        """Set up test cases."""
+        self.model = MockModelForErrors()
+
+    def test_lambda_error(self):
+        """Test error when lambda tries to access non-existent attribute."""
+        dc_lambda = DataCollector(
+            model_reporters={"bad_lambda": lambda m: m.nonexistent_attr}
+        )
+        with self.assertRaises(RuntimeError):
+            dc_lambda.collect(self.model)
+
+    def test_method_error(self):
+        """Test error when accessing non-existent method."""
+
+        def bad_method(model):
+            raise RuntimeError("This method is not valid.")
+
+        dc_method = DataCollector(model_reporters={"test": bad_method})
+
+        with self.assertRaises(RuntimeError):
+            dc_method.collect(self.model)
+
+    def test_attribute_error(self):
+        """Test error when accessing non-existent attribute."""
+        dc_attribute = DataCollector(
+            model_reporters={"bad_attribute": "nonexistent_attribute"}
+        )
+        with self.assertRaises(Exception):
+            dc_attribute.collect(self.model)
+
+    def test_function_error(self):
+        """Test error when function list is not callable."""
+        dc_function = DataCollector(
+            model_reporters={"bad_function": ["not_callable", [1, 2]]}
+        )
+        with self.assertRaises(ValueError):
+            dc_function.collect(self.model)
+
+
 if __name__ == "__main__":
     unittest.main()