Repository: modelscope/evalscope
Issue: BFCL-v3数据集type字段解析错误
Root cause: Validation Gap

## 自查清单

在提交 issue 之前，请确保您已完成以下步骤:
- [x] 我已仔细阅读了[相关使用说明文档](https://evalscope.readthedocs.io/zh-cn/latest/get_started/parameters.html)
- [x] 我已查看了[常见问题解答](https://evalscope.readthedocs.io/zh-cn/latest/get_started/faq.html)
- [x] 我已搜索并查看了现有的 issues，确认这不是一个重复的问题

## 问题描述

对bfcl_v3进行评估之后会出现如下的报错：

```
prediction failed: due to 1 validation error for ToolInfo
parameters.properties.exclusion
  Value error, Unsupported type: {'type': 'string', 'description': 'The type of the exclusion e.g window, door etc.'} for Python to JSON conversion. [type=value_error, input_value={'type': 'object', 'prope...sion if not specified.'}, input_type=dict]
```

在bfcl_v3中会有多条数据会引起上述的报错，以其中的某条数据为例进行说明。数据内容如下：

```
{
  "id": "simple_260",
  ... ...
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "paint_requirement_calculate",
        "description": "Calculate the amount of paint required to paint a given area. Account for coverage efficiency of the paint and exclusions (like windows). Note that the provided function is in Python 3 syntax.",
        "parameters": {
          "type": "object",
          "properties": {
            "area": {
              "type": "object",
              "properties": {
                "width": {
                  "type": "integer",
                  "description": "The width of the area to be painted in feet."
                },
                "height": {
                  "type": "integer",
                  "description": "The height of the area to be painted in feet."
                }
              },
              "description": "The area to be painted."
            },
            "paint_coverage": {
              "type": "integer",
              "description": "Coverage area per gallon of the paint in square feet.",
              "default": 350
            },
            "exclusion": {
              "type": "object",
              "properties": {
                "type": {
                  "type": "string",
                  "description": "The type of the exclusion e.g window, door etc."
                },
                "area": {
                  "type": "integer",
                  "description": "The area of the exclusion in square feet."
                }
              },
              "description": "Area not to be painted. Default to not use any exclusion if not specified."
            }
          },
          "required": [
            "area",
            "paint_coverage"
          ]
        }
      }
    }
  ],
  ......
}
```

引起报错的是上述数据中的第35行部分，如下：

```
                "type": {
                  "type": "string",
                  "description": "The type of the exclusion e.g window, door etc."
                },
```

报错对应的代码为：https://github.com/modelscope/evalscope/blob/main/evalscope/utils/json_schema.py#L195，如下：

```
def python_type_to_json_type(python_type: Optional[str]) -> JSONType:
    if python_type is not None and python_type in get_args(JSONType):
        return python_type
    if python_type == 'str':
        return 'string'
    elif python_type == 'int':
        return 'integer'
    elif python_type == 'float':
        return 'number'
    elif python_type == 'bool':
        return 'boolean'
    elif python_type == 'list':
        return 'array'
    elif python_type == 'dict':
        return 'object'
    elif python_type == 'None':
        return 'null'
    elif python_type is None:
        # treat 'unknown' as string as anything can be converted to string
        return 'string'
    else:
        raise ValueError(f'Unsupported type: {python_type} for Python to JSON conversion.')
```

实际出问题是在调用 `python_type_to_json_type` 的 `convert_type_before_validation` 函数中 （https://github.com/modelscope/evalscope/blob/main/evalscope/utils/json_schema.py#L67），如下：

```
    @model_validator(mode='before')
    def convert_type_before_validation(cls, values):
        values = deepcopy(values)

        def recursive_convert_type(obj):
            if isinstance(obj, dict):
                if 'type' in obj:
                    obj['type'] = python_type_to_json_type(obj['type'])
                for k, v in obj.items():
                    obj[k] = recursive_convert_type(v)
            elif isinstance(obj, list):
                return [recursive_convert_type(item) for item in obj]
            return obj

        return recursive_convert_type(values)
```

这个函数认为当 key 为 "type" 时一定是用来指定数据类型的，但是就如上面展示的，在bfcl_v3中有多条数据，它的tools中的property命名为了 type，所以导致了报错。

建议可以将函数`convert_type_before_validation`修改为如下内容。不过我不清楚这样修改对于其他benchmark是否会影响，仅做建议。

```
    @model_validator(mode='before')
    def convert_type_before_validation(cls, values):
        values = deepcopy(values)

        def recursive_convert_type(obj):
            if isinstance(obj, dict):
                if ('type' in obj) and (obj['type'] is None or isinstance(obj['type'], str)):
                    obj['type'] = python_type_to_json_type(obj['type'])
                for k, v in obj.items():
                    obj[k] = recursive_convert_type(v)
            elif isinstance(obj, list):
                return [recursive_convert_type(item) for item in obj]
            return obj

        return recursive_convert_type(values)
```


## EvalScope 版本（必填）

v1.0.2

## 使用的工具
- [x] Native / 原生框架
- [ ] Opencompass backend
- [ ] VLMEvalKit backend
- [ ] RAGEval backend
- [ ] Perf / 模型推理压测工具
- [ ] Arena / 竞技场模式

## 执行的代码或指令

```
from evalscope import TaskConfig, run_task


def run_qwen3_eval(model_config):
    task_cfg = TaskConfig(
        model=model_config["model"],
        api_url=model_config["api_url"],
        eval_type="openai_api",
        datasets=[
            "bfcl_v3",
        ],
        dataset_args={},
        eval_batch_size=8192,
        generation_config={
            "max_tokens": 8192,  # 最大生成token数，建议设置为较大值避免输出截断
            "temperature": 0.2,  # 采样温度 (qwen 报告推荐值)
            "top_p": 0.8,  # top-p采样 (qwen 报告推荐值)
            "top_k": 20,  # top-k采样 (qwen 报告推荐值)
            "n": 1,  # 每个请求产生的回复数量
            "extra_body": {"chat_template_kwargs": {"enable_thinking": model_config["enable_thinking"]}},  # 关闭思考模式
            "timeout": 60000,
            "stream": True,
        },
    )
    run_task(task_cfg=task_cfg)


if __name__ == "__main__":
    model_config_qwen3_32b_no_thinking = {
        "model": "Qwen3-32B",
        "api_url": "http://127.0.0.1:8801/v1/chat/completions",
        "enable_thinking": True,
    }
    run_qwen3_eval(model_config_qwen3_32b_no_thinking)
```

## 错误日志

```
prediction failed: due to 1 validation error for ToolInfo
parameters.properties.exclusion
  Value error, Unsupported type: {'type': 'string', 'description': 'The type of the exclusion e.g window, door etc.'} for Python to JSON conversion. [type=value_error, input_value={'type': 'object', 'prope...sion if not specified.'}, input_type=dict]
```


## 运行环境

- 操作系统：Ubuntu 22.04.4
- Python版本：3.10.12

## 其他信息

如果有其他相关信息，请在此处提供。


感谢你的反馈，我修复一下

已修复，可以用main分支代码试试

测试没问题了，感谢🙏