Repository: modelscope/evalscope
Issue: The evaluation results (aime24, math-500, gpqa) difference betweern EvalType.CHECKPOINT and EvalType.SERVICE
Root cause: Documentation Deficiency

## Self-Check List

Before submitting an issue, please ensure you have completed the following steps:
- [x] I have carefully read the [relevant user documentation](https://evalscope.readthedocs.io/en/latest/get_started/parameters.html)
- [x] I have reviewed the [Frequently Asked Questions](https://evalscope.readthedocs.io/en/latest/get_started/faq.html)
- [x] I have searched and reviewed existing issues to confirm this is not a duplicate problem

## Problem Description

I have the same question when I start to use this framework.
When the `eval_type=EvalType.CHECKPOINT`, the results of aime-24 on dpsk-r1-distill-qwen-1.5b is 0.0667. The following is my code.

```python
from evalscope import TaskConfig, run_task
from evalscope.constants import EvalType

task_cfg = TaskConfig(
    model='/model/DeepSeek-R1-Distill-Qwen-1.5B',   # 模型名称 (需要与部署时的模型名称一致)
    # api_url='http://127.0.0.1:8801/v1',  # 推理服务地址
    # api_key='EMPTY',
    # eval_type=EvalType.SERVICE,   # 评测类型，SERVICE表示评测推理服务
    eval_type=EvalType.CHECKPOINT,
    datasets=[
        # 'math_500',  
        # 'gpqa',
        'aime24'
    ],
    dataset_args={ # EvalScope内置支持，无需指定数据集ID
        # 'math_500': {'few_shot_num': 8,'prompt_type': 'cot' } ,
        # 'gpqa': {'subset_list': ['gpqa_diamond'], 'few_shot_num': 8},
        'aime24': {'few_shot_num': 8, 'prompt_type': 'cot'}
    },
    eval_batch_size=32,       # 发送请求的并发数
    generation_config={       # 模型推理配置
        'max_tokens': 32768,  # 最大生成token数，建议设置为较大值避免输出截断
        'temperature': 0.6,   # 采样温度 (deepseek 报告推荐值)
        'top_p': 0.95,        # top-p采样 (deepseek 报告推荐值)
        'n': 1                # 每个请求产生的回复数量 (注意 lmdeploy 目前只支持 n=1)
    },
    # limit=5,
    # stream=True               # 是否使用流式请求，推荐设置为True防止请求超时
)
run_task(task_cfg=task_cfg)
```

After that, I learn and try to use the service mode (eval_type=EvalType.SERVICE). The following is my code: 
```python
from evalscope import TaskConfig, run_task
from evalscope.constants import EvalType

task_cfg = TaskConfig(
    model='/model/DeepSeek-R1-Distill-Qwen-1.5B',   # 模型名称 (需要与部署时的模型名称一致)
    api_url='http://127.0.0.1:8801/v1',  # 推理服务地址
    api_key='EMPTY',
    eval_type=EvalType.SERVICE,   # 评测类型，SERVICE表示评测推理服务
    # eval_type=EvalType.CHECKPOINT,
    datasets=[
        # 'math_500',  
        # 'gpqa',
        'aime24'
    ],
    dataset_args={ # EvalScope内置支持，无需指定数据集ID
        # 'math_500': {'few_shot_num': 8,'prompt_type': 'cot' } ,
        # 'gpqa': {'subset_list': ['gpqa_diamond'], 'few_shot_num': 8},
        'aime24': {'few_shot_num': 8, 'prompt_type': 'cot'}
    },
    eval_batch_size=32,       # 发送请求的并发数
    generation_config={       # 模型推理配置
        'max_tokens': 32768,  # 最大生成token数，建议设置为较大值避免输出截断
        'temperature': 0.6,   # 采样温度 (deepseek 报告推荐值)
        'top_p': 0.95,        # top-p采样 (deepseek 报告推荐值)
        'n': 1                # 每个请求产生的回复数量 (注意 lmdeploy 目前只支持 n=1)
    },
    # limit=5,
    # stream=True               # 是否使用流式请求，推荐设置为True防止请求超时
)
run_task(task_cfg=task_cfg)
```

Before executing this code, I set the `api_url`. For me, the code is:
```shell
python -m vllm.entrypoints.openai.api_server \
    --model /localssd/wmhu/model/DeepSeek-R1-Distill-Qwen-1.5B \
    --host 0.0.0.0 \
    --port 8801 \
    --tensor-parallel-size 1
```

Then I can get the results that are almost close to the dpsk-r1 paper.
|                         | aime-24 @ pass1 | math-500 @ overall | gpqa @ pass1 |
| ----------------------- | --------------- | ------------------ | ------------ |
| dpsk-r1 paper                   | 0.289           | 0.839              | 0.338        |
| evalscope, vllm backend | 0.267           | 0.814              | 0.303        |

So, my question is, what causes this difference? Is there any detail I miss? In my understanding, the `eval_type=EvalType.CHECKPOINT` has the same parameter as service mode.

## EvalScope Version (Required)
v0.xx.x

## Tools Used
- [x] Native / Native framework
- [ ] Opencompass backend
- [ ] VLMEvalKit backend
- [ ] RAGEval backend
- [ ] Perf / Model inference stress testing tool
- [ ] Arena / Arena mode

## Executed Code or Instructions

Please provide the main code or instructions you executed.

## Error Log

No error.

## Running Environment

- Operating System: Ubuntu 20.04.6 LTS
- Python Version: Python 3.10.18

## Additional Information

None.


When using a local checkpoint, the model’s generation parameters will be different. For details, please refer to : https://evalscope.readthedocs.io/en/latest/get_started/parameters.html#model-parameters

<img width="1592" height="1090" alt="Image" src="https://github.com/user-attachments/assets/3f2c15bf-a801-4aac-8b7f-90d74646e14a" />

> 感谢你的反馈！我们将关闭此问题。如果您有任何疑问，请随时重新打开它。如果EvalScope对您有所帮助，欢迎给我们点个STAR以示支持，谢谢！