## 4 Experiments
### Experimental Results
#### 4.3.3 Cost Comparison  
In addition to performance evaluation, a comparative analysis of the cost associated with using different LLMs is conducted. The API services for these models were primarily accessed through third-party platforms such as Alibaba Cloud [52]. While pricing may vary depending on deployment region, the rates used in this study are based on publicly available pricing from each provider. The total number of tokens processed across all test samples ranges approximately from 6M to 8.8M. A detailed cost is provided in Table 5.  
As shown in Table 5, reasoning models, including DeepSeek-R1 and O3-Mini, generally incur higher costs compared to non-reasoning models. This difference is largely attributable to their increased output token usage, which stems from generating detailed step-by-step reasoning explanations. The token consumption distribution between input and output for reasoning and non-reasoning models is summarized in Table 6.  
For instance, DeepSeek-R1, a reasoning model, utilizes approximately 3.081M output tokens, more than twice the amount produced by DeepSeek-V3, a non-reasoning model with 1.426M output tokens. Despite comparable input token usage, the substantial increase in output reflects the cost overhead incurred by the reasoning mechanism. This distinction underscores the necessity of separating reasoning models from non-reasoning models when analyzing cost-effectiveness, as reasoning tasks inherently demand more resources due to their extended output generation and intermediate reasoning traces.  
Among the non-reasoning models, Claude-3.5-Haiku, GPT-4o-Mini, and DeepSeek-V3 produce fewer output token counts and thus incur lower costs. Notably, GPT-4o-Mini  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **Input Tokens (M)** & **Output Tokens (M)** & **Total Tokens (M)** \\ \hline DeepSeek-V3 & 4.758 & 1.426 & 6.184 \\ \hline DeepSeek-R1 & 4.722 & 3.081 & 7.803 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Token Distribution Comparison. DeepSeek-V3 vs. DeepSeek-R1.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **Input Price** & **Output Price** & **Total Tokens * & **Total Cost** \\  & **(USD 5/1M)** & **(USD 5/1M)** & **(M)** & **(USD 5)** \\ \hline Claude-3.5-Haiku & 0.25 & 1.25 & 6.057 & 11.62 \\ \hline GPT-4o-Mini & 0.15 & 0.6 & 6.463 & 1.53 \\ \hline O3-Mini & 1.1 & 4.4 & 8.797 & 19.96 \\ \hline DeepSeek-V3 & 0.276 (\%2 CNY) & 1.1 (\%8 CNY) & 6.184 & 2.85 (\%20.67 CNY) \\ \hline
**DeepSeek-R1** & **0.552 (\%4 CNY)** & **2.21 (\%16 CNY)** & **7.803** & **9.38 (\%68.04 CNY)** \\ \hline \hline \end{tabular}
\end{table}
Table 5: Cost Comparison of LLMs in API Services.  
achieves the lowest total cost at $1.53, though it comes with a trade-off in performance as its mAP is the second lowest (0.8670). DeepSeek-V3, on the other hand, maintains a strong mAP of 0.9411 while keeping the total cost at a modest $2.85 (\(\forall\)20.67), demonstrating a favorable performance-to-cost ratio within the non-reasoning group.  
For reasoning models, DeepSeek-R1 stands out as the most cost-efficient option. Despite the increased output token count, it maintains a relatively low total cost of $9.38 (\(\forall\)68.04) while delivering the highest performance across all evaluation metrics. In comparison, O3-Mini consumes significantly 8.797M tokens in total and incurs a much higher cost of $19.96, which is more than twice that of DeepSeek-R1.  
In summary, DeepSeek-R1 offers the best trade-off between cost and performance. Its ability to deliver state-of-the-art accuracy while maintaining reasonable token usage makes it particularly well-suited for practical deployment in real-world vulnerability detection systems, where both efficiency and reliability are critical.