Update README.md

README.md

@@ -2,36 +2,33 @@
 license: cc-by-4.0
 datasets:
 - Salesforce/xlam-function-calling-60k
-- MadeAgents/XLAM-7.5k-Irrelevance
+- MadeAgents/xlam-irrelevance-7.5k
 base_model:
 - Qwen/Qwen2.5-3B-Instruct
 ---
 
-# Hammer2.0-3b Function Calling Model
-
 ## Introduction
 We're excited to release lightweight Hammer 2.0 models ([0.5B](https://huggingface.co/MadeAgents/Hammer2.0-0.5b) ,   [1.5B](https://huggingface.co/MadeAgents/Hammer2.0-1.5b) ,   [3B](https://huggingface.co/MadeAgents/Hammer2.0-3b) ,  and  [7B](https://huggingface.co/MadeAgents/Hammer2.0-7b)) with strong function calling capability, which empower developers to build personalized, on-device agentic applications.
 
 ## Model Details
-Hammer2.0 finetuned based on [Qwen 2.5 series](https://huggingface.co/collections/Qwen/qwen25-66e81a666513e518adb90d9e) and [Qwen 2.5 coder series](https://huggingface.co/collections/Qwen/qwen25-coder-66eaa22e6f99801bf65b0c2f) using function masking techniques. It's trained using the [APIGen Function Calling Datasets](https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k) containing 60,000 samples, supplemented by [XLAM-7.5k-Irrelevance](https://huggingface.co/datasets/MadeAgents/XLAM-7.5k-Irrelevance) we generated. Hammer2.0 has achieved exceptional performances across numerous function calling benchmarks. For detailed data construction, training methods, and evaluation strategies, please refer to our paper [Hammer: Robust Function-Calling for On-Device Language Models via Function Masking](https://arxiv.org/abs/2410.04587) and the [Hammer GitHub repository](https://github.com/MadeAgents/Hammer) .
-
+Hammer2.0 finetuned based on [Qwen 2.5 series](https://huggingface.co/collections/Qwen/qwen25-66e81a666513e518adb90d9e) and [Qwen 2.5 coder series](https://huggingface.co/collections/Qwen/qwen25-coder-66eaa22e6f99801bf65b0c2f) using function masking techniques. It's trained using the [APIGen Function Calling Datasets](https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k) containing 60,000 samples, supplemented by [xlam-irrelevance-7.5k](https://huggingface.co/datasets/MadeAgents/xlam-irrelevance-7.5k) we generated. Hammer2.0 has achieved exceptional performances across numerous function calling benchmarks. For detailed data construction, training methods, and evaluation strategies, please refer to our paper [Hammer: Robust Function-Calling for On-Device Language Models via Function Masking](https://arxiv.org/abs/2410.04587) and the [Hammer GitHub repository](https://github.com/MadeAgents/Hammer) .
 
 ## Evaluation
-The evaluation results of Hammer 2.0 series on the Berkeley Function-Calling Leaderboard (BFCL) are presented in the following table:
+The evaluation results of Hammer 2.0 models on the Berkeley Function-Calling Leaderboard (BFCL-v3) are presented in the following table:
 <div style="text-align: center;">
     <img src="v2_figures/bfcl.PNG" alt="overview" width="1000" style="margin: auto;">
 </div>
+Our Hammer 2.0 series consistently achieves corresponding best performance at comparable scales. The 7B model outperforms most function calling enchanced models, and the 1.5B model also achieves unexpected performance.
 
+In addition, we evaluated the Hammer 2.0 models on other academic benchmarks to further demonstrate the generalization ability of our models.
 
-In addition, we evaluated Hammer2.0 on other academic benchmarks to further show our model's generalization ability:
 <div style="text-align: center;">
     <img src="v2_figures/others.PNG" alt="overview" width="1000" style="margin: auto;">
 </div>
-
-On comparison, Hammer 2.0 outperforms models with similar sizes and even surpass many larger models overall. 
+Hammer 2.0 models showcase highly stable performance, suggesting the robustness of Hammer 2.0 series. In contrast, the baseline approaches display varying levels of effectiveness on these other benchmarks. 
 
 ## Requiements
-The code of Hammer2.0-7b has been in the latest Hugging face transformers and we advise you to install `transformers>=4.37.0`.
+The code of Hammer 2.0 models have been in the latest Hugging face transformers and we advise you to install `transformers>=4.37.0`.
 
 ## How to Use
 This is a simple example of how to use our model.
@@ -39,18 +36,15 @@ This is a simple example of how to use our model.
 import json
 import torch 
 from transformers import AutoModelForCausalLM, AutoTokenizer
-
 model_name = "MadeAgents/Hammer2.0-3b"
 model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype="auto", trust_remote_code=True)
 tokenizer = AutoTokenizer.from_pretrained(model_name) 
-
 # Please use our provided instruction prompt for best performance
 TASK_INSTRUCTION = """You are a tool calling assistant. In order to complete the user's request, you need to select one or more appropriate tools from the following tools and fill in the correct values for the tool parameters. Your specific tasks are:
 1. Make one or more function/tool calls to meet the request based on the question.
 2. If none of the function can be used, point it out and refuse to answer.
 3. If the given question lacks the parameters required by the function, also point it out.
 """
-
 FORMAT_INSTRUCTION = """
 The output MUST strictly adhere to the following JSON format, and NO other text MUST be included.
 The example format is as follows. Please make sure the parameter type is correct. If no function call is needed, please directly output an empty list '[]'
@@ -61,10 +55,8 @@ The example format is as follows. Please make sure the parameter type is correct
 ]
 ```
 """
-
 # Define the input query and available tools
 query = "Where can I find live giveaways for beta access and games? And what's the weather like in New York, US?" 
-
 live_giveaways_by_type = {
     "name": "live_giveaways_by_type",
     "description": "Retrieve live giveaways from the GamerPower API based on the specified type.",
@@ -108,7 +100,6 @@ get_stock_price={
             "required": ["ticker"]
         }
     }
-
 def convert_to_format_tool(tools):
     ''''''
     if isinstance(tools, dict):
@@ -141,12 +132,10 @@ def build_prompt(task_instruction: str, format_instruction: str, tools: list, qu
 openai_format_tools = [live_giveaways_by_type, get_current_weather,get_stock_price]
 format_tools = convert_to_format_tool(openai_format_tools)
 content = build_prompt(TASK_INSTRUCTION, FORMAT_INSTRUCTION, format_tools, query)
-
 messages=[
     { 'role': 'user', 'content': content}
 ]
 inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
-
 # tokenizer.eos_token_id is the id of <|EOT|> token
 outputs = model.generate(inputs, max_new_tokens=512, do_sample=False, num_return_sequences=1, eos_token_id=tokenizer.eos_token_id)
 print(tokenizer.decode(outputs[0][len(inputs[0]):], skip_special_tokens=True))