README.md

metadata

license: cc-by-nc-nd-4.0
datasets:
  - kwaikeg/KAgentInstruct
  - kwaikeg/KAgentBench
language:
  - en
  - zh
pipeline_tag: text-generation

KwaiAgents (Github) is a series of Agent-related works open-sourced by the KwaiKEG from Kuaishou Technology. The open-sourced content includes:

1. KAgentSys-Lite: An experimental Agent Loop implemented based on open-source search engines, browsers, time, calendar, weather, and other tools, which is only missing the memory mechanism and some search capabilities compared to the system in the paper.
2. KAgentLMs: A series of large language models with Agent capabilities such as planning, reflection, and tool-use, acquired through the Meta-agent tuning proposed in the paper.
3. KAgentInstruct: Fine-tuned data of instructions generated by the Meta-agent in the paper.
4. KAgentBench: Over 3,000 human-edited, automated evaluation data for testing Agent capabilities, with evaluation dimensions including planning, tool-use, reflection, concluding, and profiling.

User Guide

Direct usage

Tutorial can refer to QwenLM/Qwen

from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers.generation import GenerationConfig

tokenizer = AutoTokenizer.from_pretrained("kwaikeg/kagentlms_qwen_7b_mat", trust_remote_code=True)

model = AutoModelForCausalLM.from_pretrained(
    "kwaikeg/kagentlms_qwen_7b_mat",
    device_map="auto",
    trust_remote_code=True
).eval()

response, history = model.chat(tokenizer, "你好", history=None)
print(response)

AgentLMs as service

We recommend using vLLM and FastChat to deploy the model inference service. First, you need to install the corresponding packages (for detailed usage, please refer to the documentation of the two projects):

1. For Qwen-7B-MAT, install the corresponding packages with the following commands

pip install vllm
pip install "fschat[model_worker,webui]"

2. For Baichuan-13B-MAT, install the corresponding packages with the following commands

pip install "fschat[model_worker,webui]"
pip install vllm==0.2.0
pip install transformers==4.33.2

To deploy KAgentLMs, you first need to start the controller in one terminal.

python -m fastchat.serve.controller

Secondly, you should use the following command in another terminal for single-gpu inference service deployment:

python -m fastchat.serve.vllm_worker --model-path $model_path --trust-remote-code

Where $model_path is the local path of the model downloaded. If the GPU does not support Bfloat16, you can add --dtype half to the command line.

Thirdly, start the REST API server in the third terminal.

python -m fastchat.serve.openai_api_server --host localhost --port 8888

Finally, you can use the curl command to invoke the model same as the OpenAI calling format. Here's an example:

curl http://localhost:8888/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model": "kagentlms_qwen_7b_mat", "messages": [{"role": "user", "content": "Who is Andy Lau"}]}'

Here, change kagentlms_qwen_7b_mat to the model you deployed.