examples/cookbook/01-Chat-Completion.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 对话补全(Chat Completion)\n",
    "\n",
    "文心一言系列对话模型可以理解自然语言，并以文本输出与用户进行对话。将对话上下文与输入文本提供给模型，由模型给出新的回复，即为对话补全。对话补全功能可应用于广泛的实际场景，例如对话沟通、内容创作、分析控制等。\n",
    "\n",
    "Chat Completion模型包括以下三类，用户可自行选择哪一类模型\n",
    "\n",
    "* ernie-bot             文心一言旗舰模型（ernie-bot）\n",
    "\n",
    "* ernie-bot-turbo       文心一言轻量模型（ernie-bot-turbo）\n",
    "\n",
    "* ernie-bot-4           文心一言4.0模型（ernie-bot-4）\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. 接口以及API调用\n",
    "此文档中简单介绍一下相关的参数，更详细的内容请参考[对话补全API文档](../../docs/api_reference/chat_completion.md)。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Python接口为\n",
    "\n",
    "```python\n",
    "erniebot.ChatCompletion.create(**kwargs: Any) -> Union[EBResponse, Iterator[EBResponse]]\n",
    "```\n",
    "\n",
    "\n",
    "以下是该接口的所有参数\n",
    "\n",
    "| 参数名 | 类型 | 必填 | 描述 |\n",
    "| :--- | :--- | :------- | :---- |\n",
    "| model | str | 是 | 模型名称。当前支持`'ernie-bot'`、`'ernie-bot-turbo'`、`'ernie-bot-4'`和`'ernie-bot-8k'`。 |\n",
    "| messages | list[dict] | 是 | 对话上下文信息。列表中的元素个数须为奇数。|\n",
    "| functions | list[dict] | 否 | 可触发函数的描述列表。ernie-bot-turbo模型暂不支持此参数。 |\n",
    "| top_p | float | 否 | 生成的token从概率和恰好达到或超过`top_p`的token集合中采样得到。 <br>(1) 影响生成文本的多样性，取值越大，生成文本的多样性越强； <br>(2) 默认`0.8`，取值范围为`[0, 1.0]`； <br>(3) 建议只设置此参数和`temperature`中的一个。 |\n",
    "| temperature | float | 否 | 用于调节模型输出概率分布，从而控制生成结果的随机性。 <br>(1) 较高的数值会使生成结果更加随机，而较低的数值会使结果更加集中和确定； <br>(2) 默认`0.95`，范围为`(0, 1.0]`，不能为`0`； <br>(3) 建议只设置此参数和`top_p`中的一个。 |\n",
    "| penalty_score | float | 否 | 通过对已生成的token增加惩罚，减少重复生成的现象。此参数值越高则惩罚越大。 <br>(1) 值越大表示惩罚越大； <br>(2) 默认`1.0`，取值范围：`[1.0, 2.0]`。 |\n",
    "| system | str | 否 | 提示模型行为的文本。如果设置了`functions`，则不支持设置此参数。 |\n",
    "| user_id | str | 否 | 终端用户的唯一标识符，可以监视和检测滥用行为，防止接口被恶意调用。 |\n",
    "| stream | bool | 否 | 是否流式返回数据，默认`False`。 |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. 单轮对话"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先需要申请以及配置ACCESS_TOKEN，具体请参考[认证鉴权](../../docs/authentication.md)，然后将其配置到系统环境变量，或者直接指定使用。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import erniebot,os\n",
    "erniebot.api_type = 'aistudio'\n",
    "erniebot.access_token = '<eb-access-token>'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1 关键输入变量Messages\n",
    "`messages: List[dict]`为一个Python list，其中每个元素为一个dict。\n",
    "\n",
    "每个元素包含如下键值对：\n",
    "\n",
    "| 键名 | 值类型 | 必填 | 值描述 |\n",
    "|:--- | :---- | :--- | :---- |\n",
    "| role | str | 是 | `'user'`表示用户，`'assistant'`表示对话助手，`'function'`表示函数：详见[函数调用.ipynb](02-Function-Calling.ipynb)。 |\n",
    "| content | str or None | 是 | 当`role`不为`'function'`时，表示消息内容；当`role`为`'function'`时，表示函数响应参数。若未设置`function_call`，则`content`不能为`None`。 |\n",
    "| name | str | 否 | 消息的作者。当`role`为`'function'`时，必须设置`name`，此时`name`为函数名称。 |\n",
    "| function_call | dict | 否 | 由模型生成的函数调用信息，包含函数名称和请求参数等。若设置`function_call`，则`role`必须为`'assistant'`，`content`可以为`None`。 |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "chat_message = [\n",
    "    {'role': 'user', 'content': \"请给百度写一首诗\"}\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 不同模型结果对比"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`ernie-bot`: 文心一言旗舰模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "百度高楼立，信息丰富多。\n",
      "搜索引擎鼎盛，知识库无限多。\n",
      "万千用户信赖，每时每刻在线。\n",
      "服务全球各地，为人类知识铺路。\n"
     ]
    }
   ],
   "source": [
    "response = erniebot.ChatCompletion.create(model='ernie-bot', \n",
    "                                          messages=chat_message)\n",
    "print(response.get_result())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`ernie-bot-turbo`: 文心一言轻量版模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "百度如海纳百川，\n",
      "科技之舟航向前。\n",
      "搜索答案如探囊，\n",
      "知识之库任我观。\n",
      "\n",
      "人工智能领风骚，\n",
      "智能问答解疑难。\n",
      "信息时代先锋者，\n",
      "创新引领未来篇。\n"
     ]
    }
   ],
   "source": [
    "response = erniebot.ChatCompletion.create(model='ernie-bot-turbo', \n",
    "                                          messages=chat_message)\n",
    "print(response.get_result())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`ernie-bot-4`: 基于文心大模型4.0版本的文心一言"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "百度\n",
      "百度搜索引擎强，信息丰富多样长。\n",
      "万千用户信赖它，快速准确找答案。\n"
     ]
    }
   ],
   "source": [
    "response = erniebot.ChatCompletion.create(model='ernie-bot-4', \n",
    "                                          messages=chat_message)\n",
    "print(response.get_result())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 不同参数结果对比"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "chat_message = [\n",
    "    {'role': 'user', 'content': \"请用文心一言写一首藏头诗\"}\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`temperature`\n",
    "\n",
    "* 默认0.95, 取值范围：(0,1.0]\n",
    "\n",
    "* 描述：更改temperature，temperature越高，随机性越强，表现为创造性更强。temperature越低，随机性越弱，更倾向于生成固定的置信内容。\n",
    "\n",
    "* 和top_p设置一个即可"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "下面是根据您的要求写的一首藏头诗：\n",
      "\n",
      "文风洒脱似飞鸿，心境宽广如海阔。 一言惊醒梦中人，言语犀利透纸背。\n"
     ]
    }
   ],
   "source": [
    "\n",
    "response = erniebot.ChatCompletion.create(\n",
    "    model = 'ernie-bot-4', \n",
    "    messages = chat_message,\n",
    "    temperature = 0.1,\n",
    ")\n",
    "print(response.get_result())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`penalty_score`\n",
    "*  默认1.0，取值范围：[1.0, 2.0]\n",
    "*  越大则越不容易产生重复的token"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "下面是根据您的要求写的一首藏头诗：\n",
      "\n",
      "文风洒脱笔墨情，心灵手巧赋诗歌。 一腔热血抒胸臆，言语真挚感人多。\n"
     ]
    }
   ],
   "source": [
    "response = erniebot.ChatCompletion.create(\n",
    "    model = 'ernie-bot-4', \n",
    "    messages = chat_message,\n",
    "    penalty_score = 1.8,\n",
    ")\n",
    "print(response.get_result())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`system`\n",
    "* 提示模型行为的文本。如果设置了`functions`，则不支持设置此参数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "你好，我是马斯克。很高兴与你交流。有什么想法或者问题想要讨论吗？\n"
     ]
    }
   ],
   "source": [
    "chat_message = [\n",
    "    {'role': 'user', 'content': \"你好\"}\n",
    "]\n",
    "\n",
    "response = erniebot.ChatCompletion.create(\n",
    "    system = '假设你是马斯克，请用马斯克的语气和用户进行对话',\n",
    "    model = 'ernie-bot-4', \n",
    "    messages = chat_message,\n",
    ")\n",
    "print(response.get_result())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.4 返回Response参数解读\n",
    "当返回结果非流式时，以下为返回结果的所有键值对。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  'rcode': 200,\n",
      "  'rheaders': {\n",
      "    'Connection': 'keep-alive',\n",
      "    'Content-Security-Policy': 'frame-ancestors https://*.baidu.com/',\n",
      "    'Content-Type': 'application/json',\n",
      "    'Date': 'Tue, 07 Nov 2023 08:07:01 GMT',\n",
      "    'Server': 'nginx',\n",
      "    'Statement': 'AI-generated',\n",
      "    'Vary': 'Origin, Access-Control-Request-Method, Access-Control-Request-Headers',\n",
      "    'X-Frame-Options': 'allow-from https://*.baidu.com/',\n",
      "    'X-Request-Id': '1e721d46a2ceebdabf98485d28534d75',\n",
      "    'Transfer-Encoding': 'chunked'\n",
      "  },\n",
      "  'id': 'as-jtt9xe2vfb',\n",
      "  'object': 'chat.completion',\n",
      "  'created': 1699344421,\n",
      "  'result': '百度高楼立，信息丰富多。\\n搜索便捷快，知识库无限。\\n问题解答准，学习效率高。\\n生活好帮手，感恩百度好。',\n",
      "  'usage': {\n",
      "    'prompt_tokens': 6,\n",
      "    'completion_tokens': 37,\n",
      "    'total_tokens': 43\n",
      "  },\n",
      "  'need_clear_history': False,\n",
      "  'is_truncated': False\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| 字段名 | 类型 | 描述 |\n",
    "| :--- | :---- | :---- |\n",
    "| rcode | int | HTTP响应状态码。 |\n",
    "| result | str | 模型生成的回复文本。 |\n",
    "| is_truncated | bool | 生成文本是否被长度限制截断。 |\n",
    "| sentence_id | int | 当前片段的序号，从`0`开始计数。仅流式模式下包含该字段。 |\n",
    "| need_clear_history | bool | 用户输入是否存在安全风险，是否应该关闭当前会话、清理历史会话信息。 <br>`True`：是，表示用户输入存在安全风险，建议关闭当前会话，清理历史会话信息； <br>`False`：否，表示用户输入无安全风险。 |\n",
    "| ban_round | int | 当`need_clear_history`为`True`时，此字段表示第几轮对话有敏感信息。如果是当前轮次存在问题，则`ban_round=-1`。 |\n",
    "| is_end | bool | 当前片段是否为生成结果的最后一段文本。仅流式模式下包含该字段。 |\n",
    "| usage | dict | 输入、输出token统计信息。token数量采用如下公式估算：`token数 = 汉字数 + 单词数 * 1.3`。 <br>`prompt_tokens`：输入token数量（含上下文拼接）； <br>`completion_tokens`：当前生成结果包含的token数量； <br>`total_tokens`：输入与输出的token总数； <br>`plugins`：插件消耗的token数量。 |\n",
    "| function_call | dict | 由模型生成的函数调用信息，包含函数名称和请求参数等。 |\n",
    "\n",
    "假设`resp`为一个`erniebot.response.EBResponse`对象，字段的访问方式有2种：`resp['result']`或`resp.result`均可获取`result`字段的内容。此外，可以使用`resp.get_result()`获取响应中的“主要结果”：当模型给出函数调用信息时（此时，`resp`具有`function_call`字段），`resp.get_result()`的返回结果与`resp.function_call`一致；否则，`resp.get_result()`的返回结果与`resp.result`一致，即模型给出的回复文本。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. 单轮对话(流式)\n",
    "当输入参数`steam = True`时，返回结果为一个生成器。使用前需先配置认证鉴权，详情参考[认证鉴权](../../docs/authentication.md)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "import platform,sys\n",
    "from IPython.display import clear_output as clear\n",
    "def _clear_screen():\n",
    "    os.system(\"cls\" if platform.system() == \"Windows\" else \"clear\")\n",
    "    if 'ipykernel' in sys.modules:\n",
    "        clear()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "深圳周末可以去玩的地方有很多，以下是一些推荐：\n",
      "\n",
      "* 深圳湾公园：集休闲娱乐、健身运动、观光旅游、体验自然等多功能活动的区域，更成为展现深圳现代滨海城市魅力和形象的标志。\n",
      "* 大梅沙海滨公园：一个著名的海滩度假胜地，可以在海滩上休闲、游泳、冲浪等。\n",
      "* 南头古城：一个历史悠久的古城，可以了解深圳的历史文化。\n",
      "* 西丽湖度假村：一个环境优美、空气清新的度假胜地，可以在这里进行休闲度假。\n",
      "* 较场尾海滩：一个美丽的海滩，可以在这里进行游泳、冲浪等水上活动。\n",
      "* 茶溪谷度假公园：一个集休闲度假、观光旅游、户外探险等多种功能于一体的综合性度假胜地。\n",
      "* 塘朗山郊野公园：一个自然环境优美的地方，可以在这里进行登山、野餐等活动。\n",
      "* 立新湖景观长廊：一个拥有美丽湖景的地方，可以在这里散步、观景。\n",
      "* 深圳锦绣中华：一个集合了中华各地名胜古迹的旅游景点。\n",
      "* 香蜜公园：一个环境优美、空气清新的公园，可以在这里进行休闲散步、健身等活动。\n",
      "* 沙头角：一个美丽安静的地方，城市喧嚣中难得的一片净土。周末的时候，可以坐在海边怡然自得的钓鱼，可以在街道上悠闲的闲逛。\n",
      "* 南海意库：在深圳众多的文创园里，南海意库说不上大，却自有他独特的风味。这里有主题餐饮、创意零售、咖啡酒吧等数十家特色店。\n",
      "\n",
      "总的来说，深圳周末可以去玩的地方非常多，可以根据个人兴趣和偏好来选择合适的景点。\n"
     ]
    }
   ],
   "source": [
    "import erniebot\n",
    "\n",
    "erniebot.api_type = 'aistudio'\n",
    "erniebot.access_token = '<eb-access-token>'\n",
    "\n",
    "response = erniebot.ChatCompletion.create(\n",
    "    model='ernie-bot',\n",
    "    messages=[{\n",
    "        'role': 'user',\n",
    "        'content': \"周末深圳去哪里玩？\"\n",
    "    }],\n",
    "    top_p=0.95,\n",
    "    stream=True)\n",
    "\n",
    "result = \"\"\n",
    "\n",
    "for resp in response:\n",
    "    result += resp.get_result()\n",
    "    _clear_screen()\n",
    "    print(result,flush=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. 多轮对话\n",
    "在如下示例中，为了与模型进行多轮对话，可以参考使用以下示例代码，我们将模型的回复结果插入到messages中再继续请求，您可以选择chat或者stream chat进行简单体验。使用前需先配置认证鉴权，详情参考[认证鉴权](../../docs/authentication.md)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "百度\n",
      "百度搜索万象新，信息丰富无边际。 一搜即得千万答，知识渊博难相匹。 网络世界任我行，寻找答案如觅食。 便捷高效人人赞，百度引领新时代。\n"
     ]
    }
   ],
   "source": [
    "class ERNIE:\n",
    "    def __init__(\n",
    "            self,\n",
    "            model: str,\n",
    "            access_token: str = None\n",
    "    ) -> None:\n",
    "        self.chat_history = []\n",
    "        self.model = model\n",
    "        erniebot.api_type = 'aistudio'\n",
    "        erniebot.access_token = os.getenv(\"EB_ACCESS_TOKEN\") if not access_token else access_token \n",
    "\n",
    "    def chat(\n",
    "        self,query:str\n",
    "    ) -> str:\n",
    "        \"Use this function to chat with ERNIE BOT\"\n",
    "        self.chat_history.append({'role': 'user', 'content': query})\n",
    "        response = erniebot.ChatCompletion.create(\n",
    "                model=self.model, \n",
    "                messages=self.chat_history,\n",
    "                )\n",
    "        self.chat_history.append({'role': 'assistant', 'content': response.get_result()})\n",
    "        return response.get_result()\n",
    "\n",
    "    def chat_stream(\n",
    "        self,query:str\n",
    "    ) -> None:\n",
    "        \"Use this function to chat with ERNIE BOT\"\n",
    "        self.chat_history.append({'role': 'user', 'content': query})\n",
    "        response = erniebot.ChatCompletion.create(\n",
    "                model=self.model, \n",
    "                messages=self.chat_history,\n",
    "                stream=True\n",
    "                )\n",
    "        result = \"\"\n",
    "\n",
    "        for resp in response:\n",
    "            result += resp.get_result()\n",
    "            _clear_screen()\n",
    "            print(result,flush=True)\n",
    "        self.chat_history.append({'role': 'assistant', 'content': result})\n",
    "        \n",
    "       \n",
    "    def clear(self):\n",
    "        self.chat_history = []\n",
    "\n",
    "ERNIE_BOT = ERNIE(model='ernie-bot-4')\n",
    "print(ERNIE_BOT.chat('请给百度写一首诗'))\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "百度文心一言新，智能对话引风潮。 语言理解深似海，回答精准似琼瑶。 网络世界畅游行，知识问答如飞鸟。 科技魅力人人赞，文心一言领未来。\n"
     ]
    }
   ],
   "source": [
    "ERNIE_BOT.chat_stream('请帮我加入文心一言的元素后，重新写一首诗')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "“百度文心一言新，智能对话引风潮”这两句诗主要描述了百度文心一言的新颖性和它在智能对话领域的引领作用。“百度文心一言新”主要强调了百度文心一言作为一种新的、先进的技术的出现；“智能对话引风潮”则描述了百度文心一言的智能对话功能引领了科技潮流，引发了广泛关注和讨论。\n"
     ]
    }
   ],
   "source": [
    "ERNIE_BOT.chat_stream('帮我解释一下前两句')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Gradio应用体验\n",
    "具体调用方式请见[quick_start](../quick_start)。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "ernie",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}