Hugging Face's logo

Chang-Su
/
llama-2-13b-chat-ko-adapter

Model card Files Community
Chang-Su commited on
Commit
06ffc4a
·
1 Parent(s): b6d7904

step 16000

Browse files
Files changed (4) hide show
  1. adapter_config.json +1 -1
  2. adapter_model.bin +1 -1
  3. data_pointer.txt +1 -0
  4. notebooks/gradio_web_demo.ipynb +430 -0
adapter_config.json CHANGED
@@ -1,5 +1,5 @@
1
  {
2
- "base_model_name_or_path": "meta-llama/Llama-2-13b-chat-hf",
3
  "bias": "none",
4
  "enable_lora": null,
5
  "fan_in_fan_out": false,
 
1
  {
2
+ "base_model_name_or_path": "Chang-Su/llama-2-7b-chat-ko",
3
  "bias": "none",
4
  "enable_lora": null,
5
  "fan_in_fan_out": false,
adapter_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:dbfae777b6915012cab74a8b63907446249417418d01f89b890549a5f5415541
3
  size 871184829
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e4cc1a37c016bfb072cfddffbf5491757b040faf9b9e03eb6c59307068c8ed22
3
  size 871184829
data_pointer.txt ADDED
@@ -0,0 +1 @@
 
 
1
+ 16000
notebooks/gradio_web_demo.ipynb ADDED
@@ -0,0 +1,430 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "nbformat": 4,
3
+ "nbformat_minor": 0,
4
+ "metadata": {
5
+ "colab": {
6
+ "provenance": [],
7
+ "machine_shape": "hm",
8
+ "gpuType": "V100"
9
+ },
10
+ "kernelspec": {
11
+ "name": "python3",
12
+ "display_name": "Python 3"
13
+ },
14
+ "language_info": {
15
+ "name": "python"
16
+ },
17
+ "accelerator": "GPU"
18
+ },
19
+ "cells": [
20
+ {
21
+ "cell_type": "markdown",
22
+ "source": [
23
+ "## 通过Colab启动Web Demo\n",
24
+ "\n",
25
+ "本notebook提供了使用Colab中的GPU架设基于Gradio的demo示例。\n",
26
+ "\n",
27
+ "项目地址：https://github.com/ymcui/Chinese-LLaMA-Alpaca-2\n",
28
+ "\n",
29
+ "启动前准备：\n",
30
+ "- V100/A100：可直接加载7B模型\n",
31
+ "- T4：需要启用8bit加载（见以下启动说明）"
32
+ ],
33
+ "metadata": {
34
+ "id": "xidqUxwtsBnl"
35
+ }
36
+ },
37
+ {
38
+ "cell_type": "markdown",
39
+ "source": [
40
+ "### 克隆代码并安装依赖"
41
+ ],
42
+ "metadata": {
43
+ "id": "BNc29-zks3p5"
44
+ }
45
+ },
46
+ {
47
+ "cell_type": "code",
48
+ "execution_count": 1,
49
+ "metadata": {
50
+ "colab": {
51
+ "base_uri": "https://localhost:8080/"
52
+ },
53
+ "id": "zPqpnapOr70x",
54
+ "outputId": "e297adf1-b934-492d-ff97-4f1c2c8c738a"
55
+ },
56
+ "outputs": [
57
+ {
58
+ "output_type": "stream",
59
+ "name": "stdout",
60
+ "text": [
61
+ "Cloning into 'Chinese-LLaMA-Alpaca-2'...\n",
62
+ "remote: Enumerating objects: 242, done.\u001b[K\n",
63
+ "remote: Counting objects: 100% (84/84), done.\u001b[K\n",
64
+ "remote: Compressing objects: 100% (66/66), done.\u001b[K\n",
65
+ "remote: Total 242 (delta 37), reused 39 (delta 18), pack-reused 158\u001b[K\n",
66
+ "Receiving objects: 100% (242/242), 362.76 KiB | 3.42 MiB/s, done.\n",
67
+ "Resolving deltas: 100% (104/104), done.\n",
68
+ "Collecting git+https://github.com/huggingface/peft.git@13e53fc (from -r Chinese-LLaMA-Alpaca-2/requirements.txt (line 2))\n",
69
+ " Cloning https://github.com/huggingface/peft.git (to revision 13e53fc) to /tmp/pip-req-build-edd6aj5p\n",
70
+ " Running command git clone --filter=blob:none --quiet https://github.com/huggingface/peft.git /tmp/pip-req-build-edd6aj5p\n",
71
+ "\u001b[33m WARNING: Did not find branch or tag '13e53fc', assuming revision or ref.\u001b[0m\u001b[33m\n",
72
+ "\u001b[0m Running command git checkout -q 13e53fc\n",
73
+ " Resolved https://github.com/huggingface/peft.git to commit 13e53fc\n",
74
+ " Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
75
+ " Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
76
+ " Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
77
+ "Requirement already satisfied: torch==2.0.1 in /usr/local/lib/python3.10/dist-packages (from -r Chinese-LLaMA-Alpaca-2/requirements.txt (line 1)) (2.0.1+cu118)\n",
78
+ "Collecting transformers==4.31.0 (from -r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3))\n",
79
+ " Downloading transformers-4.31.0-py3-none-any.whl (7.4 MB)\n",
80
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.4/7.4 MB\u001b[0m \u001b[31m30.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
81
+ "\u001b[?25hCollecting sentencepiece==0.1.97 (from -r Chinese-LLaMA-Alpaca-2/requirements.txt (line 4))\n",
82
+ " Downloading sentencepiece-0.1.97-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n",
83
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m57.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
84
+ "\u001b[?25hCollecting bitsandbytes==0.39.1 (from -r Chinese-LLaMA-Alpaca-2/requirements.txt (line 5))\n",
85
+ " Downloading bitsandbytes-0.39.1-py3-none-any.whl (97.1 MB)\n",
86
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m97.1/97.1 MB\u001b[0m \u001b[31m11.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
87
+ "\u001b[?25hRequirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 1)) (3.12.2)\n",
88
+ "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 1)) (4.7.1)\n",
89
+ "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 1)) (1.11.1)\n",
90
+ "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 1)) (3.1)\n",
91
+ "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 1)) (3.1.2)\n",
92
+ "Requirement already satisfied: triton==2.0.0 in /usr/local/lib/python3.10/dist-packages (from torch==2.0.1->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 1)) (2.0.0)\n",
93
+ "Collecting huggingface-hub<1.0,>=0.14.1 (from transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3))\n",
94
+ " Downloading huggingface_hub-0.16.4-py3-none-any.whl (268 kB)\n",
95
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m268.8/268.8 kB\u001b[0m \u001b[31m31.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
96
+ "\u001b[?25hRequirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3)) (1.22.4)\n",
97
+ "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3)) (23.1)\n",
98
+ "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3)) (6.0.1)\n",
99
+ "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3)) (2022.10.31)\n",
100
+ "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3)) (2.27.1)\n",
101
+ "Collecting tokenizers!=0.11.3,<0.14,>=0.11.1 (from transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3))\n",
102
+ " Downloading tokenizers-0.13.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)\n",
103
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.8/7.8 MB\u001b[0m \u001b[31m120.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
104
+ "\u001b[?25hCollecting safetensors>=0.3.1 (from transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3))\n",
105
+ " Downloading safetensors-0.3.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n",
106
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m81.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
107
+ "\u001b[?25hRequirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3)) (4.65.0)\n",
108
+ "Requirement already satisfied: cmake in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch==2.0.1->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 1)) (3.25.2)\n",
109
+ "Requirement already satisfied: lit in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch==2.0.1->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 1)) (16.0.6)\n",
110
+ "Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from peft==0.3.0.dev0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 2)) (5.9.5)\n",
111
+ "Collecting accelerate (from peft==0.3.0.dev0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 2))\n",
112
+ " Downloading accelerate-0.21.0-py3-none-any.whl (244 kB)\n",
113
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m244.2/244.2 kB\u001b[0m \u001b[31m30.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
114
+ "\u001b[?25hRequirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.14.1->transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3)) (2023.6.0)\n",
115
+ "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch==2.0.1->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 1)) (2.1.3)\n",
116
+ "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3)) (1.26.16)\n",
117
+ "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3)) (2023.7.22)\n",
118
+ "Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3)) (2.0.12)\n",
119
+ "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.31.0->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 3)) (3.4)\n",
120
+ "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch==2.0.1->-r Chinese-LLaMA-Alpaca-2/requirements.txt (line 1)) (1.3.0)\n",
121
+ "Building wheels for collected packages: peft\n",
122
+ " Building wheel for peft (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
123
+ " Created wheel for peft: filename=peft-0.3.0.dev0-py3-none-any.whl size=40651 sha256=b925814e35e3f918ed8cf768c39e74046f9687af061d572ba75f446b8cdb631d\n",
124
+ " Stored in directory: /tmp/pip-ephem-wheel-cache-oj3_2rx9/wheels/d9/13/c6/404d5f8a81c5620f65f7fd75b6a66619f013cd79c2875b981c\n",
125
+ "Successfully built peft\n",
126
+ "Installing collected packages: tokenizers, sentencepiece, safetensors, bitsandbytes, huggingface-hub, transformers, accelerate, peft\n",
127
+ "Successfully installed accelerate-0.21.0 bitsandbytes-0.39.1 huggingface-hub-0.16.4 peft-0.3.0.dev0 safetensors-0.3.1 sentencepiece-0.1.97 tokenizers-0.13.3 transformers-4.31.0\n",
128
+ "Collecting gradio\n",
129
+ " Downloading gradio-3.39.0-py3-none-any.whl (19.9 MB)\n",
130
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m19.9/19.9 MB\u001b[0m \u001b[31m72.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
131
+ "\u001b[?25hCollecting aiofiles<24.0,>=22.0 (from gradio)\n",
132
+ " Downloading aiofiles-23.1.0-py3-none-any.whl (14 kB)\n",
133
+ "Requirement already satisfied: aiohttp~=3.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (3.8.5)\n",
134
+ "Requirement already satisfied: altair<6.0,>=4.2.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (4.2.2)\n",
135
+ "Collecting fastapi (from gradio)\n",
136
+ " Downloading fastapi-0.100.1-py3-none-any.whl (65 kB)\n",
137
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m65.8/65.8 kB\u001b[0m \u001b[31m7.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
138
+ "\u001b[?25hCollecting ffmpy (from gradio)\n",
139
+ " Downloading ffmpy-0.3.1.tar.gz (5.5 kB)\n",
140
+ " Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
141
+ "Collecting gradio-client>=0.3.0 (from gradio)\n",
142
+ " Downloading gradio_client-0.3.0-py3-none-any.whl (294 kB)\n",
143
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m294.2/294.2 kB\u001b[0m \u001b[31m31.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
144
+ "\u001b[?25hCollecting httpx (from gradio)\n",
145
+ " Downloading httpx-0.24.1-py3-none-any.whl (75 kB)\n",
146
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.4/75.4 kB\u001b[0m \u001b[31m10.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
147
+ "\u001b[?25hRequirement already satisfied: huggingface-hub>=0.14.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (0.16.4)\n",
148
+ "Requirement already satisfied: jinja2<4.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (3.1.2)\n",
149
+ "Requirement already satisfied: markdown-it-py[linkify]>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (3.0.0)\n",
150
+ "Requirement already satisfied: markupsafe~=2.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (2.1.3)\n",
151
+ "Requirement already satisfied: matplotlib~=3.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (3.7.1)\n",
152
+ "Collecting mdit-py-plugins<=0.3.3 (from gradio)\n",
153
+ " Downloading mdit_py_plugins-0.3.3-py3-none-any.whl (50 kB)\n",
154
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m50.5/50.5 kB\u001b[0m \u001b[31m6.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
155
+ "\u001b[?25hRequirement already satisfied: numpy~=1.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (1.22.4)\n",
156
+ "Collecting orjson~=3.0 (from gradio)\n",
157
+ " Downloading orjson-3.9.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)\n",
158
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━���━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m138.7/138.7 kB\u001b[0m \u001b[31m16.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
159
+ "\u001b[?25hRequirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from gradio) (23.1)\n",
160
+ "Requirement already satisfied: pandas<3.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (1.5.3)\n",
161
+ "Requirement already satisfied: pillow<11.0,>=8.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (9.4.0)\n",
162
+ "Requirement already satisfied: pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,<3.0.0,>=1.7.4 in /usr/local/lib/python3.10/dist-packages (from gradio) (1.10.12)\n",
163
+ "Collecting pydub (from gradio)\n",
164
+ " Downloading pydub-0.25.1-py2.py3-none-any.whl (32 kB)\n",
165
+ "Collecting python-multipart (from gradio)\n",
166
+ " Downloading python_multipart-0.0.6-py3-none-any.whl (45 kB)\n",
167
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m45.7/45.7 kB\u001b[0m \u001b[31m5.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
168
+ "\u001b[?25hRequirement already satisfied: pyyaml<7.0,>=5.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (6.0.1)\n",
169
+ "Requirement already satisfied: requests~=2.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (2.27.1)\n",
170
+ "Collecting semantic-version~=2.0 (from gradio)\n",
171
+ " Downloading semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)\n",
172
+ "Requirement already satisfied: typing-extensions~=4.0 in /usr/local/lib/python3.10/dist-packages (from gradio) (4.7.1)\n",
173
+ "Collecting uvicorn>=0.14.0 (from gradio)\n",
174
+ " Downloading uvicorn-0.23.2-py3-none-any.whl (59 kB)\n",
175
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m59.5/59.5 kB\u001b[0m \u001b[31m7.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
176
+ "\u001b[?25hCollecting websockets<12.0,>=10.0 (from gradio)\n",
177
+ " Downloading websockets-11.0.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (129 kB)\n",
178
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m129.9/129.9 kB\u001b[0m \u001b[31m14.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
179
+ "\u001b[?25hRequirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp~=3.0->gradio) (23.1.0)\n",
180
+ "Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp~=3.0->gradio) (2.0.12)\n",
181
+ "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp~=3.0->gradio) (6.0.4)\n",
182
+ "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.10/dist-packages (from aiohttp~=3.0->gradio) (4.0.2)\n",
183
+ "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp~=3.0->gradio) (1.9.2)\n",
184
+ "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp~=3.0->gradio) (1.4.0)\n",
185
+ "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp~=3.0->gradio) (1.3.1)\n",
186
+ "Requirement already satisfied: entrypoints in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio) (0.4)\n",
187
+ "Requirement already satisfied: jsonschema>=3.0 in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio) (4.3.3)\n",
188
+ "Requirement already satisfied: toolz in /usr/local/lib/python3.10/dist-packages (from altair<6.0,>=4.2.0->gradio) (0.12.0)\n",
189
+ "Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from gradio-client>=0.3.0->gradio) (2023.6.0)\n",
190
+ "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.14.0->gradio) (3.12.2)\n",
191
+ "Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.14.0->gradio) (4.65.0)\n",
192
+ "Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py[linkify]>=2.0.0->gradio) (0.1.2)\n",
193
+ "Requirement already satisfied: linkify-it-py<3,>=1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py[linkify]>=2.0.0->gradio) (2.0.2)\n",
194
+ "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio) (1.1.0)\n",
195
+ "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio) (0.11.0)\n",
196
+ "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio) (4.41.1)\n",
197
+ "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio) (1.4.4)\n",
198
+ "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio) (3.1.0)\n",
199
+ "Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.10/dist-packages (from matplotlib~=3.0->gradio) (2.8.2)\n",
200
+ "INFO: pip is looking at multiple versions of mdit-py-plugins to determine which version is compatible with other requirements. This could take a while.\n",
201
+ "Collecting mdit-py-plugins<=0.3.3 (from gradio)\n",
202
+ " Downloading mdit_py_plugins-0.3.2-py3-none-any.whl (50 kB)\n",
203
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m50.4/50.4 kB\u001b[0m \u001b[31m6.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
204
+ "\u001b[?25h Downloading mdit_py_plugins-0.3.1-py3-none-any.whl (46 kB)\n",
205
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m46.5/46.5 kB\u001b[0m \u001b[31m6.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
206
+ "\u001b[?25h Downloading mdit_py_plugins-0.3.0-py3-none-any.whl (43 kB)\n",
207
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.7/43.7 kB\u001b[0m \u001b[31m5.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
208
+ "\u001b[?25h Downloading mdit_py_plugins-0.2.8-py3-none-any.whl (41 kB)\n",
209
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m41.0/41.0 kB\u001b[0m \u001b[31m4.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
210
+ "\u001b[?25h Downloading mdit_py_plugins-0.2.7-py3-none-any.whl (41 kB)\n",
211
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m41.0/41.0 kB\u001b[0m \u001b[31m5.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
212
+ "\u001b[?25h Downloading mdit_py_plugins-0.2.6-py3-none-any.whl (39 kB)\n",
213
+ " Downloading mdit_py_plugins-0.2.5-py3-none-any.whl (39 kB)\n",
214
+ "INFO: pip is looking at multiple versions of mdit-py-plugins to determine which version is compatible with other requirements. This could take a while.\n",
215
+ " Downloading mdit_py_plugins-0.2.4-py3-none-any.whl (39 kB)\n",
216
+ " Downloading mdit_py_plugins-0.2.3-py3-none-any.whl (39 kB)\n",
217
+ " Downloading mdit_py_plugins-0.2.2-py3-none-any.whl (39 kB)\n",
218
+ " Downloading mdit_py_plugins-0.2.1-py3-none-any.whl (38 kB)\n",
219
+ " Downloading mdit_py_plugins-0.2.0-py3-none-any.whl (38 kB)\n",
220
+ "INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.\n",
221
+ " Downloading mdit_py_plugins-0.1.0-py3-none-any.whl (37 kB)\n",
222
+ "Collecting markdown-it-py[linkify]>=2.0.0 (from gradio)\n",
223
+ " Downloading markdown_it_py-3.0.0-py3-none-any.whl (87 kB)\n",
224
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m87.5/87.5 kB\u001b[0m \u001b[31m11.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
225
+ "\u001b[?25h Downloading markdown_it_py-2.2.0-py3-none-any.whl (84 kB)\n",
226
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m84.5/84.5 kB\u001b[0m \u001b[31m11.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
227
+ "\u001b[?25hRequirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas<3.0,>=1.0->gradio) (2022.7.1)\n",
228
+ "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests~=2.0->gradio) (1.26.16)\n",
229
+ "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests~=2.0->gradio) (2023.7.22)\n",
230
+ "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests~=2.0->gradio) (3.4)\n",
231
+ "Requirement already satisfied: click>=7.0 in /usr/local/lib/python3.10/dist-packages (from uvicorn>=0.14.0->gradio) (8.1.6)\n",
232
+ "Collecting h11>=0.8 (from uvicorn>=0.14.0->gradio)\n",
233
+ " Downloading h11-0.14.0-py3-none-any.whl (58 kB)\n",
234
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m7.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
235
+ "\u001b[?25hCollecting starlette<0.28.0,>=0.27.0 (from fastapi->gradio)\n",
236
+ " Downloading starlette-0.27.0-py3-none-any.whl (66 kB)\n",
237
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m67.0/67.0 kB\u001b[0m \u001b[31m8.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
238
+ "\u001b[?25hCollecting httpcore<0.18.0,>=0.15.0 (from httpx->gradio)\n",
239
+ " Downloading httpcore-0.17.3-py3-none-any.whl (74 kB)\n",
240
+ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m74.5/74.5 kB\u001b[0m \u001b[31m9.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
241
+ "\u001b[?25hRequirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx->gradio) (1.3.0)\n",
242
+ "Requirement already satisfied: anyio<5.0,>=3.0 in /usr/local/lib/python3.10/dist-packages (from httpcore<0.18.0,>=0.15.0->httpx->gradio) (3.7.1)\n",
243
+ "Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair<6.0,>=4.2.0->gradio) (0.19.3)\n",
244
+ "Requirement already satisfied: uc-micro-py in /usr/local/lib/python3.10/dist-packages (from linkify-it-py<3,>=1->markdown-it-py[linkify]>=2.0.0->gradio) (1.0.2)\n",
245
+ "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.7->matplotlib~=3.0->gradio) (1.16.0)\n",
246
+ "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5.0,>=3.0->httpcore<0.18.0,>=0.15.0->httpx->gradio) (1.1.2)\n",
247
+ "Building wheels for collected packages: ffmpy\n",
248
+ " Building wheel for ffmpy (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
249
+ " Created wheel for ffmpy: filename=ffmpy-0.3.1-py3-none-any.whl size=5579 sha256=57ade8007aa0450b3a03c935d39c4bf930a2cd6896c0e9d5c0687136d78ccfcc\n",
250
+ " Stored in directory: /root/.cache/pip/wheels/01/a6/d1/1c0828c304a4283b2c1639a09ad86f83d7c487ef34c6b4a1bf\n",
251
+ "Successfully built ffmpy\n",
252
+ "Installing collected packages: pydub, ffmpy, websockets, semantic-version, python-multipart, orjson, markdown-it-py, h11, aiofiles, uvicorn, starlette, mdit-py-plugins, httpcore, httpx, fastapi, gradio-client, gradio\n",
253
+ " Attempting uninstall: markdown-it-py\n",
254
+ " Found existing installation: markdown-it-py 3.0.0\n",
255
+ " Uninstalling markdown-it-py-3.0.0:\n",
256
+ " Successfully uninstalled markdown-it-py-3.0.0\n",
257
+ " Attempting uninstall: mdit-py-plugins\n",
258
+ " Found existing installation: mdit-py-plugins 0.4.0\n",
259
+ " Uninstalling mdit-py-plugins-0.4.0:\n",
260
+ " Successfully uninstalled mdit-py-plugins-0.4.0\n",
261
+ "Successfully installed aiofiles-23.1.0 fastapi-0.100.1 ffmpy-0.3.1 gradio-3.39.0 gradio-client-0.3.0 h11-0.14.0 httpcore-0.17.3 httpx-0.24.1 markdown-it-py-2.2.0 mdit-py-plugins-0.3.3 orjson-3.9.2 pydub-0.25.1 python-multipart-0.0.6 semantic-version-2.10.0 starlette-0.27.0 uvicorn-0.23.2 websockets-11.0.3\n"
262
+ ]
263
+ }
264
+ ],
265
+ "source": [
266
+ "!git clone https://github.com/ymcui/Chinese-LLaMA-Alpaca-2.git\n",
267
+ "!pip install -r Chinese-LLaMA-Alpaca-2/requirements.txt\n",
268
+ "!pip install gradio"
269
+ ]
270
+ },
271
+ {
272
+ "cell_type": "markdown",
273
+ "source": [
274
+ "### 下载Alpaca-2-7B完整模型"
275
+ ],
276
+ "metadata": {
277
+ "id": "aqD4OUDBtaKM"
278
+ }
279
+ },
280
+ {
281
+ "cell_type": "code",
282
+ "source": [
283
+ "!git clone https://huggingface.co/ziqingyang/chinese-alpaca-2-7b"
284
+ ],
285
+ "metadata": {
286
+ "colab": {
287
+ "base_uri": "https://localhost:8080/"
288
+ },
289
+ "id": "4R7_FhEUtfFq",
290
+ "outputId": "321d12ae-9a56-47bc-8d0e-c2adc35e3c1d"
291
+ },
292
+ "execution_count": 2,
293
+ "outputs": [
294
+ {
295
+ "output_type": "stream",
296
+ "name": "stdout",
297
+ "text": [
298
+ "Cloning into 'chinese-alpaca-2-7b'...\n",
299
+ "remote: Enumerating objects: 18, done.\u001b[K\n",
300
+ "remote: Counting objects: 5% (1/18)\u001b[K\rremote: Counting objects: 11% (2/18)\u001b[K\rremote: Counting objects: 16% (3/18)\u001b[K\rremote: Counting objects: 22% (4/18)\u001b[K\rremote: Counting objects: 27% (5/18)\u001b[K\rremote: Counting objects: 33% (6/18)\u001b[K\rremote: Counting objects: 38% (7/18)\u001b[K\rremote: Counting objects: 44% (8/18)\u001b[K\rremote: Counting objects: 50% (9/18)\u001b[K\rremote: Counting objects: 55% (10/18)\u001b[K\rremote: Counting objects: 61% (11/18)\u001b[K\rremote: Counting objects: 66% (12/18)\u001b[K\rremote: Counting objects: 72% (13/18)\u001b[K\rremote: Counting objects: 77% (14/18)\u001b[K\rremote: Counting objects: 83% (15/18)\u001b[K\rremote: Counting objects: 88% (16/18)\u001b[K\rremote: Counting objects: 94% (17/18)\u001b[K\rremote: Counting objects: 100% (18/18)\u001b[K\rremote: Counting objects: 100% (18/18), done.\u001b[K\n",
301
+ "remote: Compressing objects: 5% (1/17)\u001b[K\rremote: Compressing objects: 11% (2/17)\u001b[K\rremote: Compressing objects: 17% (3/17)\u001b[K\rremote: Compressing objects: 23% (4/17)\u001b[K\rremote: Compressing objects: 29% (5/17)\u001b[K\rremote: Compressing objects: 35% (6/17)\u001b[K\rremote: Compressing objects: 41% (7/17)\u001b[K\rremote: Compressing objects: 47% (8/17)\u001b[K\rremote: Compressing objects: 52% (9/17)\u001b[K\rremote: Compressing objects: 58% (10/17)\u001b[K\rremote: Compressing objects: 64% (11/17)\u001b[K\rremote: Compressing objects: 70% (12/17)\u001b[K\rremote: Compressing objects: 76% (13/17)\u001b[K\rremote: Compressing objects: 82% (14/17)\u001b[K\rremote: Compressing objects: 88% (15/17)\u001b[K\rremote: Compressing objects: 94% (16/17)\u001b[K\rremote: Compressing objects: 100% (17/17)\u001b[K\rremote: Compressing objects: 100% (17/17), done.\u001b[K\n",
302
+ "remote: Total 18 (delta 2), reused 0 (delta 0), pack-reused 0\u001b[K\n",
303
+ "Unpacking objects: 100% (18/18), 4.58 KiB | 1.14 MiB/s, done.\n",
304
+ "Filtering content: 100% (3/3), 4.90 GiB | 24.03 MiB/s, done.\n",
305
+ "Encountered 1 file(s) that may not have been copied correctly on Windows:\n",
306
+ "\tpytorch_model-00001-of-00002.bin\n",
307
+ "\n",
308
+ "See: `git lfs help smudge` for more details.\n"
309
+ ]
310
+ }
311
+ ]
312
+ },
313
+ {
314
+ "cell_type": "markdown",
315
+ "source": [
316
+ "### 启动Demo\n"
317
+ ],
318
+ "metadata": {
319
+ "id": "EZCv3HS3tohu"
320
+ }
321
+ },
322
+ {
323
+ "cell_type": "code",
324
+ "source": [
325
+ "!python Chinese-LLaMA-Alpaca-2/scripts/inference/gradio_demo.py --base_model /content/chinese-alpaca-2-7b --load_in_8bit"
326
+ ],
327
+ "metadata": {
328
+ "colab": {
329
+ "base_uri": "https://localhost:8080/"
330
+ },
331
+ "id": "l9lURhKSuJVM",
332
+ "outputId": "ef5ea01e-e5bb-4ceb-dcc3-4af5e4083027"
333
+ },
334
+ "execution_count": 23,
335
+ "outputs": [
336
+ {
337
+ "output_type": "stream",
338
+ "name": "stdout",
339
+ "text": [
340
+ "\n",
341
+ "===================================BUG REPORT===================================\n",
342
+ "Welcome to bitsandbytes. For bug reports, please run\n",
343
+ "\n",
344
+ "python -m bitsandbytes\n",
345
+ "\n",
346
+ " and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues\n",
347
+ "================================================================================\n",
348
+ "bin /usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda118_nocublaslt.so\n",
349
+ "/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: /usr/lib64-nvidia did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...\n",
350
+ " warn(msg)\n",
351
+ "/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/sys/fs/cgroup/memory.events /var/colab/cgroup/jupyter-children/memory.events')}\n",
352
+ " warn(msg)\n",
353
+ "/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('8013'), PosixPath('http'), PosixPath('//172.28.0.1')}\n",
354
+ " warn(msg)\n",
355
+ "/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('//colab.research.google.com/tun/m/cc48301118ce562b961b3c22d803539adc1e0c19/gpu-v100-hm-2ecbo83kb4j2y --tunnel_background_save_delay=10s --tunnel_periodic_background_save_frequency=30m0s --enable_output_coalescing=true --output_coalescing_required=true'), PosixPath('--logtostderr --listen_host=172.28.0.12 --target_host=172.28.0.12 --tunnel_background_save_url=https')}\n",
356
+ " warn(msg)\n",
357
+ "/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/env/python')}\n",
358
+ " warn(msg)\n",
359
+ "/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('//ipykernel.pylab.backend_inline'), PosixPath('module')}\n",
360
+ " warn(msg)\n",
361
+ "CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...\n",
362
+ "/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: Found duplicate ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] files: {PosixPath('/usr/local/cuda/lib64/libcudart.so.11.0'), PosixPath('/usr/local/cuda/lib64/libcudart.so')}.. We'll flip a coin and try one of these, in order to fail forward.\n",
363
+ "Either way, this might cause trouble in the future:\n",
364
+ "If you get `CUDA error: invalid device function` errors, the above might be the cause and the solution is to make sure only one ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] in the paths that we search based on your env.\n",
365
+ " warn(msg)\n",
366
+ "CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so.11.0\n",
367
+ "CUDA SETUP: Highest compute capability among GPUs detected: 7.0\n",
368
+ "CUDA SETUP: Detected CUDA version 118\n",
369
+ "/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU!\n",
370
+ " warn(msg)\n",
371
+ "CUDA SETUP: Loading binary /usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda118_nocublaslt.so...\n",
372
+ "2023-08-02 06:10:09.023871: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT\n",
373
+ "USE_MEM_EFF_ATTENTION: True\n",
374
+ "STORE_KV_BEFORE_ROPE: False\n",
375
+ "Apply NTK scaling with ALPHA=1.0\n",
376
+ "You are using the legacy behaviour of the <class 'transformers.models.llama.tokenization_llama.LlamaTokenizer'>. This means that tokens that come after special tokens will not be properly handled. We recommend you to read the related pull request available at https://github.com/huggingface/transformers/pull/24565\n",
377
+ "Loading checkpoint shards: 100% 2/2 [01:08<00:00, 34.32s/it]\n",
378
+ "Vocab of the base model: 55296\n",
379
+ "Vocab of the tokenizer: 55296\n",
380
+ "/content/Chinese-LLaMA-Alpaca-2/scripts/inference/gradio_demo.py:444: GradioDeprecationWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.\n",
381
+ " user_input = gr.Textbox(\n",
382
+ "Running on local URL: http://0.0.0.0:19324\n",
383
+ "Running on public URL: https://70d599ba8bb44ae10a.gradio.live\n",
384
+ "\n",
385
+ "This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from Terminal to deploy to Spaces (https://huggingface.co/spaces)\n",
386
+ "Keyboard interruption in main thread... closing server.\n",
387
+ "Traceback (most recent call last):\n",
388
+ " File \"/usr/local/lib/python3.10/dist-packages/gradio/blocks.py\", line 2133, in block_thread\n",
389
+ " time.sleep(0.1)\n",
390
+ "KeyboardInterrupt\n",
391
+ "\n",
392
+ "During handling of the above exception, another exception occurred:\n",
393
+ "\n",
394
+ "Traceback (most recent call last):\n",
395
+ " File \"/content/Chinese-LLaMA-Alpaca-2/scripts/inference/gradio_demo.py\", line 538, in <module>\n",
396
+ " demo.queue().launch(\n",
397
+ " File \"/usr/local/lib/python3.10/dist-packages/gradio/blocks.py\", line 2049, in launch\n",
398
+ " self.block_thread()\n",
399
+ " File \"/usr/local/lib/python3.10/dist-packages/gradio/blocks.py\", line 2137, in block_thread\n",
400
+ " self.server.close()\n",
401
+ " File \"/usr/local/lib/python3.10/dist-packages/gradio/networking.py\", line 49, in close\n",
402
+ " self.thread.join()\n",
403
+ " File \"/usr/lib/python3.10/threading.py\", line 1096, in join\n",
404
+ " self._wait_for_tstate_lock()\n",
405
+ " File \"/usr/lib/python3.10/threading.py\", line 1116, in _wait_for_tstate_lock\n",
406
+ " if lock.acquire(block, timeout):\n",
407
+ "KeyboardInterrupt\n",
408
+ "Killing tunnel 0.0.0.0:19324 <> https://70d599ba8bb44ae10a.gradio.live\n",
409
+ "^C\n"
410
+ ]
411
+ }
412
+ ]
413
+ },
414
+ {
415
+ "cell_type": "markdown",
416
+ "source": [
417
+ "启动之后，日志中的\n",
418
+ "```\n",
419
+ "Running on public URL: https://70d599ba8bb44ae10a.gradio.live\n",
420
+ "```\n",
421
+ "就是可以在线体验的demo链接（每次都不一样）。用完记得及时关闭和删除运行时，以免不必要的浪费。\n",
422
+ "\n",
423
+ "\n"
424
+ ],
425
+ "metadata": {
426
+ "id": "S4BLss6S0AuD"
427
+ }
428
+ }
429
+ ]
430
+ }