zwgao commited on
Commit
e3e0027
โ€ข
1 Parent(s): daf67f0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -130
README.md CHANGED
@@ -10,7 +10,9 @@ pinned: false
10
  license: apache-2.0
11
  ---
12
 
13
- <img src="https://github.com/liu-zhy/graph-of-thought/assets/26198430/4e8b2511-ce69-4c1a-95a1-5aed4d432a82" width=10% align="left" />
 
 
14
 
15
  # ControlLLM
16
 
@@ -21,19 +23,16 @@ We present ControlLLM, a novel framework that enables large language models (LLM
21
 
22
  ## ๐Ÿค– Video Demo
23
 
24
- <!-- <table>
25
- <tr>
26
- <td><img width="450" src="https://github.com/liu-zhy/graph-of-thought/assets/26198430/7fe7d1ec-e37e-4ea8-8201-dc639c82ba66" alt="Image 1"></td>
27
- <td><img width="450" src="https://github.com/liu-zhy/graph-of-thought/assets/26198430/a8bc6644-368b-42e3-844a-9962fdc9bd01" alt="Image 2"></td>
28
- </tr>
29
- </table>
30
- -->
31
 
32
- https://github.com/OpenGVLab/ControlLLM/assets/13723743/cf72861e-0e7b-4c15-89ee-7fa1d838d00f
 
 
 
33
 
34
  ## ๐Ÿ  System Overview
35
 
36
- ![arch](https://github.com/liu-zhy/graph-of-thought/assets/95175307/ad3db5c1-f1c7-4e1f-be48-81ed5228f2b0#center)
 
37
 
38
  ## ๐ŸŽ Major Features
39
  - Image Perception
@@ -52,126 +51,6 @@ https://github.com/OpenGVLab/ControlLLM/assets/13723743/cf72861e-0e7b-4c15-89ee-
52
 
53
  - [ ] Launch online demo
54
 
55
- ## ๐Ÿ› ๏ธInstallation
56
-
57
- ### Basic requirements
58
-
59
- * Linux
60
- * Python 3.10+
61
- * PyTorch 2.0+
62
- * CUDA 11.8+
63
-
64
- ### Clone project
65
-
66
- Execute the following command in the root directory:
67
-
68
- ```bash
69
- git clone https://github.com/OpenGVLab/ControlLLM.git
70
- ```
71
-
72
- ### Install dependencies
73
-
74
- Setup environment:
75
-
76
- ```bash
77
-
78
- conda create -n cllm python=3.10
79
-
80
- conda activate cllm
81
-
82
- conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.8 -c pytorch -c nvidia
83
- ```
84
-
85
- Install [LLaVA](https://github.com/haotian-liu/LLaVA?tab=readme-ov-file):
86
-
87
- ```bash
88
- pip install git+https://github.com/haotian-liu/LLaVA.git
89
- ```
90
-
91
- Then install other dependencies:
92
-
93
- ```bash
94
- cd controlllm
95
-
96
- pip install -r requirements.txt
97
- ```
98
-
99
- ## ๐Ÿ‘จโ€๐Ÿซ Get Started
100
-
101
- ### Launch tool services
102
-
103
- Please put your personal OpenAI Key and [Weather Key](https://www.visualcrossing.com/weather-api) into the corresponding environment variables.
104
- ```bash
105
-
106
- cd ./controlllm
107
- # openai key
108
- export OPENAI_API_KEY="..."
109
- # openai base
110
- export OPENAI_BASE_URL="..."
111
- # weather api key
112
- export WEATHER_API_KEY="..."
113
-
114
- python -m cllm.services.launch --port 10011 --host 0.0.0.0
115
- ```
116
-
117
- ### Launch ToG service
118
-
119
- ```bash
120
- cd ./controlllm
121
-
122
- export TOG_SERVICES_PORT=10011
123
- export OPENAI_BASE_URL="..."
124
- export OPENAI_API_KEY="..."
125
- python -m cllm.services.tog.launch --port 10012 --host 0.0.0.0
126
-
127
- ```
128
-
129
- ### Launch gradio demo
130
-
131
- Use `openssl` to generate the certificate:
132
- ```shell
133
- mkdir certificate
134
-
135
- openssl req -x509 -newkey rsa:4096 -keyout certificate/key.pem -out certificate/cert.pem -sha256 -days 365 -nodes
136
- ```
137
- Launch gradio demo:
138
- ```bash
139
- cd ./controlllm
140
-
141
- export TOG_PORT=10012
142
- export TOG_SERVICES_PORT=10011
143
- export RESOURCE_ROOT="./client_resources"
144
- export GRADIO_TEMP_DIR="$HOME/.tmp"
145
- export OPENAI_BASE_URL="..."
146
- export OPENAI_API_KEY="..."
147
-
148
- python -m cllm.app.gradio --controller "cllm.agents.tog.Controller" --server_port 10024
149
-
150
- ```
151
-
152
- ### Tools as Services
153
-
154
- Take image generation as an example, we first launch the service.
155
-
156
- ```bash
157
-
158
- python -m cllm.services.image_generation.launch --port 10011 --host 0.0.0.0
159
-
160
- ```
161
-
162
- Then, we call the services via python api.
163
-
164
- ```python
165
- from cllm.services.image_generation.api import *
166
- setup(port=10011)
167
- text2image('A horse')
168
- ```
169
-
170
- ๐Ÿ˜ฌ Launch all in one endpoint
171
-
172
- ```bash
173
- python -m cllm.services.launch --port 10011 --host 0.0.0.0
174
- ```
175
 
176
  ## ๐Ÿ› ๏ธ Support Tools
177
 
 
10
  license: apache-2.0
11
  ---
12
 
13
+
14
+ <img src="https://cdn-uploads.huggingface.co/production/uploads/64564b0e4a7ffb7d5a47f412/T8e6gFBYtSxo4qPE7cRwK.png" width=10% align="left" />
15
+
16
 
17
  # ControlLLM
18
 
 
23
 
24
  ## ๐Ÿค– Video Demo
25
 
 
 
 
 
 
 
 
26
 
27
+
28
+
29
+ <video controls autoplay src="https://cdn-uploads.huggingface.co/production/uploads/64564b0e4a7ffb7d5a47f412/vM9d412xQx2ajNkMew2ut.mp4"></video>
30
+
31
 
32
  ## ๐Ÿ  System Overview
33
 
34
+
35
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/64564b0e4a7ffb7d5a47f412/0LB4HVZlt02Qhg5xXLCop.png)
36
 
37
  ## ๐ŸŽ Major Features
38
  - Image Perception
 
51
 
52
  - [ ] Launch online demo
53
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
54
 
55
  ## ๐Ÿ› ๏ธ Support Tools
56