Spaces:
Running
Running
File size: 3,528 Bytes
ed4d993 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 |
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# HuggingGPT\n",
"Implementation of [HuggingGPT](https://github.com/microsoft/JARVIS). HuggingGPT is a system to connect LLMs (ChatGPT) with ML community (Hugging Face).\n",
"\n",
"+ 🔥 Paper: https://arxiv.org/abs/2303.17580\n",
"+ 🚀 Project: https://github.com/microsoft/JARVIS\n",
"+ 🤗 Space: https://huggingface.co/spaces/microsoft/HuggingGPT"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Set up tools\n",
"\n",
"We set up the tools available from [Transformers Agent](https://huggingface.co/docs/transformers/transformers_agents#tools). It includes a library of tools supported by Transformers and some customized tools such as image generator, video generator, text downloader and other tools."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from transformers import load_tool"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"hf_tools = [\n",
" load_tool(tool_name)\n",
" for tool_name in [\n",
" \"document-question-answering\",\n",
" \"image-captioning\",\n",
" \"image-question-answering\",\n",
" \"image-segmentation\",\n",
" \"speech-to-text\",\n",
" \"summarization\",\n",
" \"text-classification\",\n",
" \"text-question-answering\",\n",
" \"translation\",\n",
" \"huggingface-tools/text-to-image\",\n",
" \"huggingface-tools/text-to-video\",\n",
" \"text-to-speech\",\n",
" \"huggingface-tools/text-download\",\n",
" \"huggingface-tools/image-transformation\",\n",
" ]\n",
"]"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Setup model and HuggingGPT\n",
"\n",
"We create an instance of HuggingGPT and use ChatGPT as the controller to rule the above tools."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain_experimental.autonomous_agents import HuggingGPT\n",
"from langchain_openai import OpenAI\n",
"\n",
"# %env OPENAI_API_BASE=http://localhost:8000/v1"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"llm = OpenAI(model_name=\"gpt-3.5-turbo\")\n",
"agent = HuggingGPT(llm, hf_tools)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Run an example\n",
"\n",
"Given a text, show a related image and video."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"agent.run(\"please show me a video and an image of 'a boy is running'\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "langchain",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.17"
},
"orig_nbformat": 4
},
"nbformat": 4,
"nbformat_minor": 2
}
|