Package: grazier
Grazier: Easily call Large Language Models from a unified APIGrazier is a Python library for easily calling large language models from a unified API.Supported Large ModelsFrom OpenAI:GPT-4 (Base, 32K) (Chat and Completion Engines)GPT-3.5 (ChatGPT) (Chat and Completion Engines)GPT-3 (Davinci (v2,v3), Ada, Babbage, Curie) (Completion Engine)From Anthropic:Claude 2 (Base) (Chat and Completion Engines)Claude (Base, 100K) (Chat and Completion Engines)Claude Instant (Base, 100K) (Chat and Completion Engines)From Google/GCP:PaLM (Chat and Completion Engines)From HuggingfaceGPT-2 (Base, Medium, Large, XL) (Completion Engine)GPT-Neo (125M, 1.3B, 2.7B) (Completion Engine)GPT-J (6B) (Completion Engine)Falcon (7B, 40B, rw-1B, rw-7B) (Completion Engine)Dolly (v1 - 6B, v2 - 3B, 7B, 12B) (Chat and Completion Engines)MPT (Instruct - 7B, 30B) (Chat and Completion Engines)From Facebook (via Huggingface)Llama (7B, 13B, 30B, 65B) (Completion Engine)Llama 2 (7B, 13B, 70B) (Completion Engine)OPT (125M, 350M, 1.3B, 2.7B, 6.7B, 13B, 30B, 66B) (Completion Engine)From Stanford (via Huggingface)Alpaca (7B) (Chat and Completion Engines)From Berkeley (via Huggingface)Koala (7B, 13B_v1, 13B_v2) (Chat and Completion Engines)Vicuna (7B, 13B) (Chat and Completion Engines)From StabilityAI (via Huggingface)StableLM (7B, 13B) (Chat and Completion Engines)From AllenAI (via Huggingface)Tulu (7B, 13B, 30B, 65B) (Chat and Completion Engines)Open Instruct (ShareGPT) (7B, 13B, 30B, 65B) (Chat and Completion Engines)From AI21Jurassic 2 (Light, Mid, Ultra) (Completion Engines)Image/Language Models:Blip2 (2.7B, 6.7B) (Image + Text Engine)Open Flamingo (3B, 4B, 9B) (Image + Text Engine)InstallationGrazier can easily be installed using pip:pipinstallgrazierEach of the LLMs may need additional setup, which you can find in the engine setup section below.UsageFor completion engines, it's as simple as:importgraziergrazier.list_models()['gptj-6B','gpt2','gpt2-med','gpt2-lg','gpt2-xl','distilgpt2','gptneo-125M','gptneo-1.3B','gptneo-2.7B','stablelm-3B','stablelm-7B','opt-125M','opt-350M','opt-1.3b','opt-2.7b','opt-6.7b','opt-13b','opt-30b','opt-66b','llama-7B','llama-13B','llama-30B','llama-65B','gpt3-davinci3','gpt3-davinci2','gpt3-curie','gpt3-babbage','gpt3-ada','palm']gpt2=grazier.get("gpt2")completion=gpt2("I enjoy walking with my cute dog, but sometimes he gets scared and")print(completion)For chat engines, all you need to do is add thetype="chat"parameter:fromgrazierimportConversation,Speaker,get,list_modelsconversation=Conversation()conversation.add_turn("You are a funny person.",speaker=Speaker.SYSTEM)conversation.add_turn("Hi, how are you?",speaker=Speaker.USER)conversation.add_turn("I am doing well, how about you?",speaker=Speaker.AI)conversation.add_turn("What are you planning to do today?",speaker=Speaker.USER)list_models(type="chat")['claude','claude-100k','claude-instant','claude-instant-100k','bard','koala-7b','koala-13b-v1','koala-13b-v2','vicuna-7b','vicuna-13b','alpaca-13b','chat-gpt','gpt4','gpt4-32k','stablelm-3b','stablelm-7b','palm']gpt4=get("gpt4",type="chat")next_turn=gpt4(conversation)print(next_turn)For vision-augmented (image) engines, usetype="image"importgrazierfromPILimportImagegrazier.list_models(type="image")['blip2-opt-2.7b','blip2-opt-6.7b','blip2-opt-2.7b-coco','blip2-opt-6.7b-coco','blip2-flan-t5-xl','blip2-flan-t5-xxl','blip2-flan-t5-xl-coco','openflamingo-3b-vitl-mpt1b','openflamingo-3b-vitl-mpt1b-dolly','openflamingo-9b-vitl-mpt7b','openflamingo-4b-vitl-rpj3b']blip2=grazier.get("blip2-opt-2.7b",type="image")image=Image.open('test_data/dog.jpg')completion=blip2(image,"A photo of")print(completion)Individual Engine SetupEach engine may require some specific details to be passed in. For example, OpenAI engines require an API key. These
details are generally set up with environment variables.OpenAI EnginesFor OpenAI engines, you will need to set theOPENAI_API_KEYandOPENAI_API_ORGenvironment variables. You can find
your API key and organization ID on theOpenAI dashboard. You can set these environment
variables in your shell or in a.envfile in the root of your project. For example, in a.envfile, you would have:OPENAI_API_KEY=<yourkey>OPENAI_API_ORG=<yourorgid>or on the command line:exportOPENAI_API_KEY=<yourkey>exportOPENAI_API_ORG=<yourorgid>Anthropic EnginesFor Anthropic engines, you will need to set theANTHROPIC_API_KEYenvironment variable. You can find your API key at
theAnthropic dashboard. You can set this environment variable in
your shell or in a.envfile in the root of your project. For example, in a.envfile, you would have:ANTHROPIC_API_KEY=<yourkey>or on the command line:exportANTHROPIC_API_KEY=<yourkey>Vertex Engines (PaLM)For Google engines, we use the Vertex cloud API, which requires a Google Cloud Platform (GCP) project. You can create a
GCP project at theGCP console. You will also need to enable the Vertex AI API for
your project, set up a services account, and download the account JSON credentials. You can find instructions for that
following steps 1 to 6 of the tutorialhere.
Finally, you will need to set theGOOGLE_APPLICATION_CREDENTIALSenvironment variable to the path of the JSON file.
You can set this environment variable in your shell or in a.envfile in the root of your project. For example, in a.envfile, you would have:GOOGLE_APPLICATION_CREDENTIALS=<pathtoyourJSONfile>or on the command line:exportGOOGLE_APPLICATION_CREDENTIALS=<pathtoyourJSONfile>BardFor the Bard engine, you will need to get your Bard __Secure-1PSID and __Secure-1PSIDTS tokens.  Get the value of this
variable by first going tohttps://bard.google.com/, then log in, press F12 for console, and go to the "Application" tab,
then "Cookies", then copy the value of the "__Secure-1PSID" and "__Secure-1PSIDTS" cookies. You can then set the
environment variables:BARD__Secure_1PSID=<yoursessionid>BARD__Secure_1PSIDTS=<yoursessionidtimestamp>Huggingface EnginesMost of the huggingface engines require no additional setup, however, some of the larger models require a GPU to run
with any kind of efficiency (and some require multiple GPUs with large amounts of memory). You can find more details
about the requirements for each model on theHuggingface model hub.Llama, Alpaca, Koala, Vicuna and AllenAI EnginesFor these engines, you will need to obtain and postprocess the weights yourself (due to Facebook's licensing). You can
find the instructions for doing so on each model page:Llama:https://huggingface.co/docs/transformers/main/model_doc/llamaAlpaca:https://github.com/tatsu-lab/stanford_alpaca#recovering-alpaca-weightsKoala:https://github.com/young-geng/EasyLM/blob/main/docs/koala.mdVicuna:https://github.com/lm-sys/FastChat#vicuna-weightsAllenAI:https://huggingface.co/allenai/tulu-65bOnce the weights have been downloaded and processed, you can set the following environment variables to the root
directory containing a folder for each variant (The format is,{root_dir}/{model-prefix}/weights.bin, the root directory would
beroot_dir, and the model-prefix is the name of the model, e.g.tulu-65b):LLAMA_WEIGHTS_ROOT=<pathtothellamaweights>ALPACA_WEIGHTS_ROOT=<pathtothealpacaweights>KOALA_WEIGHTS_ROOT=<pathtothekoalaweights>VICUNA_WEIGHTS_ROOT=<pathtothevicunaweights>ALLENAI_WEIGHTS_ROOT=<pathtotheallenaiweights>AI21 Models (Jurassic)For AI21 models, you will need to set theAI21_API_KEYenvironment variable. You can find your API key at
theAI21 Studio Dashboard. You can set this environment variable in
your shell or in a.envfile in the root of your project. For example, in a.envfile, you would have:AI21_API_KEY=<yourkey>or on the command line:exportAI21_API_KEY=<yourkey>CitationIf you use grazier in your work, please cite:@misc{grazier,
  author = {David Chan},
  title = {grazier: Easily call Large Language Models from a unified API},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{
    https://github.com/DavidMChan/grazier
  }}
}Licensegrazier is licensed under the terms of the MIT license. SeeLICENSEfor more information.