Package: llm-replicate
llm-replicateLLMplugin for models hosted onReplicateInstallationFirst,install the LLM command-line utility.Now install this plugin in the same environment as LLM.llminstallllm-replicateConfigurationYou will need an API key from Replicate. You canobtain one here.You can set that as an environment variable calledREPLICATE_API_TOKEN, or add it to thellmset of saved keys using:llmkeyssetreplicateEnter key: <paste key here>To fetch and save details ofthe default collectionof language models hosted on Replicate, run this:llmreplicatefetch-modelsTo add specific models that aren't listed in that collection, use thellm replicate addcommand.For the Llama 2 model froma16z-infra/llama13b-v2-chatrun this:llmreplicateadda16z-infra/llama13b-v2-chat\--chat--aliasllama2The--chatflag indicates that this is a chat model, which means it will be able to work with-ccontinue mode.UsageTo run a prompt against a model, pass its name or an alias tollm -m:llm-mllama2"Ten great names for a pet pelican"Sure, here are ten great names for a pet pelican:PeltyPeanutPuddlesNibblesFuzzyGizmoHankLunaScooterSplishyI hope these suggestions help you find the perfect name for your pet pelican! Do you have any other questions?Chat models can support continuing conversations, for example:llm-c"Five more and make them more nautical"Ahoy matey! Here be five more nautical-themed names for yer pet pelican:Captain HookAnchorynSeadogPlunderPointe PelicanI hope these suggestions help ye find the perfect name for yer feathered friend! Do ye have any other questions, matey?Runllm models listto see the full list of models:llmmodelslistYou should see something like this:Replicate: replicate-flan-t5-xl
Replicate: replicate-llama-7b
Replicate: replicate-gpt-j-6b
Replicate: replicate-dolly-v2-12b
Replicate: replicate-oasst-sft-1-pythia-12b
Replicate: replicate-stability-ai-stablelm-tuned-alpha-7b
Replicate: replicate-vicuna-13b
Replicate: replicate-replit-code-v1-3b
Replicate: replicate-replit-replit-code-v1-3b
Replicate: replicate-joehoover-falcon-40b-instruct (aliases: falcon)
Replicate (chat): replicate-a16z-infra-llama13b-v2-chat (aliases: llama2)Then run a prompt through a specific model like this:llm-mreplicate-vicuna-13b"Five great names for a pet llama"Registering extra modelsTo register additional models that are not included in the defaultLanguage models collection, find their ID on Replicate and use thellm replicate addcommand.For example, to add thejoehoover/falcon-40b-instructmodel, run this:llmreplicateaddjoehoover/falcon-40b-instruct\--aliasfalconThis adds the model with the aliasfalcon- you can have 0 or more aliases for a model.Now you can run it like this:llm-mreplicate-joehoover-falcon-40b-instruct\"Three reasons to get a pet falcon"Or using the alias like this:llm-mfalcon"Three reasons to get a pet falcon"You can edit the list of models you have registered using the default$EDITORlike this:llmreplicateedit-modelsIf you register a model using the--chatoption that model will be treated slightly differently. Prompts sent to the model will be formatted like this:User: user input here
Assistant:If you use-cconversation modethe prompt will include previous messages in the conversation, like this:User: Ten great names for a pet pelican
Assistant: Sure, here are ten great names for a pet pelican:

1. Pelty
2. Peanut
3. Puddles
4. Nibbles
5. Fuzzy
6. Gizmo
7. Hank
8. Luna
9. Scooter
10. Splishy

I hope these suggestions help you find the perfect name for your pet pelican! Do you have any other questions?
User: Five more and make them more nautical
Assistant:DevelopmentTo set up this plugin locally, first checkout the code. Then create a new virtual environment:cd llm-palm
python3 -m venv venv
source venv/bin/activateNow install the dependencies and test dependencies:pip install -e '.[test]'To run the tests:pytest