Package: llm-gpt4all
llm-gpt4allPlugin forLLMadding support for theGPT4Allcollection of models.InstallationInstall this plugin in the same environment as LLM.llminstallllm-gpt4allAfter installing the plugin you can see a new list of available models like this:llmmodelslistThe output will include something like this:gpt4all: orca-mini-3b-gguf2-q4_0 - Mini Orca (Small), 1.84GB download, needs 4GB RAM (installed)
gpt4all: nous-hermes-llama2-13b - Hermes, 6.86GB download, needs 16GB RAM (installed)
gpt4all: all-MiniLM-L6-v2-f16 - SBert, 43.76MB download, needs 1GB RAM
gpt4all: replit-code-v1_5-3b-q4_0 - Replit, 1.74GB download, needs 4GB RAM
gpt4all: mpt-7b-chat-merges-q4_0 - MPT Chat, 3.54GB download, needs 8GB RAM
gpt4all: rift-coder-v0-7b-q4_0 - Rift coder, 3.56GB download, needs 8GB RAM
gpt4all: em_german_mistral_v01 - EM German Mistral, 3.83GB download, needs 8GB RAM
gpt4all: mistral-7b-instruct-v0 - Mistral Instruct, 3.83GB download, needs 8GB RAM
gpt4all: mistral-7b-openorca - Mistral OpenOrca, 3.83GB download, needs 8GB RAM
gpt4all: gpt4all-falcon-q4_0 - GPT4All Falcon, 3.92GB download, needs 8GB RAM
gpt4all: gpt4all-13b-snoozy-q4_0 - Snoozy, 6.86GB download, needs 16GB RAM
gpt4all: wizardlm-13b-v1 - Wizard v1.2, 6.86GB download, needs 16GB RAM
gpt4all: starcoder-q4_0 - Starcoder, 8.37GB download, needs 4GB RAMFurther details on these models can be foundin this Observable notebook.UsageModel Installation and PromptingYou can execute a model using the name displayed in thellm models listoutput. The model file will be downloaded the first time you attempt to run it.llm-morca-mini-3b-gguf2-q4_0'3 names for a pet cow'The first time you run this you will see a progress bar for the model file download:31%|█████████▋                        | 1.16G/3.79G [00:26<01:02, 42.0MiB/s]On subsequent uses the model output will be displayed immediately.Note that the models will be downloaded to~/.cache/gpt4all.Model optionsRunllm models --optionsfor a list of available model options, which should include:gpt4all: mistral-7b-instruct-v0 - Mistral Instruct, 3.83GB download, needs 8GB RAM (installed)
  max_tokens: int
    The maximum number of tokens to generate.
  temp: float
    The model temperature. Larger values increase creativity but decrease
    factuality.
  top_k: int
    Randomly sample from the top_k most likely tokens at each generation
    step. Set this to 1 for greedy decoding.
  top_p: float
    Randomly sample at each generation step from the top most likely
    tokens whose probabilities add up to top_p.
  repeat_penalty: float
    Penalize the model for repetition. Higher values result in less
    repetition.
  repeat_last_n: int
    How far in the models generation history to apply the repeat penalty.
  n_batch: int
    Number of prompt tokens processed in parallel. Larger values decrease
    latency but increase resource requirements.Use them like this:llm-mmistral-7b-instruct-v0-omax_tokens2'hi'Hello!ChattingTo chat with a model, avoiding the need to load it into memory for every message, usellm chat:llmchat-morca-mini-3b-gguf2-q4_0Chatting with orca-mini-3b-gguf2-q4_0
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
> hi
 Hello! How can I assist you today?
> three jokes about a pelican and a chicken who are friends
 Sure, here are three jokes about a pelican and a chicken who are friends:

1. Why did the pelican cross the road? To get to the other side where the chicken was waiting for him!
2. What do you call a group of chickens playing basketball? A flock of feathers!
3. Why did the chicken invite the pelican over for dinner? Because it had nothing else to eat and needed some extra protein in its diet!Removing ModelsTo remove a downloaded model, delete the.gguffile from~/.cache/gpt4all.DevelopmentTo set up this plugin locally, first checkout the code. Then create a new virtual environment:cdllm-gpt4all
python3-mvenvvenvsourcevenv/bin/activateNow install the dependencies and test dependencies:pipinstall-e'.[test]'To run the tests:pytest