Package: openplayground
openplaygroundAn LLM playground you can run on your laptop.https://user-images.githubusercontent.com/111631/227399583-39b23f48-9823-4571-a906-985dbe282b20.mp4FeaturesUse any model fromOpenAI,Anthropic,Cohere,Forefront,HuggingFace,Aleph Alpha, andllama.cpp.Full playground UI, including history, parameter tuning, keyboard shortcuts, and logprops.Compare models side-by-side with the same prompt, individually tune model parameters, and retry with different parameters.Automatically detects local models in your HuggingFace cache, and lets you install new ones.Works OK on your phone.Probably won't kill everyone.Try on nat.devTry the hosted version:nat.dev.How to install and run$pipinstallopenplayground
$openplaygroundrunAlternatively, run it as a docker container:$dockerrun--nameopenplayground-p5432:5432-d--volumeopenplayground:/web/confignatorg/openplaygroundThis runs a Flask process, so you can add the typical flags such as setting a different portopenplayground run -p 1235and others.How to run for development$gitclonehttps://github.com/nat/openplayground
$cdapp&&npminstall&&npxparcelwatchsrc/index.html--no-cache
$cdserver&&pip3install-rrequirements.txt&&cd..&&python3-mserver.appDocker$dockerbuild.--tag"openplayground"$dockerrun--nameopenplayground-p5432:5432-d--volumeopenplayground:/web/configopenplaygroundFirst volume is optional. It's used to store API keys, models settings.Ideas for contributionsAdd a token counter to the playgroundAdd a cost counter to the playground and the compare pageMeasure and display time to first tokenSetup automatic builds with GitHub ActionsThe default parameters for each model are configured in theserver/models.jsonfile. If you find better default parameters for a model, please submit a pull request!Someone can help us make a homebrew package, and a dockerfileEasier way to install open source models directly from openplayground, withopenplayground install <model>or in the UI.Find and fix bugsChatGPT UI, with turn-by-turn, markdown rendering, chatgpt plugin support, etc.We will probably need multimodal inputs and outputs at some point in 2023llama.cppAdding models to openplaygroundModels and providers have three types in openplayground:SearchableLocal inferenceAPIYou can add models inserver/models.jsonwith the following schema:Local inferenceFor models running locally on your device you can add them to openplayground like the following (a minimal example):"llama":{"api_key":false,"models":{"llama-70b":{"parameters":{"temperature":{"value":0.5,"range":[0.1,1.0]},}}}}Keep in mind you will need to add a generation method for your model inserver/app.py. Take a look atlocal_text_generation()as an example.API Provider InferenceThis is for model providers like OpenAI, cohere, forefront, and more. You can connect them easily into openplayground (a minimal example):"cohere":{"api_key":true,"models":{"xlarge":{"parameters":{"temperature":{"value":0.5,"range":[0.1,1.0]},}}}}Keep in mind you will need to add a generation method for your model inserver/app.py. Take a look atopenai_text_generation()orcohere_text_generation()as an example.Searchable modelsWe use this for Huggingface Remote Inference models, the search endpoint is useful for scaling to N models in the settings page."provider_name":{"api_key":true,"search":{"endpoint":"ENDPOINT_URL"},"parameters":{"parameter":{"value":1.0,"range":[0.1,1.0]},}}CreditsInstigated by Nat Friedman. Initial implementation byZain Hudaas a repl.it bounty. Many features and extensive refactoring by Alex Lourenco.