Package: easyllm
EasyLLM -EasyLLMis an open source project that provideshelpful tools and methods for working with large language models(LLMs), both open source and closed source. Get immediataly started or check out thedocumentation.EasyLLM implements clients that arecompatible with OpenAI's Completion API. This means you can easily replaceopenai.ChatCompletion,openai.Completion,openai.Embeddingwith, for example,huggingface.ChatCompletion,huggingface.Completionorhuggingface.Embeddingby changing one line of code.Supported Clientshuggingface-HuggingFacemodelshuggingface.ChatCompletion- Chat with LLMshuggingface.Completion- Text completion with LLMshuggingface.Embedding- Create embeddings with LLMssagemaker- Open LLMs deployed on Amazon SageMakersagemaker.ChatCompletion- Chat with LLMssagemaker.Completion- Text completion with LLMssagemaker.Embedding- Create embeddings with LLMsbedrock- Amazon Bedrock LLMsCheck out theExamplesto get started.🚀 Getting StartedInstall EasyLLM via pip:pipinstalleasyllmThen import and start using the clients:fromeasyllm.clientsimporthuggingface# helper to build llama2 prompthuggingface.prompt_builder="llama2"response=huggingface.ChatCompletion.create(model="meta-llama/Llama-2-70b-chat-hf",messages=[{"role":"system","content":"\nYou are a helpful assistant speaking like a pirate. argh!"},{"role":"user","content":"What is the sun?"},],temperature=0.9,top_p=0.6,max_tokens=256,)print(response)the result will look like{"id":"hf-lVC2iTMkFJ","object":"chat.completion","created":1690661144,"model":"meta-llama/Llama-2-70b-chat-hf","choices":[{"index":0,"message":{"role":"assistant","content":" Arrrr, the sun be a big ol' ball o' fire in the sky, me hearty! It be the source o' light and warmth for our fair planet, and it be a mighty powerful force, savvy? Without the sun, we'd be sailin' through the darkness, lost and cold, so let's give a hearty \"Yarrr!\" for the sun, me hearties! Arrrr!"},"finish_reason":null}],"usage":{"prompt_tokens":111,"completion_tokens":299,"total_tokens":410}}Check out other examples:Detailed ChatCompletion ExampleExample how to stream chat requestsExample how to stream text requestsDetailed Completion ExampleCreate EmbeddingsSee thedocumentationfor more detailed usage and examples.💪🏻 Migration from OpenAI to HuggingFaceMigrating from OpenAI to HuggingFace is easy. Just change the import statement and the client you want to use and optionally the prompt builder.- import openai+ from easyllm.clients import huggingface+ huggingface.prompt_builder = "llama2"- response = openai.ChatCompletion.create(+ response = huggingface.ChatCompletion.create(-    model="gpt-3.5-turbo",+    model="meta-llama/Llama-2-70b-chat-hf",messages=[{"role": "system", "content": "You are a helpful assistant."},{"role": "user", "content": "Knock knock."},],
)Make sure when you switch your client that your hyperparameters are still valid. For example,temperatureof GPT-3 might be different thantemperatureofLlama-2.☑️ Key Features🤝 Compatible ClientsImplementation of clients compatible with OpenAI API format ofopenai.ChatCompletion,openai.Completion,openai.Embedding.Easily switch between different LLMs likeopenai.ChatCompletionandhuggingface.ChatCompletionby changing one line of code.Support for streaming of completions, checkout exampleHow to stream completions.⚙️ Helper Modules ⚙️evol_instruct(work in progress) - Use evolutionary algorithms create instructions for LLMs.prompt_utils- Helper methods to easily convert between prompt formats like OpenAI Messages to prompts for open source models like Llama 2.🙏 ContributingEasyLLM is an open source project and welcomes contributions of all kinds.The project useshatchfor development. To get started, fork the repository and clone
it to your local machine.Confirmhatchis installed (pipx is great to make it available globally on your machine)Once in the project directory, runhatch env createto create a default virtual environment for development.Activate the virtual environment withhatch shellStart developing! 🤩📔 Citation & AcknowledgementsIf you use EasyLLM, please share it with me on social media or email. I would love to hear about it!
You can also cite the project using the following BibTeX:@software{Philipp_Schmid_EasyLLM_2023,author={PhilippSchmid},license={Apache-2.0},month=juj,title={EasyLLM:StreamlinedToolsforLLMs},url={https://github.com/philschmid/easyllm},year={2023}}